DEA-C01 Data Ingestion and Transformation Practice Test 38 — 15 Questions

Question 1

A data engineer is building a data ingestion pipeline using AWS Glue. The source is an Amazon DynamoDB table, and the target is an Amazon S3 data lake in Parquet format. The pipeline must handle large volumes and ensure exactly-once processing. Which THREE features should the engineer use together to achieve this? (Choose THREE.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure the Glue job to convert data to Parquet format.. Converting data to Parquet format is a core requirement for an S3 data lake, as Parquet offers columnar storage, compression, and efficient querying via services like Amazon Athena and Amazon Redshift Spectrum. AWS Glue natively supports Parquet as an output format, enabling the engineer to specify it in the job's output schema or transformation logic.

Answer

Use Amazon Kinesis Data Streams to capture DynamoDB Streams changes.

Answer

Use Amazon S3 Object Lambda to transform data on the fly.

Question 2

A healthcare company processes patient records in near-real-time using Amazon Kinesis Data Streams. Each record contains sensitive personal health information (PHI). The data must be encrypted at rest and in transit. The company also needs to audit access to the data. The data engineer is designing the ingestion pipeline. Which combination of services and configurations meets these requirements?

Accepted Answer

Use Kinesis Data Streams with SSE-KMS and TLS, and enable CloudTrail for data events.. Kinesis Data Streams supports server-side encryption with AWS KMS (SSE-KMS) for encryption at rest, and TLS is used for encryption in transit by default. Enabling CloudTrail data events captures API-level access to Kinesis streams (e.g., PutRecord, GetRecords), satisfying the audit requirement. This combination meets all three requirements: encryption at rest, encryption in transit, and auditable access.

Answer

Use Kinesis Data Firehose to deliver data to S3 with SSE-S3, and enable CloudTrail for S3.

Answer

Use Kinesis Data Streams with TLS and enable CloudTrail for auditing. Do not enable SSE.

Answer

Use Kinesis Data Streams with SSE-KMS and TLS. Do not enable any auditing.

Question 3

A social media company ingests user activity data from multiple sources using Amazon Kinesis Data Firehose. The data is delivered to Amazon S3 in near-real-time. The company wants to transform the data by adding a timestamp and masking email addresses before storing it in S3. The transformation should be applied to all records. What is the most cost-effective way to implement this transformation?

Accepted Answer

Configure the Firehose delivery stream to invoke a Lambda function for data transformation.. Amazon Kinesis Data Firehose supports invoking an AWS Lambda function for inline data transformation before delivering data to the destination. This allows adding timestamps and masking email addresses in near-real-time as data flows through the delivery stream. It is the most cost-effective and operationally efficient way because it avoids additional storage, batch processing, or separate compute resources.

Answer

Use Amazon Athena to run a CTAS query that transforms the data and writes to a new location.

Answer

Use AWS Glue to schedule a batch job every 5 minutes to transform the data.

Answer

Use Amazon S3 Events to trigger a Lambda function whenever a new object is created.

Question 4

A retail company uses Amazon Kinesis Data Firehose to ingest clickstream data from its website into an Amazon S3 bucket. The data includes fields: user_id, event_type, timestamp, page_url. Recently, the data engineering team noticed that some records have malformed JSON (missing commas, extra brackets) causing delivery failures to S3. The Firehose delivery stream is configured to retry failed records for 300 seconds, after which the records are sent to an S3 bucket for failed records. The team wants to transform the data to correct malformed JSON before delivery to the main S3 bucket. They need a solution that does not require managing servers and can handle high throughput. What should the team do?

Accepted Answer

Configure an AWS Lambda function as a data transformation in Kinesis Data Firehose to correct malformed JSON.. Kinesis Data Firehose supports Lambda-based data transformation natively: you attach a Lambda function to the delivery stream, and Firehose invokes it on each batch before delivery. This is serverless, scales with throughput, and lets you parse and repair malformed JSON before it reaches the main S3 bucket.

Answer

Set up an Amazon EMR cluster with Apache Spark to process the data in micro-batches and fix JSON errors.

Answer

Use an AWS Glue streaming ETL job to read from Firehose and write corrected data to S3.

Answer

Use Amazon Kinesis Data Analytics with a SQL application to parse and fix JSON.

Question 5

A marketing analytics team needs to ingest customer transaction data from an on-premises PostgreSQL database into Amazon S3 for analysis. The data volume is about 10 GB daily, and the team wants to perform full refresh daily (truncate and load) into S3 as Parquet files. The company has a Direct Connect connection to AWS. The team needs a simple, managed solution that minimizes operational overhead. What should the team use?

Accepted Answer

Use an AWS Glue ETL job with a JDBC connection to the PostgreSQL database, extract data, and write to S3 in Parquet format.. AWS Glue ETL jobs provide a serverless, managed environment to extract data from JDBC sources like PostgreSQL, transform it, and write to S3 in Parquet format. It minimizes operational overhead as it handles provisioning, scaling, and job execution. For a daily full refresh, a Glue job can be scheduled to truncate and load data.

Answer

Set up AWS Database Migration Service (DMS) to continuously replicate data to S3 in Parquet format.

Answer

Use Amazon EMR with a Spark job that reads from PostgreSQL and writes to S3.

Answer

Use AWS Data Pipeline with a SQLActivity to extract data and copy to S3.

Question 6

A healthcare company is building a data pipeline to ingest electronic health records (EHR) from hospitals. The data is sent as JSON files via SFTP to an on-premises server. The company wants to move this data to AWS using AWS Transfer Family (SFTP) and then process it with AWS Glue. Data sovereignty regulations require that all data remain within the EU (Frankfurt) region. The pipeline must detect when a new file arrives and start the Glue job automatically. The engineer has set up an AWS Transfer Family server in Frankfurt, and files are uploaded to an S3 bucket in the same region. However, the Glue job is not triggering automatically. The engineer needs to implement automated triggering. What should the engineer do?

Accepted Answer

Set up an S3 event notification on the bucket to invoke an AWS Lambda function that starts the Glue job.. S3 event notifications can be configured to invoke an AWS Lambda function when objects are created, and the Lambda function can call the Glue StartJobRun API to launch the job. This provides event-driven, near-real-time triggering without polling, and keeps all processing within the Frankfurt region to satisfy data sovereignty requirements.

Answer

Configure AWS Step Functions to poll the S3 bucket every minute and start the Glue job if new files exist.

Answer

Configure Amazon CloudWatch Events to trigger the Glue job on a schedule that checks for new files.

Answer

Use Amazon Simple Queue Service (SQS) to queue file metadata and have a Lambda function poll the queue to start the Glue job.

Question 7

A data engineer needs to ingest log files from multiple EC2 instances into Amazon S3. The logs are written to local disk on each instance. The engineer wants a simple agent-based solution that can collect, compress, and upload logs to S3 with minimal configuration. The solution must support incremental uploads (only new log lines) and handle log rotation. What should the engineer use?

Accepted Answer

Install and configure Amazon CloudWatch Agent to collect logs and send them to Amazon CloudWatch Logs, then use a subscription filter to export logs to S3.. Amazon CloudWatch Agent is a lightweight agent that can tail log files, compress them on the fly, and send them to CloudWatch Logs. From CloudWatch Logs, a subscription filter can export the logs to Amazon S3, supporting incremental uploads and log rotation. Option B (AWS CLI cp) is manual and does not handle incremental uploads efficiently. Option C (AWS DataSync) is designed for bulk data transfers, not real-time log ingestion. Option D (S3 sync) is also not real-time and lacks agent-based tailing and compression.

Answer

Use AWS CLI cp command with --recursive in a cron job to copy logs to S3 every minute.

Answer

Install AWS DataSync agent on each EC2 instance to sync logs to S3 daily.

Answer

Use an S3 sync command from the AWS CLI scheduled every hour.

Question 8

A data pipeline ingests daily CSV files from an FTP server into an Amazon S3 bucket. The files must be converted to Parquet format and partitioned by date for efficient querying using Amazon Athena. Which AWS service is most suitable for this transformation?

Accepted Answer

AWS Glue. AWS Glue is the most suitable service because it provides a fully managed ETL (Extract, Transform, Load) capability that can natively read CSV files from S3, convert them to Parquet format, and write the output partitioned by date. Glue's built-in transform 'ConvertToParquet' and dynamic frame partitioning make this a straightforward, serverless solution without needing to manage infrastructure.

Answer

Amazon Kinesis Data Firehose

Answer

Amazon EMR

Answer

AWS Lambda

Question 9

A company wants to ingest real-time clickstream data from a website into Amazon S3 with a maximum latency of 60 seconds. The data volume peaks at 500 MB/s. Which service should they use to buffer and deliver the data to S3?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is designed to ingest streaming data, buffer it, and deliver it to destinations like Amazon S3 with configurable buffer intervals (e.g., 60 seconds) and buffer sizes (e.g., up to 128 MB). It can handle the peak throughput of 500 MB/s by automatically scaling, and it meets the maximum latency requirement of 60 seconds by flushing data to S3 based on time or size thresholds.

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon Kinesis Data Streams

Answer

AWS Lambda

Question 10

A data engineer is designing a pipeline that ingests JSON logs from an application into Amazon S3. The logs contain a timestamp field. The pipeline must partition the data by date in S3 (e.g., year=2024/month=10/day=01). Which approach minimizes transformation effort?

Accepted Answer

Use Amazon Kinesis Data Firehose with dynamic partitioning. Amazon Kinesis Data Firehose with dynamic partitioning can automatically partition incoming JSON data based on the timestamp field without requiring custom transformation code. It evaluates the timestamp using a JQ expression or inline parsing, then writes records directly to S3 prefixes like year=2024/month=10/day=01. This minimizes transformation effort because the partitioning logic is configured declaratively in the Firehose delivery stream, eliminating the need for Lambda functions or post-ingestion processing.

Answer

Use AWS Glue crawlers to infer schema and create partitions

Answer

Use AWS Lambda to process each object and copy to the appropriate prefix

Answer

Use Amazon Athena to create partitions on the existing data

Question 11

A company is using AWS DMS to replicate data from an on-premises Oracle database to Amazon RDS for MySQL. The replication is working, but the target table has a different schema. Which DMS feature should be used to transform the source schema to match the target?

Accepted Answer

Use DMS transformation rules. AWS DMS transformation rules allow you to modify the schema, table, or column names and data types during the migration process. This feature is specifically designed to handle schema transformations within the DMS task itself, enabling you to map the source Oracle schema to the target MySQL schema without external tools or services.

Answer

Use AWS Schema Conversion Tool (SCT)

Answer

Use AWS Glue ETL jobs

Answer

Use AWS Lambda triggers

Question 12

A data engineer needs to ingest data from an Amazon S3 bucket into Amazon Redshift for analytics. The data is in CSV format and the Redshift table already exists. Which service can be used to perform this ingestion with minimal configuration?

Accepted Answer

Amazon Redshift COPY command. The Amazon Redshift COPY command is the most direct and minimal-configuration method to load data from an S3 bucket into an existing Redshift table. It is purpose-built for bulk data ingestion from S3, supports CSV format natively, and requires only the table name, S3 path, IAM role, and format options — no additional services or pipelines needed.

Answer

AWS Glue

Answer

Amazon Kinesis Data Firehose

Answer

AWS Database Migration Service (DMS)

Question 13

A streaming application sends data to Amazon Kinesis Data Streams. The data must be enriched with reference data from an Amazon DynamoDB table in real-time. Which AWS service can be used to perform this enrichment with minimal latency?

Accepted Answer

Amazon Kinesis Data Analytics for Apache Flink. Amazon Kinesis Data Analytics for Apache Flink is correct because it allows you to run Apache Flink applications that can read from a Kinesis data stream, perform stateful stream processing, and enrich records in real-time by joining with reference data stored in DynamoDB. Flink's asynchronous I/O and managed state enable sub-second enrichment latency without the cold-start delays or concurrency limits of Lambda-based approaches.

Answer

Amazon Kinesis Data Firehose with Lambda transformation

Answer

AWS Lambda function triggered by Kinesis Data Streams

Answer

AWS Glue streaming ETL

Question 14

Refer to the exhibit. A data engineer runs an AWS Glue ETL job that writes output to an S3 bucket. The job fails with the error shown. What is the most likely cause?

Accepted Answer

The IAM role used by the Glue job lacks the s3:PutObject permission for the output bucket. The error shown in the exhibit indicates an access denied or permission failure when the AWS Glue ETL job attempts to write its output to the S3 bucket. The most likely cause is that the IAM role assigned to the Glue job does not include the s3:PutObject permission for the target bucket, which is required to upload objects. Without this permission, the job cannot complete the write operation, resulting in the failure.

Answer

The Glue job attempted to write data in an unsupported format

Answer

The S3 bucket does not exist

Answer

The output file name contains invalid characters

Question 15

Refer to the exhibit. A data engineer is troubleshooting an AWS Lambda function that reads from an S3 bucket and writes to a Kinesis Data Stream. The Lambda function fails with an AccessDeniedException when calling the kinesis:PutRecords API. Which change is needed to the IAM policy?

Accepted Answer

Change the resource ARN for Kinesis to include the correct stream name. The Lambda function's IAM policy grants kinesis:PutRecords permission but the resource ARN does not match the target Kinesis Data Stream. The AccessDeniedException occurs because IAM evaluates the resource ARN against the stream's ARN and denies access on mismatch. The correct fix is to change the resource ARN to include the correct stream name (option C). While using a wildcard (option B) would also resolve the error, it is not the recommended approach because it violates the principle of least privilege. Option A is incorrect because the error is from Kinesis, not S3. Option D is incorrect because the kinesis:PutRecords action is already present in the policy.

Answer

Add s3:PutObject permission to the policy

Answer

Change the resource ARN for Kinesis to a wildcard

Answer

Add kinesis:PutRecords permission to the policy