DEA-C01 Data Ingestion and Transformation Practice Test 40 — 15 Questions

Question 1

An IAM policy includes the above resource ARN for CloudWatch Logs. A data engineer needs to allow a Lambda function to create log streams and put logs to the log group 'my-log-group'. However, the Lambda function is failing with access denied. What is the issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The ARN does not include the log-stream portion.. The ARN provided in the policy is for the log group itself (arn:aws:logs:region:account-id:log-group:my-log-group), but CloudWatch Logs requires separate permissions for the log-stream subresource. To allow a Lambda function to create log streams and put logs, the ARN must include the log-stream portion, typically with a wildcard like arn:aws:logs:region:account-id:log-group:my-log-group:log-stream:*. Without this, the IAM policy does not grant access to the log-stream actions (CreateLogStream, PutLogEvents), causing an access denied error even if the log group ARN is correct.

Answer

The region in the ARN does not match the Lambda function's region.

Answer

The Lambda function does not have an execution role.

Answer

The ARN is incorrectly formatted because of the wildcard.

Question 2

A company wants to ingest streaming data from thousands of IoT devices into Amazon S3 with minimal latency and then transform the data using Spark SQL. Which AWS service should be used for data ingestion?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed for ingesting streaming data into Amazon S3 with near-real-time latency (typically 60 seconds or less). It can directly write data to S3 without requiring custom code or additional infrastructure, and it supports optional transformations via AWS Lambda, making it ideal for the described use case of streaming IoT data ingestion.

Answer

Amazon EMR

Answer

AWS Glue

Answer

Amazon Athena

Question 3

A data engineering team is troubleshooting a slow AWS Glue ETL job that reads from an Amazon DynamoDB table and writes to Amazon S3 in Parquet format. The job processes 50 GB of data. Which action would most effectively improve job performance?

Accepted Answer

Increase the number of DPUs. Increasing the number of DPUs (Data Processing Units) for the AWS Glue job directly allocates more distributed computing resources (CPU, memory, and network bandwidth) to parallelize the read from DynamoDB and the write to S3. Since the job processes 50 GB of data, the bottleneck is likely the throughput of the Glue Spark cluster, and adding DPUs increases parallelism, reducing overall execution time.

Answer

Use S3 Select to push down filters

Answer

Reduce the batch size in the DynamoDB connector

Answer

Change output to JSON format to reduce overhead

Question 4

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data is consumed by an AWS Lambda function that writes to Amazon DynamoDB. The Lambda function is seeing high error rates due to DynamoDB write throttling. Which action should be taken to reduce throttling?

Accepted Answer

Enable auto scaling on the DynamoDB table. Enabling DynamoDB auto scaling increases write capacity automatically when needed. Using Kinesis Data Firehose would change the architecture but does not address throttling directly. Increasing Lambda memory does not help with DynamoDB throttling. Using SQS would add a queue but does not increase DynamoDB capacity.

Answer

Use Amazon Kinesis Data Firehose instead of Kinesis Data Streams

Answer

Add an Amazon SQS queue between Lambda and DynamoDB

Answer

Increase the Lambda function memory

Question 5

A data pipeline uses AWS Glue to read from an Amazon S3 bucket containing millions of small CSV files (each < 1 MB). The ETL job is slow. Which optimization would most improve performance?

Accepted Answer

Use S3 file grouping to combine small files. AWS Glue (and Spark generally) performs poorly with millions of tiny files because each file requires a separate S3 GET request, metadata operation, and task scheduling overhead. S3 file grouping (via 'groupFiles' and 'groupSize' job parameters) coalesces multiple small files into a single read per Spark partition, drastically reducing the number of S3 operations and task overhead. This directly addresses the root cause of the slowness.

Answer

Write the ETL script using PySpark instead of Scala

Answer

Increase the number of Glue workers

Answer

Use the G.1X worker type for more memory

Question 6

A company wants to transform data in Amazon S3 using SQL queries without provisioning servers. The transformations are ad-hoc and run occasionally. Which service should be used?

Accepted Answer

Amazon Athena. Amazon Athena is the correct choice because it enables serverless, ad-hoc SQL querying directly on data stored in Amazon S3 without requiring any infrastructure provisioning. Since the transformations are occasional and ad-hoc, Athena's pay-per-query model and zero setup overhead align perfectly with the requirement.

Answer

AWS Glue

Answer

Amazon Redshift Spectrum

Answer

Amazon EMR

Question 7

A company has a nightly batch job that processes 100 GB of data from an Amazon S3 bucket and loads it into an Amazon Redshift table. The job currently runs on an Amazon EMR cluster. Which service would reduce operational overhead while providing similar functionality?

Accepted Answer

AWS Glue. AWS Glue is a serverless ETL service that can process 100 GB of data from S3 and load it into Redshift without managing any infrastructure. It provides built-in job scheduling, automatic retries, and a Spark-based engine that handles large-scale data transformations, directly replacing the EMR cluster's functionality while eliminating operational overhead.

Answer

AWS Database Migration Service

Answer

Amazon Redshift Spectrum

Answer

Amazon Athena

Question 8

Which TWO AWS services can be used to ingest streaming data from a mobile application into Amazon S3 for near-real-time analytics? (Choose 2.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to load streaming data directly into Amazon S3, Redshift, Elasticsearch, or Splunk without requiring custom code. It can capture and transform streaming data from mobile applications in near-real-time and automatically deliver it to S3, making it ideal for near-real-time analytics pipelines.

Answer

Amazon DynamoDB Streams

Answer

AWS Glue

Answer

Amazon SQS

Question 9

The exhibit shows an AWS CLI command and its output. A data engineer wants to copy only objects larger than 10 MB from the S3 bucket to another bucket for processing. Which approach should be used to automate this task?

Accepted Answer

Use AWS CLI with a script to filter and copy objects. The command lists objects larger than 10 MB. To automate copying, a script using AWS CLI with the --query parameter can filter and copy. Using S3 Batch Operations allows performing actions on a list of objects. The correct approach is to use AWS CLI with a script that iterates over the filtered list and uses aws s3 cp. S3 replication is for continuous sync, not one-time copy. Lambda with S3 events triggers only on new objects, not existing ones. S3 Inventory provides metadata but not direct copy.

Answer

Use S3 replication rules to replicate objects above 10 MB

Answer

Use S3 Inventory to generate a list and then copy

Answer

Use AWS Lambda with S3 event notifications

Question 10

The exhibit shows an IAM policy attached to a role used by an AWS Glue ETL job. The job reads from an S3 bucket and writes to another S3 bucket. However, the job fails with an access denied error when trying to write to the output bucket. What is the most likely cause?

Accepted Answer

The policy only allows PutObject on the my-data-lake bucket, but the job writes to a different bucket. The IAM policy attached to the Glue job's role grants s3:PutObject only on the my-data-lake bucket ARN, but the ETL job's output path points to a different bucket. S3 authorization is evaluated per-resource ARN, so a PutObject call to any bucket not listed in the Resource element is implicitly denied. The fix is to add the output bucket's ARN (or a wildcard) to the policy's Resource list.

Answer

The policy is missing permissions for AWS KMS to decrypt/encrypt objects

Answer

The policy does not allow glue:StartJobRun on the specific job

Answer

The policy does not include s3:ListBucket permission

Question 11

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The data is then processed by a Kinesis Data Analytics application running SQL queries. The analytics application is falling behind and processing records with increasing latency. The stream has 4 shards, and the average record size is 5 KB. What is the MOST effective way to improve processing latency?

Accepted Answer

Increase the parallelism of the Kinesis Data Analytics application.. Increasing the parallelism of the Kinesis Data Analytics application (e.g., by increasing the number of in-application streams or ParallelismPerKPU) allows it to consume from the stream faster, reducing latency. Option A is wrong because 5 KB is well below the 1 MB/s shard limit, so increasing shards is unnecessary. Option B is wrong because enhanced fan-out is for consumers that need low latency, but does not help if the application is CPU-bound. Option D is wrong because increasing the retention period does not affect processing speed; it only keeps data longer.

Answer

Increase the number of shards in the Kinesis stream to 8.

Answer

Enable enhanced fan-out on the Kinesis stream for the analytics application.

Answer

Increase the retention period of the Kinesis stream to 7 days.

Question 12

A data engineer needs to ingest data from a relational database (MySQL) into Amazon S3 for analytics. The database is 500 GB and the job must run daily with incremental updates. Which AWS service is BEST suited for this task?

Accepted Answer

AWS Database Migration Service (DMS) with a replication task.. AWS DMS with a replication task is the best choice because it is specifically designed for continuous, incremental data replication from relational databases like MySQL to Amazon S3. DMS supports ongoing replication (change data capture) to capture incremental changes without custom scripting, and it can handle the initial 500 GB load efficiently. Other services either lack native incremental support or require additional configuration for this use case.

Answer

Amazon EMR with Apache Sqoop.

Answer

Amazon Kinesis Data Firehose with a database source.

Answer

AWS Glue ETL job with a JDBC connection.

Question 13

A company uses AWS Lambda to process messages from an Amazon SQS queue. The messages contain JSON payloads that need to be transformed and written to an Amazon DynamoDB table. Recently, the Lambda function has been timing out and messages are being sent to the dead-letter queue (DLQ). What is the BEST way to troubleshoot and resolve this issue?

Accepted Answer

Increase the Lambda function timeout and monitor DynamoDB write capacity to ensure it is not throttling.. The Lambda function is timing out, which suggests the function's execution duration is exceeding its configured timeout. Increasing the timeout gives the function more time to process messages. Additionally, if DynamoDB write capacity is insufficient, throttling can cause retries that further delay processing, so monitoring and possibly increasing write capacity units addresses the root cause of timeouts.

Answer

Use a standard SQS queue instead of a DLQ to reprocess failed messages automatically.

Answer

Switch the SQS queue to a FIFO queue to ensure exactly-once processing.

Answer

Increase the visibility timeout of the SQS queue to 30 minutes.

Question 14

A company wants to ingest streaming data from IoT devices into Amazon S3 using Amazon Kinesis Data Firehose. The data must be transformed from JSON to Parquet format before landing in S3. What is the SIMPLEST way to achieve this?

Accepted Answer

Configure Kinesis Data Firehose with a built-in Parquet converter.. Amazon Kinesis Data Firehose has a built-in Parquet conversion feature that uses an AWS Glue schema to convert incoming JSON data to Parquet format. This is the simplest approach because it requires no custom code or additional services; you only need to provide a schema and enable the conversion in the Firehose delivery stream configuration. Option B (using Lambda) is more complex, as it requires writing and maintaining a custom transformation function.

Answer

Use an AWS Lambda function as a data transformation in Kinesis Data Firehose to convert JSON to Parquet.

Answer

Use Kinesis Data Firehose to deliver data directly to S3 in JSON format and run a nightly Glue job to convert to Parquet.

Answer

Use Kinesis Data Analytics to convert the data to Parquet before sending to Firehose.

Question 15

A data engineer is designing a pipeline to ingest change data capture (CDC) events from an Amazon RDS for PostgreSQL database into Amazon S3. The CDC events are captured using AWS DMS. The data must be available for querying within 5 minutes of the change. Which approach meets these requirements?

Accepted Answer

Use AWS DMS to replicate data directly to S3 in near real-time.. AWS DMS supports continuous replication (change data capture) directly to Amazon S3 as a target endpoint. DMS can write CDC events to S3 in near real-time (typically seconds to minutes), meeting the 5-minute latency requirement without intermediate services. The data is stored in comma-separated value (CSV) or Parquet format, ready for querying via Athena or Glue.

Answer

Export the database to S3 using pg_dump and then use AWS Glue to load into S3 in Parquet format.

Answer

Use AWS DMS to replicate data to Amazon Redshift, then unload to S3.

Answer

Use AWS DMS to replicate data to an SQS queue, then process with Lambda to write to S3.