DEA-C01 Data Ingestion and Transformation Practice Test 12 — 15 Questions

Question 1

A company ingests JSON logs into Amazon S3 using Kinesis Data Firehose. The logs contain a timestamp field, but the delivery to S3 is delayed by up to 15 minutes during peak hours. The business requires near-real-time availability (under 2 minutes). Which configuration change should the data engineer make?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Decrease the buffer interval to 60 seconds. Decreasing the buffer interval to 60 seconds forces Kinesis Data Firehose to deliver data to S3 more frequently, reducing the maximum delivery latency to under 2 minutes. The default buffer interval is 300 seconds (5 minutes), and by lowering it to 60 seconds, you ensure that data is flushed to S3 within one minute of ingestion, meeting the near-real-time requirement even during peak hours when buffering might otherwise delay delivery.

Answer

Increase the number of shards in the Kinesis Data Firehose stream

Answer

Increase the buffer size to 128 MB

Answer

Enable buffering hints in the Firehose delivery stream

Question 2

A data engineer runs a weekly AWS Glue ETL job that processes data from Amazon DynamoDB to Amazon S3. The job reads the entire table every time, which is slow and expensive. The job needs to process only items that changed since the last run. Which solution should the engineer implement?

Accepted Answer

Enable DynamoDB Streams and process change events with AWS Lambda to write to S3. DynamoDB Streams captures item-level changes (inserts, updates, deletes) in near real-time. An AWS Lambda function can process these change events and write only the incremental data to Amazon S3, eliminating the need to scan the entire DynamoDB table. This approach is both cost-effective and efficient for incremental data ingestion.

Answer

Use DynamoDB Scan with a LastEvaluatedKey to paginate and store the last scanned key to resume next time

Answer

Add a Global Secondary Index (GSI) on a timestamp attribute and query only new records

Answer

Use AWS Database Migration Service (DMS) with ongoing replication from DynamoDB to S3

Question 3

A company needs to ingest data from an external API that returns CSV files daily. The files range from 100 MB to 2 GB. The data should be landed in Amazon S3 and then transformed using AWS Glue. Which ingestion method is most cost-effective and requires the least operational overhead?

Accepted Answer

Schedule an AWS Lambda function to download the CSV file and upload it to Amazon S3. Scheduling an AWS Lambda function to download the CSV file from the external API and upload it to Amazon S3 is the most cost-effective and operationally lightweight approach. Lambda can handle files up to 2 GB (with appropriate memory and timeout settings) and runs on a serverless, pay-per-execution model, eliminating the need for infrastructure management. This method directly addresses the daily, batch-oriented nature of the data ingestion without requiring additional services or complex configurations.

Answer

Set up AWS DataSync to transfer the file from the API endpoint to S3

Answer

Use Amazon Kinesis Data Firehose with a direct PUT

Answer

Deploy an AWS Direct Connect connection to the external API for faster transfer

Question 4

A data engineer notices that an AWS Glue ETL job is running slower than expected. The job reads from Amazon S3, joins two datasets, and writes the result back to S3. The job uses the default worker type (G.1X) and 10 DPUs. Which action is most likely to improve performance?

Accepted Answer

Repartition the data before the join operation. The default G.1X worker type provides 16 GB of memory and 4 vCPUs per DPU. With 10 DPUs, the job likely has sufficient compute but suffers from data skew or inefficient partitioning during the join. Repartitioning the data before the join ensures that keys are evenly distributed across partitions, reducing shuffle overhead and preventing straggler tasks, which directly improves performance.

Answer

Increase the number of DPUs to 20

Answer

Use coalesce to reduce the number of output files

Answer

Change the worker type to G.2X

Question 5

A company ingests streaming data from social media feeds into Amazon Kinesis Data Streams. The data is consumed by an AWS Lambda function that transforms and writes to Amazon S3. Recently, the Lambda function started timing out and dropping records. The data volume has tripled. Which actions should the data engineer take to resolve this? (Choose TWO.)

Accepted Answer

Increase the number of shards in the Kinesis data stream. (increase shards) increases the Kinesis stream's throughput capacity to handle the tripled data volume, reducing backpressure on the Lambda consumer. Option E (increase Lambda memory) also increases CPU allocation, allowing the Lambda function to process each record faster, which helps prevent timeouts. Option B (replace Lambda with Firehose) could be an alternative but is not a direct fix for the existing Lambda-based architecture and may not be suitable for complex transformations. Option C (increase timeout to 15 minutes) might allow more time but does not address the underlying root cause of insufficient throughput or processing power. Option D (reserved concurrency) prevents other functions from affecting this function's concurrency but does not increase total processing capacity; it could even limit scaling if set too low.

Answer

Replace Lambda with Amazon Kinesis Data Firehose for the transformation

Answer

Increase the Lambda function timeout to 15 minutes

Answer

Set a reserved concurrency on the Lambda function

Question 6

A data engineer needs to design a data ingestion pipeline that captures streaming data from mobile app events into Amazon S3 for analytics. The pipeline must support real-time processing of events and allow for schema evolution over time. Which AWS services should the engineer use? (Choose THREE.)

Accepted Answer

Amazon Kinesis Data Analytics. Amazon Kinesis Data Analytics is correct because it enables real-time processing of streaming data using SQL or Apache Flink, allowing the engineer to analyze mobile app events as they arrive. This supports the requirement for real-time processing before the data is stored in Amazon S3 for analytics.

Answer

AWS Glue ETL jobs

Answer

AWS AppFlow

Question 7

A company uses AWS Glue to run ETL jobs daily. The data engineer wants to reduce costs by optimizing the job configuration. Which two actions will help reduce costs? (Choose TWO.)

Accepted Answer

Use G.1X worker type instead of G.2X. G.1X workers provide 16 GB of memory and 4 vCPUs, while G.2X workers provide 32 GB and 8 vCPUs. For many ETL jobs, especially those with smaller data volumes or less complex transformations, G.1X workers are sufficient, and using them instead of G.2X directly reduces the cost per DPU-hour since AWS Glue pricing is based on DPU capacity. This optimization lowers costs without sacrificing performance if the job is not memory- or CPU-bound.

Answer

Increase the job timeout to 48 hours

Answer

Enable Spark UI logging for debugging

Answer

Increase the number of job retries to handle transient failures

Question 8

A data engineer is designing a data transformation pipeline using AWS Glue. The source data is in Amazon S3 in Parquet format, and the transformed output must be written to another S3 bucket in Parquet format partitioned by year, month, day. The pipeline should handle incremental updates efficiently. Which three features should the engineer use? (Choose THREE.)

Accepted Answer

AWS Glue job bookmarks to track processed data. AWS Glue job bookmarks track processed data by recording the state of previously processed files and partitions, enabling incremental processing of new or changed data in subsequent runs. This is essential for efficiently handling incremental updates without reprocessing the entire dataset.

Answer

Use AWS Glue JobWatch for monitoring job progress

Answer

Use Spark SQL for transformations

Question 9

Refer to the exhibit. A data engineer is troubleshooting an AWS Glue ETL job that fails with an access denied error when writing to S3. The IAM role attached to the Glue job has the policy shown. What is the most likely cause of the error?

Accepted Answer

The Glue job is writing to an S3 bucket that is not included in the Resource ARN. The IAM policy shown in the exhibit explicitly restricts the `s3:PutObject` action to a specific ARN pattern (e.g., `arn:aws:s3:::example-bucket/*`). If the Glue job attempts to write to a different S3 bucket—one not matching that ARN—the request will be denied with an access denied error. AWS Glue ETL jobs use the attached IAM role's permissions, and a mismatch between the target bucket and the resource ARN in the policy is the most common cause of such failures.

Answer

The Glue job is missing the s3:ListBucket permission on the bucket

Answer

The S3 bucket is encrypted with AWS KMS and the policy does not include kms:Decrypt permissions

Answer

The Glue job does not have permission to call glue:StartJobRun

Question 10

Refer to the exhibit. A data engineer runs an AWS Glue job that fails with an 'Access Denied' error when writing to S3. The IAM role attached to the job has s3:PutObject permission on the output bucket. What additional configuration is most likely missing?

Accepted Answer

The S3 bucket policy does not grant access to the Glue job's IAM role. The IAM role attached to the Glue job has s3:PutObject permission, but the S3 bucket policy must explicitly grant access to that IAM role (or its principal) for the write operation to succeed. Even if the IAM role allows the action, the bucket policy acts as a separate access control layer; if it denies or does not include the role, the request fails with 'Access Denied'. This is a common cross-account or service-specific permission issue where both identity-based and resource-based policies must align.

Answer

The Glue job is not configured to write to S3 with the correct prefix

Answer

The Glue job is running in a VPC without an S3 VPC endpoint

Answer

The S3 bucket is encrypted with AWS KMS and the IAM role lacks kms:Decrypt permission

Question 11

Refer to the exhibit. A data engineer is troubleshooting a Kinesis Data Streams consumer that is falling behind. The stream has 2 shards and is receiving data at a rate of 2 MB/s. The consumer is an AWS Lambda function with a batch size of 100 records. What should the engineer do to improve consumer throughput?

Accepted Answer

Increase the number of shards in the stream to 4. Increasing the number of shards to 4 doubles the stream's total ingestion capacity to 4 MB/s, which directly increases the number of concurrent Lambda invocations and thus consumer throughput. With 2 shards, each shard can support up to 1 MB/s input and 2 MB/s output, so the current 2 MB/s load is at the shard-level output limit, causing the consumer to fall behind.

Answer

Decrease the Lambda batch size to 10 records

Answer

Increase the retention period of the stream to 168 hours

Answer

Increase the memory allocation of the Lambda function

Question 12

A company uses AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job fails intermittently with a 'MemoryError'. What is the MOST likely cause?

Accepted Answer

The Glue job worker type is too small for the data volume. The 'MemoryError' in AWS Glue indicates that the worker type allocated to the job does not have sufficient memory to process the data volume. Glue workers (Standard, G.1X, G.2X) have fixed memory allocations (e.g., 16 GB for Standard), and if the streaming data from Kinesis exceeds this, the job fails. Increasing the worker type or the number of workers resolves this.

Answer

The Glue job uses too many DynamicFrames

Answer

The S3 output bucket is in a different region

Answer

The Kinesis stream has insufficient shards

Question 13

A data pipeline uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The delivery occasionally fails with 'Firehose is throttled'. What should be done to reduce throttling?

Accepted Answer

Increase the buffer size and buffer interval. Increasing the buffer size and buffer interval gives Kinesis Data Firehose more time and data volume to accumulate before delivering to S3, reducing the frequency of PutRecord.Batch calls to the underlying Kinesis stream. This directly mitigates throttling by lowering the request rate, as Firehose throttling typically occurs when the per-shard write throughput limit (1,000 records/second or 1 MB/second) is exceeded.

Answer

Enable compression on the Firehose delivery stream

Answer

Decrease the buffer size to flush more frequently

Answer

Increase the number of shards in the Kinesis stream

Question 14

A company uses AWS DMS to migrate a 2 TB Oracle database to Amazon RDS for PostgreSQL. The migration is taking longer than expected. The task status shows 'Full load in progress' with a low 'Table throughput (rows/s)'. Which action would MOST improve throughput?

Accepted Answer

Increase the number of parallel tasks in the DMS task settings. The low 'Table throughput (rows/s)' during the full load phase indicates that the DMS task is not processing tables with enough parallelism. Increasing the number of parallel tasks in the DMS task settings allows the replication instance to load multiple tables concurrently, which directly improves throughput by utilizing available CPU and memory resources more efficiently.

Answer

Enable Multi-AZ on the DMS replication instance

Answer

Change the target table preparation mode to 'Do nothing'

Answer

Increase the number of shards in the source database

Question 15

A data engineer is building a pipeline to ingest JSON files from Amazon S3 into Amazon Redshift. The files are 100 MB each and arrive every 5 minutes. Which service is BEST suited for this ingestion?

Accepted Answer

Amazon Redshift COPY command. The Amazon Redshift COPY command is the most efficient and best-suited service for bulk loading 100 MB JSON files from S3 into Redshift at regular 5-minute intervals. It is optimized for high-throughput, parallel ingestion directly from S3, minimizing latency and resource overhead compared to other services.

Answer

AWS Glue ETL job

Answer

AWS Lambda with Redshift Data API

Answer

Amazon Kinesis Data Firehose with Redshift destination