DEA-C01 Data Ingestion and Transformation Practice Test 11 — 15 Questions

Question 1

A data engineer runs a weekly AWS Glue ETL job that processes data from Amazon DynamoDB to Amazon S3. The job reads the entire table every time, which is slow and expensive. The job needs to process only items that changed since the last run. Which solution should the engineer implement?

Accepted Answer

Enable DynamoDB Streams and process change events with AWS Lambda to write to S3. Option D is correct because DynamoDB Streams captures changes (inserts, updates, deletes) and can be read by AWS Lambda to write to S3, enabling incremental processing. Option A (Scan with LastEvaluatedKey) still scans entire table. Option B (GSI) does not capture changes automatically. Option C (DMS) is overkill and not as seamless with DynamoDB Streams.

Answer

Use DynamoDB Scan with a LastEvaluatedKey to paginate and store the last scanned key to resume next time

Answer

Add a Global Secondary Index (GSI) on a timestamp attribute and query only new records

Answer

Use AWS Database Migration Service (DMS) with ongoing replication from DynamoDB to S3

Question 2

A company needs to ingest data from an external API that returns CSV files daily. The files range from 100 MB to 2 GB. The data should be landed in Amazon S3 and then transformed using AWS Glue. Which ingestion method is most cost-effective and requires the least operational overhead?

Accepted Answer

Schedule an AWS Lambda function to download the CSV file and upload it to Amazon S3. Option A is correct because AWS Lambda with a Python script can download the CSV files and upload to S3, and it is cost-effective for periodic small-to-medium files. Option B (Kinesis Data Firehose) is for streaming. Option C (DataSync) is for large-scale data transfers, not API pulling. Option D (Direct Connect) is a network connection, not an ingestion method.

Answer

Set up AWS DataSync to transfer the file from the API endpoint to S3

Answer

Use Amazon Kinesis Data Firehose with a direct PUT

Answer

Deploy an AWS Direct Connect connection to the external API for faster transfer

Question 3

A data engineer notices that an AWS Glue ETL job is running slower than expected. The job reads from Amazon S3, joins two datasets, and writes the result back to S3. The job uses the default worker type (G.1X) and 10 DPUs. Which action is most likely to improve performance?

Accepted Answer

Repartition the data before the join operation. Option B is correct because increasing the number of partitions (via repartition) can improve parallelism and performance when the data is skewed. Option A (increasing DPUs) may help but is not the most targeted fix. Option C (using coalesce) reduces partitions, which can hurt performance. Option D (changing to G.2X) provides more memory per worker but may not address the core issue of partition skew.

Answer

Increase the number of DPUs to 20

Answer

Use coalesce to reduce the number of output files

Answer

Change the worker type to G.2X

Question 4

A company ingests streaming data from social media feeds into Amazon Kinesis Data Streams. The data is consumed by an AWS Lambda function that transforms and writes to Amazon S3. Recently, the Lambda function started timing out and dropping records. The data volume has tripled. Which actions should the data engineer take to resolve this? (Choose TWO.)

Accepted Answer

Increase the number of shards in the Kinesis data stream. Options A and D are correct because increasing the number of shards increases throughput, and increasing Lambda memory can improve processing speed and reduce timeouts. Option B (Lambda concurrency) may cause throttling. Option C (Lambda timeout) may not be sufficient without more resources. Option E (Data Firehose) is a different service; replacing Lambda with Firehose could be an alternative but not a direct fix for current architecture.

Answer

Replace Lambda with Amazon Kinesis Data Firehose for the transformation

Answer

Increase the Lambda function timeout to 15 minutes

Answer

Set a reserved concurrency on the Lambda function

Question 5

A data engineer needs to design a data ingestion pipeline that captures streaming data from mobile app events into Amazon S3 for analytics. The pipeline must support real-time processing of events and allow for schema evolution over time. Which AWS services should the engineer use? (Choose THREE.)

Accepted Answer

Amazon Kinesis Data Analytics. Options A, C, and D are correct. Kinesis Data Streams ingests streaming data. Kinesis Data Analytics can perform real-time processing. Kinesis Data Firehose delivers data to S3. Option B (Glue) is batch-oriented and not real-time. Option E (AppFlow) is for SaaS data ingestion.

Answer

AWS Glue ETL jobs

Answer

AWS AppFlow

Question 6

A company uses AWS Glue to run ETL jobs daily. The data engineer wants to reduce costs by optimizing the job configuration. Which two actions will help reduce costs? (Choose TWO.)

Accepted Answer

Use G.1X worker type instead of G.2X. Options A and D are correct. Using a smaller DPU count reduces compute cost. Using G.1X worker type (default) is cost-effective; using G.2X would increase cost. Option B (increasing retries) may increase cost. Option C (increasing timeout) has no cost impact. Option E (Spark UI logging) adds cost.

Answer

Increase the job timeout to 48 hours

Answer

Enable Spark UI logging for debugging

Answer

Increase the number of job retries to handle transient failures

Question 7

A data engineer is designing a data transformation pipeline using AWS Glue. The source data is in Amazon S3 in Parquet format, and the transformed output must be written to another S3 bucket in Parquet format partitioned by year, month, day. The pipeline should handle incremental updates efficiently. Which three features should the engineer use? (Choose THREE.)

Accepted Answer

AWS Glue job bookmarks to track processed data. Options A, B, and D are correct. Bookmarks track processed data for incremental jobs. DynamicFrames are flexible for schema evolution. Partition pruning helps read only relevant partitions. Option C (Spark SQL) is not specific to incremental. Option E (JobWatch) is not an AWS Glue feature.

Answer

Use AWS Glue JobWatch for monitoring job progress

Answer

Use Spark SQL for transformations

Question 8

Refer to the exhibit. A data engineer is troubleshooting an AWS Glue ETL job that fails with an access denied error when writing to S3. The IAM role attached to the Glue job has the policy shown. What is the most likely cause of the error?

Accepted Answer

The Glue job is writing to an S3 bucket that is not included in the Resource ARN. Option B is correct because the policy only allows s3:PutObject on s3://my-data-bucket/*, but if the job writes to a different bucket or a path that doesn't exist, it may fail. Option A (KMS) is plausible but not indicated. Option C (Glue StartJobRun) is allowed. Option D (S3 list) is not needed for writing.

Answer

The Glue job is missing the s3:ListBucket permission on the bucket

Answer

The S3 bucket is encrypted with AWS KMS and the policy does not include kms:Decrypt permissions

Answer

The Glue job does not have permission to call glue:StartJobRun

Question 9

Refer to the exhibit. A data engineer runs an AWS Glue job that fails with an 'Access Denied' error when writing to S3. The IAM role attached to the job has s3:PutObject permission on the output bucket. What additional configuration is most likely missing?

Accepted Answer

The S3 bucket policy does not grant access to the Glue job's IAM role. Option C is correct because the error shows 'Access Denied' when writing, and the job uses Glue version 3.0 (which includes Spark 3.1), which may require the S3 bucket to have a bucket policy that allows the Glue service principal or a VPC endpoint. Often the issue is that the S3 bucket policy does not allow the Glue job's IAM role. Option A (KMS) is possible but not mentioned. Option B (VPC) might be needed if the job is in a VPC. Option D (CloudWatch) is not related.

Answer

The Glue job is not configured to write to S3 with the correct prefix

Answer

The Glue job is running in a VPC without an S3 VPC endpoint

Answer

The S3 bucket is encrypted with AWS KMS and the IAM role lacks kms:Decrypt permission

Question 10

Refer to the exhibit. A data engineer is troubleshooting a Kinesis Data Streams consumer that is falling behind. The stream has 2 shards and is receiving data at a rate of 2 MB/s. The consumer is an AWS Lambda function with a batch size of 100 records. What should the engineer do to improve consumer throughput?

Accepted Answer

Increase the number of shards in the stream to 4. Option A is correct because the stream is at its write limit (2 MB/s, but each shard supports 1 MB/s input, so 2 shards = 2 MB/s, but the consumer may need more read capacity. Increasing the number of shards increases both write and read capacity. Option B (Lambda memory) may help but not as effective. Option C (increasing retention) does not help throughput. Option D (decreasing batch size) reduces throughput.

Answer

Decrease the Lambda batch size to 10 records

Answer

Increase the retention period of the stream to 168 hours

Answer

Increase the memory allocation of the Lambda function

Question 11

A company uses AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job fails intermittently with a 'MemoryError'. What is the MOST likely cause?

Accepted Answer

The Glue job worker type is too small for the data volume. The error suggests the job runs out of memory. Increasing the DPU count can allocate more memory per worker and help process larger data volumes without memory errors.

Answer

The Glue job uses too many DynamicFrames

Answer

The S3 output bucket is in a different region

Answer

The Kinesis stream has insufficient shards

Question 12

A data pipeline uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The delivery occasionally fails with 'Firehose is throttled'. What should be done to reduce throttling?

Accepted Answer

Increase the buffer size and buffer interval. Throttling occurs when the write rate exceeds the buffer limit. Increasing the buffer size or interval allows Firehose to accumulate more records before writing, reducing the number of PutRecordBatch calls.

Answer

Enable compression on the Firehose delivery stream

Answer

Decrease the buffer size to flush more frequently

Answer

Increase the number of shards in the Kinesis stream

Question 13

A company uses AWS DMS to migrate a 2 TB Oracle database to Amazon RDS for PostgreSQL. The migration is taking longer than expected. The task status shows 'Full load in progress' with a low 'Table throughput (rows/s)'. Which action would MOST improve throughput?

Accepted Answer

Increase the number of parallel tasks in the DMS task settings. Using multiple parallel tasks splits the load across threads, improving throughput. Increasing the DMS replication instance size also helps, but parallel tasks are more effective for large datasets.

Answer

Enable Multi-AZ on the DMS replication instance

Answer

Change the target table preparation mode to 'Do nothing'

Answer

Increase the number of shards in the source database

Question 14

A data engineer is building a pipeline to ingest JSON files from Amazon S3 into Amazon Redshift. The files are 100 MB each and arrive every 5 minutes. Which service is BEST suited for this ingestion?

Accepted Answer

Amazon Redshift COPY command. Amazon Redshift COPY command efficiently loads large files from S3 in parallel. It is the most direct and performant approach for bulk loading into Redshift.

Answer

AWS Glue ETL job

Answer

AWS Lambda with Redshift Data API

Answer

Amazon Kinesis Data Firehose with Redshift destination

Question 15

A company uses AWS Glue DataBrew to clean and transform data for analytics. The source data is in Parquet format in Amazon S3. The transformation includes filtering rows and adding calculated columns. What is the MOST cost-effective way to run these transformations on a schedule?

Accepted Answer

Create a Glue DataBrew recipe and schedule the job using a cron expression. DataBrew jobs can be scheduled using a cron expression. This is the native way to run transformations automatically without additional services.

Answer

Use Amazon EMR with Spark

Answer

Create an AWS Lambda function triggered by S3 events

Answer

Use AWS Glue ETL with PySpark