AWS Certified Data Engineer Associate DEA-C01 practice test

A data pipeline uses Kinesis Data Firehose to deliver streaming data to an S3 bucket. The data volume spikes occasionally, causing the Firehose buffer to fill up and leading to increased delivery latency. The latency must remain under 60 seconds. What should be done to minimize latency?

Trap 1: Enable GZIP compression on the Firehose delivery stream.

Compression adds CPU overhead and may increase latency.

Trap 2: Increase the buffer size to 128 MB to accommodate larger batches.

Larger buffer size increases latency as it waits to fill.

Trap 3: Switch to Kinesis Data Streams with a Lambda consumer.

This changes architecture but does not directly reduce Firehose latency.

A
Enable GZIP compression on the Firehose delivery stream.
Why wrong: Compression adds CPU overhead and may increase latency.
B
Increase the buffer size to 128 MB to accommodate larger batches.
Why wrong: Larger buffer size increases latency as it waits to fill.
C
Switch to Kinesis Data Streams with a Lambda consumer.
Why wrong: This changes architecture but does not directly reduce Firehose latency.
D
Reduce the buffer interval to 60 seconds.
This forces delivery every 60 seconds, meeting the latency requirement.

Question 2hardmultiple choice

An e-commerce company uses AWS Glue to run ETL jobs that transform clickstream data from Amazon S3. The job reads Parquet files, performs aggregations, and writes the results to Amazon Redshift. The job runs successfully but takes longer than expected. The data volume is increasing. Which design change would MOST improve the job's performance?

Trap 1: Write the aggregated results to a single large file instead of…

Single file reduces parallelism and increases shuffle overhead.

Trap 2: Convert the Parquet files to CSV to simplify the schema.

CSV is less efficient than Parquet for columnar storage and compression.

Trap 3: Replace the Redshift target with Amazon Redshift Spectrum.

Spectrum is for querying S3, not for loading transformed data into Redshift.

A
Write the aggregated results to a single large file instead of multiple partitions.
Why wrong: Single file reduces parallelism and increases shuffle overhead.
B
Convert the Parquet files to CSV to simplify the schema.
Why wrong: CSV is less efficient than Parquet for columnar storage and compression.
C
Replace the Redshift target with Amazon Redshift Spectrum.
Why wrong: Spectrum is for querying S3, not for loading transformed data into Redshift.
D
Increase the number of Glue worker nodes (DPUs) for the job.
More workers parallelize tasks and reduce runtime.

Question 3hardmultiple choice

A data engineering team uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. They notice that the application's checkpointing is failing intermittently, causing data reprocessing. The application uses a large state. Which configuration change should the team make to improve checkpoint reliability?

Trap 1: Disable checkpointing to avoid failures.

Disabling checkpointing removes fault tolerance.

Trap 2: Switch the state backend from in-memory to RocksDB.

RocksDB helps manage large state but does not directly fix checkpoint failures.

Trap 3: Increase the parallelism of the application.

Higher parallelism can increase checkpoint overhead.

A
Disable checkpointing to avoid failures.
Why wrong: Disabling checkpointing removes fault tolerance.
B
Switch the state backend from in-memory to RocksDB.
Why wrong: RocksDB helps manage large state but does not directly fix checkpoint failures.
C
Increase the parallelism of the application.
Why wrong: Higher parallelism can increase checkpoint overhead.
D
Increase the checkpointing interval.
Longer intervals reduce checkpoint frequency and associated failures.

Question 4mediummultiple choice

A company uses AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job reads JSON records and writes Parquet to Amazon S3. Recently, the job started failing with 'Out of Memory' errors. Which change is MOST likely to resolve the issue?

Trap 1: Enable compression on the Kinesis stream.

Compression on Kinesis does not affect Glue memory.

Trap 2: Change the output format from Parquet to ORC.

ORC is similar to Parquet; format change does not fix OOM.

Trap 3: Reduce the streaming batch size in the Glue job configuration.

Reducing batch size can help but does not address the root cause of insufficient memory.

A
Enable compression on the Kinesis stream.
Why wrong: Compression on Kinesis does not affect Glue memory.
B
Change the output format from Parquet to ORC.
Why wrong: ORC is similar to Parquet; format change does not fix OOM.
C
Increase the number of DPUs allocated to the Glue job.
More DPUs provide more memory and CPU.
D
Reduce the streaming batch size in the Glue job configuration.
Why wrong: Reducing batch size can help but does not address the root cause of insufficient memory.

Question 5easymulti select

A data engineer is designing a serverless data ingestion pipeline that uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data must be transformed using AWS Lambda before being written to S3. Which two steps are required to enable this transformation? (Select TWO.)

Trap 1: Set up an S3 event notification to trigger the Lambda function on…

This triggers Lambda after data is written, not before.

Trap 2: Subscribe the Lambda function to the CloudWatch Logs log group for…

This is for monitoring, not for data transformation.

Trap 3: Have the Lambda function write the transformed data directly to the…

This bypasses Firehose and is not the intended design.

A
Set up an S3 event notification to trigger the Lambda function on object creation.
Why wrong: This triggers Lambda after data is written, not before.
B
Configure a Lambda function as a data transformation source in the Firehose delivery stream.
This enables Firehose to invoke Lambda for transformation.
C
Ensure the Lambda function returns the transformed data in the format required by Firehose.
Firehose expects a specific response format from the Lambda function.
D
Subscribe the Lambda function to the CloudWatch Logs log group for the Firehose stream.
Why wrong: This is for monitoring, not for data transformation.
E
Have the Lambda function write the transformed data directly to the S3 bucket.
Why wrong: This bypasses Firehose and is not the intended design.

Question 6hardmultiple choice

A company runs a nightly AWS Glue ETL job that reads from a JDBC source (PostgreSQL) and writes to S3 in Parquet format. The job takes over 6 hours, but the SLA requires completion within 4 hours. The source table has 500 million rows and is updated frequently. Which approach will most reliably reduce job duration?

Trap 1: Enable job bookmark and schedule the job to run more frequently.

Bookmarks help with incremental processing but not with the initial full load.

Trap 2: Use multiple JDBC connections in parallel by setting…

Glue supports parallel JDBC reads using hash partitioning; this can improve performance.

Trap 3: Increase the number of DPUs for the Glue job to 100.

More DPUs can help but are limited by source parallelism and may not reduce time enough.

A
Enable job bookmark and schedule the job to run more frequently.
Why wrong: Bookmarks help with incremental processing but not with the initial full load.
B
Use multiple JDBC connections in parallel by setting 'hashexpression' and 'hashfield'.
Why wrong: Glue supports parallel JDBC reads using hash partitioning; this can improve performance.
C
Partition the source table by year and use pushdown predicates in the Glue job.
This reduces the data scanned by filtering on partition columns.
D
Increase the number of DPUs for the Glue job to 100.
Why wrong: More DPUs can help but are limited by source parallelism and may not reduce time enough.