DEA-C01 Mock Exam 96 — 25 Questions

Question 1

A logistics company uses AWS Glue to process GPS data from delivery trucks. The data is stored in Amazon S3 as JSON files. The Glue job reads the JSON files, converts them to Parquet, and writes them back to S3. The company notices that the Glue job takes too long to complete. The data engineer wants to improve the job's performance without changing the code. What should the data engineer do?

Accepted Answer

Change the worker type to G.2X.. Option D is correct. Using a G.2X worker type provides more memory and CPU per worker, improving performance. Option A is wrong because increasing DPUs may help but is less efficient than using larger workers for memory-intensive tasks. Option B is wrong because decreasing DPUs would worsen performance. Option C is wrong because using G.1X is the default; upgrading to G.2X is better.

Answer

Increase the number of DPUs to 20.

Answer

Change the worker type to G.1X.

Answer

Decrease the number of DPUs to 5 to reduce overhead.

Question 2

A logistics company ingests real-time GPS location data from thousands of delivery vehicles into Amazon Kinesis Data Streams. Each vehicle sends a JSON payload every 10 seconds containing vehicle_id, latitude, longitude, timestamp, and speed. The data must be stored in Amazon S3 for historical analysis, but the company wants to first aggregate the data per vehicle per minute (average speed, min/max coordinates) to reduce storage costs. The solution must be serverless and handle potential duplicate records without double-counting. What should the engineer do?

Accepted Answer

Use Amazon Kinesis Data Analytics for Apache Flink to aggregate data in a 1-minute tumbling window with deduplication logic, then output to Kinesis Data Firehose for delivery to S3.. Option D is correct: Kinesis Data Analytics for Flink can aggregate data in real-time using tumbling windows and deduplication. Option A (Firehose with Lambda) would require per-record processing and state management. Option B (Glue streaming) adds latency. Option C (EMR) is not serverless.

Answer

Use Amazon EMR with Spark Streaming to perform the aggregation and write to S3.

Answer

Use Kinesis Data Firehose with a Lambda transformation to aggregate records in a 1-minute window.

Answer

Use an AWS Glue streaming ETL job with Spark Structured Streaming to aggregate and deduplicate.

Question 3

A financial company needs to ingest real-time stock trade data from multiple sources and store it in Amazon S3 for compliance. The data must be delivered within 1 minute of the trade occurring. The data volume is approximately 10,000 records per second, with occasional spikes to 50,000 records per second. The engineer has set up Amazon Kinesis Data Streams with 10 shards and a Kinesis Data Firehose delivery stream that reads from the Kinesis stream and writes to S3. However, during spikes, the Firehose delivery stream falls behind, causing data to be delayed beyond the 1-minute SLA. What should the engineer do to meet the SLA without over-provisioning?

Accepted Answer

Increase the number of shards in the Kinesis data stream to handle peak throughput and enable auto-scaling.. Option A is correct: Increase shard count to handle peak throughput, and enable automatic scaling for Kinesis Data Streams. Option B (increase buffer size) would increase latency. Option C (use Lambda) may not handle the throughput. Option D (use SQS) would not reduce latency.

Answer

Increase the buffer size in Kinesis Data Firehose from 1 MB to 5 MB to batch more data per delivery.

Answer

Use Amazon SQS as a buffer between Kinesis and Firehose to absorb spikes.

Answer

Replace Firehose with an AWS Lambda function that writes directly to S3 for lower latency.

Question 4

A financial services company ingests stock trade data from multiple exchanges into an Amazon S3 bucket (trade-bucket). Each exchange sends a CSV file every 5 minutes. The data must be transformed into Parquet format and partitioned by exchange and date (trade_date) for efficient querying using Amazon Athena. The pipeline must handle late-arriving data (files up to 2 hours late) and ensure exactly-once processing to avoid duplicates. Currently, a scheduled AWS Glue ETL job runs every hour, reads new CSV files, converts them to Parquet, and writes to an output bucket. However, the team is experiencing data duplication: if the job fails midway, upon retry it reprocesses all files in the input folder, causing duplicates in the output. Additionally, the job takes too long because it scans all files each run. The engineer must redesign the pipeline to eliminate duplicates and improve efficiency. What should the engineer do?

Accepted Answer

Set up an S3 event notification to invoke an AWS Lambda function that starts a Glue job with a parameter containing the S3 object key of the new file; modify the Glue job to process only that file and use the file key to avoid duplicates.. Option D is the best approach: S3 event notifications trigger Lambda to start a Glue job passing the file key; the job processes only that file, using the file key to avoid reprocessing. Option A still processes all files and may cause duplicates if a file arrives after the job starts. Option B (Glue Workflows) does not solve the file-level tracking. Option C (EMR) is overkill and still scans the entire input folder.

Answer

Use AWS Glue Workflows to orchestrate the job and add a condition to check for duplicates before writing.

Answer

Modify the Glue job to move processed CSV files to an archive folder after successful transformation, and process only unprocessed files.

Answer

Replace Glue with Amazon EMR and use Spark Structured Streaming with checkpointing to process files incrementally.

Question 5

A data engineer is designing a data ingestion pipeline for a social media company. The pipeline ingests user posts from a REST API into Amazon S3. The API returns JSON data with an array of posts. The engineer needs to transform the data into individual JSON objects per post and store them in S3 with a partition structure of year/month/day/hour. The data should be available in S3 within 15 minutes of ingestion. The engineer decides to use AWS Lambda for transformation. Which combination of services should the engineer use to meet these requirements with minimal operational overhead?

Accepted Answer

Use Amazon CloudWatch Events to trigger an AWS Lambda function every 15 minutes. The Lambda function calls the API, transforms the data, and writes individual JSON objects to S3 with the required partition structure.. Option A is correct because using CloudWatch Events to trigger Lambda on a schedule reduces complexity and Lambda can fetch API data and write to S3. Option B is wrong because Kinesis Firehose expects streaming data, not batch API calls. Option C is wrong because Glue is heavier for simple transformation. Option D is wrong because Step Functions adds unnecessary complexity.

Answer

Use AWS Step Functions to orchestrate an API call and data transformation with Lambda, running every 15 minutes.

Answer

Use AWS Glue ETL jobs scheduled with AWS Glue triggers to run every 15 minutes.

Answer

Use Amazon Kinesis Data Firehose with a Lambda function for transformation. Configure Firehose to pull from the API every 15 minutes.

Question 6

A data engineering team is responsible for ingesting streaming data from a fleet of IoT devices into Amazon S3 using Kinesis Data Firehose. The data volume spikes unpredictably, and the team has configured Kinesis Data Firehose with a buffer size of 5 MB and buffer interval of 60 seconds. During spikes, the team notices that the delivery to S3 is delayed, and some records are lost due to exceeding the service limits. The team needs to ensure no data loss and reduce delivery latency. What should the team do?

Accepted Answer

Increase the buffer size to 10 MB and buffer interval to 120 seconds to allow more data accumulation before delivery.. Option A is correct because increasing the buffer size and interval reduces the number of PutRecord.Batch calls and allows Firehose to handle larger spikes without throttling. Option B is wrong because using Kinesis Data Streams as a source adds a buffer but does not directly fix Firehose delivery issues. Option C is wrong because Lambda cannot directly increase Firehose throughput. Option D is wrong because S3 Transfer Acceleration improves upload speed from clients, not Firehose delivery.

Answer

Implement an AWS Lambda function to pre-process the data and send it to Firehose in a throttled manner.

Answer

Use Amazon Kinesis Data Streams as the data source for Firehose to decouple ingestion and delivery.

Answer

Enable S3 Transfer Acceleration on the destination bucket.

Question 7

A financial services company is building a real-time fraud detection system. Transaction data is ingested via Amazon Kinesis Data Streams and processed by an Amazon Kinesis Data Analytics for Apache Flink application that runs sliding window aggregations. The output is written to an Amazon S3 bucket for downstream analysis. The Flink application is configured with parallelism of 4 and checkpointing every minute. The company has noticed that the application is experiencing high latency and the checkpointing is frequently failing. The CloudWatch metrics show that the Flink application's CPU utilization is near 100% and the checkpoint duration is spiking to over 5 minutes. The data engineer needs to improve performance. Which action should the data engineer take?

Accepted Answer

Increase the parallelism of the Flink application to distribute the workload across more resources.. Option C is correct because increasing parallelism allows the workload to be distributed across more resources, reducing CPU pressure and checkpoint duration. Option A is wrong because reducing checkpoint interval would increase frequency and likely worsen failures. Option B is wrong because increasing shards without increasing parallelism may not help if CPU is bottleneck. Option D is wrong because memory increase is not the primary issue.

Answer

Increase the number of shards in the source Kinesis stream to improve throughput.

Answer

Increase the heap memory of the Flink application to handle larger state.

Answer

Decrease the checkpoint interval to 30 seconds to reduce the amount of state being checkpointed.

Question 8

A company is using Amazon Kinesis Data Streams to ingest real-time clickstream data from a website. The data is consumed by an Amazon Kinesis Data Analytics for Apache Flink application that performs real-time analytics. The Flink application writes its results to an Amazon S3 bucket. The company has noticed that the Flink application is experiencing high checkpoint failure rates, causing delays. The CloudWatch metrics show that the checkpoint size is large and increasing. The data engineer needs to reduce the checkpoint size. Which action should the data engineer take?

Accepted Answer

Enable incremental checkpointing in the Flink application to only write changes since the last checkpoint.. Option D is correct because enabling incremental checkpointing in Flink reduces the amount of data written per checkpoint by only writing changes since the last checkpoint. Option A is wrong because reducing parallelism may increase load per operator. Option B is wrong because decreasing checkpoint interval increases frequency, not size. Option C is wrong because state TTL does not directly reduce checkpoint size.

Answer

Decrease the checkpoint interval to reduce the amount of state accumulated.

Answer

Reduce the parallelism of the Flink application.

Answer

Increase the state time-to-live (TTL) configuration to retain state longer.

Question 9

A company runs a data pipeline using AWS Glue ETL jobs to process daily files from an S3 bucket. The files are in CSV format and range from 1 GB to 10 GB. The Glue job runs successfully for small files but fails with an 'Out of Memory' error for files larger than 5 GB. The job uses a single G.1X DPU (16 GB memory). The company needs to process these large files without changing the existing ETL script. Which solution should the company implement?

Accepted Answer

Increase the number of DPUs and use the G.2X worker type to provide more memory per worker.. Option A is correct because increasing the DPU count and using G.2X worker type provides more memory per worker, resolving the memory issue without script changes. Option B is wrong because converting to Parquet may reduce data size but does not guarantee memory issues are resolved and may require script changes. Option C is wrong because using a different file format may not address memory issues. Option D is wrong because using Spark on EMR requires rewriting the script.

Answer

Convert the input files from CSV to Parquet format to reduce memory usage.

Answer

Use the Optimus format in AWS Glue to compress data.

Answer

Use Amazon EMR with Spark instead of AWS Glue.

Question 10

A data engineer is building a real-time data pipeline to ingest sensor data from IoT devices. The data is sent to AWS IoT Core, which publishes messages to a Kinesis Data Stream. Each message is about 1 KB in size. The data must be transformed (add a device location field) and then stored in Amazon S3 for long-term analytics. The engineer has set up a Lambda function to transform the records and write to S3. However, the engineer notices that the Lambda function is invoked thousands of times per second, causing high costs and occasional throttling. The Lambda function processes only one record at a time. The engineer wants to reduce the number of Lambda invocations and improve throughput. What should the engineer do?

Accepted Answer

Configure the event source mapping to use a larger batch size and set a batch window.. Option C is correct because increasing the batch size in the event source mapping allows Lambda to process multiple records per invocation, reducing invocations. Option A is wrong because increasing Lambda memory does not reduce invocations. Option B is wrong because Firehose can batch records, but the transformation would still be per record unless using Firehose's built-in Lambda integration. Option D is wrong because reducing concurrent executions would cause throttling and backlogs.

Answer

Reduce the number of shards in the Kinesis stream to limit concurrency.

Answer

Increase the Lambda function's memory allocation to improve performance.

Answer

Replace the Lambda function with Amazon Kinesis Data Firehose and use its built-in transformation.

Question 11

A company needs to ingest data from an on-premises MySQL database into Amazon S3 for analytics. The database is 2 TB in size. The company has a low-bandwidth internet connection (10 Mbps). They need to perform an initial full load and then incremental updates every hour. Which approach should they use?

Accepted Answer

Use AWS Database Migration Service (DMS) to perform the full load and ongoing replication.. Option B is correct because AWS Database Migration Service (DMS) supports full load and ongoing replication, and can be used with limited bandwidth. Option A is wrong because Snowball Edge is for offline transfer, not ongoing replication. Option C is wrong because Glue ETL is not optimized for continuous replication. Option D is wrong because Kinesis Data Firehose is for streaming data, not database replication.

Answer

Use Kinesis Data Firehose to stream data from MySQL to S3.

Answer

Use AWS Glue ETL jobs to extract data and load into S3.

Answer

Use AWS Snowball Edge to transfer the initial full load, then use AWS DataSync for incremental updates.

Question 12

A company is using Amazon Kinesis Data Firehose to ingest clickstream data from a website into an S3 bucket. The data is then analyzed using Amazon Athena. Recently, the company noticed that Athena queries are returning incomplete results for the last 30 minutes of data. The Firehose delivery stream is configured to buffer data for 60 seconds or 5 MB before delivering to S3. The S3 bucket has a lifecycle policy that transitions objects to Amazon S3 Glacier after 30 days. The IAM role for Firehose has permissions to write to S3 and access a CloudWatch Logs group. The engineer checks the Firehose monitoring and sees that the delivery rate is healthy, but the 'S3.Bytes' metric shows a spike in the last hour. The 'BackupToS3.Bytes' metric is zero. What is the MOST likely cause of the missing data?

Accepted Answer

The data is still being buffered in Firehose and has not yet been delivered to S3.. Option A is correct. The buffer settings (60 seconds or 5 MB) mean that data may be held in the buffer for up to 60 seconds before being written to S3. For the last 30 minutes, some data may still be in the buffer and not yet delivered. Athena queries only see data that has been delivered. Option B (lifecycle policy) would not affect recent data. Option C (IAM permissions) would cause errors, not missing data. Option D (backup) is unrelated.

Answer

The lifecycle policy is transitioning data before Athena can query it.

Answer

The backup is enabled and data is being sent to the backup bucket instead.

Answer

The IAM role for Firehose does not have permissions to write to the S3 bucket.

Question 13

A data engineer is setting up a data ingestion pipeline using Amazon Kinesis Data Firehose to deliver web server logs to Amazon S3. The logs are in JSON format and the engineer wants to convert them to Parquet format. The engineer has configured a Glue table for the schema. However, when testing, the Firehose delivery stream fails with 'Error converting to Parquet'. The engineer checks the Glue table schema and notices that it includes a column 'timestamp' of type 'string' in the format 'yyyy-MM-dd HH:mm:ss'. The logs have a 'timestamp' field in the same format. What is the MOST likely cause of the failure?

Accepted Answer

The Glue table schema does not match the data schema exactly.. Option A is correct because Firehose requires the schema to match the data exactly. If the Glue table schema is not consistent with the data, conversion fails. Option B is wrong because Firehose supports Parquet conversion with Glue schema. Option C is wrong because S3 bucket permissions would cause a different error. Option D is wrong because string type is acceptable for conversion.

Answer

Firehose does not support Parquet conversion.

Answer

The S3 bucket lacks write permissions for Firehose.

Answer

The 'timestamp' column must be of type 'timestamp' instead of 'string'.

Question 14

A logistics company ingests GPS tracking data from thousands of vehicles into Amazon S3 via AWS Direct Connect. Each vehicle sends a message every 5 seconds, resulting in about 200,000 messages per second. Each message is about 200 bytes. The company uses AWS Glue to transform the data into a parquet format and load it into Amazon Redshift for real-time analytics. However, the Glue jobs are failing due to memory issues and the data is not being loaded into Redshift quickly enough. The company needs to reduce the latency of data availability in Redshift. Which action should the data engineer take?

Accepted Answer

Use Amazon Kinesis Data Firehose to ingest the data directly into S3 and then use Redshift Spectrum to query the data without loading.. Option B is correct because Amazon Kinesis Data Firehose can ingest high-throughput streaming data and deliver it to S3 in near-real-time, reducing latency. Option A is wrong because increasing Glue DPUs may not fix memory issues and doesn't reduce latency significantly. Option C is wrong because Kinesis Data Analytics adds processing overhead. Option D is wrong because Redshift Spectrum is for querying S3, not reducing ingestion latency.

Answer

Use Amazon Kinesis Data Analytics to process the data in real-time and write to Redshift directly.

Answer

Increase the size of the Redshift cluster to improve load performance.

Answer

Increase the number of DPUs and allocate more memory to the Glue job.

Question 15

A media company ingests large video files from partners via AWS Transfer Family (SFTP) into an S3 bucket. Each file is typically 2-5 GB. Once uploaded, an AWS Lambda function is triggered to transcode the video using Amazon Elastic Transcoder. The Lambda function reads the file from S3, submits a transcoding job to Elastic Transcoder, and writes the output back to a different S3 bucket. Recently, the Lambda function has been failing intermittently with timeouts, and the company reports that some files are not being transcoded. The CloudWatch logs show that the Lambda function is timing out after 15 minutes. The average transcoding job takes about 10 minutes to complete. The data engineer needs to fix the issue without changing the architecture drastically. What should the data engineer do?

Accepted Answer

Modify the Lambda function to submit the transcoding job asynchronously and exit, using an SNS topic to trigger a second Lambda function when the job completes.. Option B is correct because the Lambda function should not wait for the transcoding job to complete; it should submit the job and exit. The current design synchronously waits, causing timeouts. Option A is wrong because increasing the timeout to 20 minutes may still fail if jobs take longer. Option C is wrong because the issue is not about concurrency. Option D is wrong because DMS is for database migration, not file processing.

Answer

Increase the Lambda function's reserved concurrency to allow multiple invocations in parallel.

Answer

Increase the Lambda function timeout to 20 minutes to accommodate longer transcoding jobs.

Answer

Replace AWS Transfer Family with AWS Database Migration Service to handle file transfers more efficiently.

Question 16

A company has a Kinesis Data Firehose delivery stream that receives JSON data from IoT devices. The data is delivered to an S3 bucket. The company notices that the data in S3 is delayed by up to 30 minutes. The Firehose stream is configured with a buffer size of 1 MB and a buffer interval of 60 seconds. The incoming data rate is approximately 100 KB per second. The company needs to reduce the delivery latency to under 5 minutes. Which action should the company take?

Accepted Answer

Decrease the buffer size to 256 KB.. Option B is correct because reducing the buffer interval to 60 seconds (already set) is not enough; the actual issue is that buffer size is too large relative to data rate. Decreasing buffer interval to 60 seconds is already set, but they need to also reduce buffer size or increase data rate. Actually, the correct action is to decrease the buffer size to 1 MB (already) and decrease buffer interval to 60 seconds (already). Wait, the latency is due to buffer interval of 60 seconds? No, the problem states latency up to 30 minutes. That suggests that the buffer interval is not the only factor; maybe the data rate is low. Actually, with 100 KB/s, it takes about 10 seconds to fill 1 MB buffer. So buffer interval of 60 seconds should cause latency up to 60 seconds. The 30-minute delay suggests another issue. Perhaps the Firehose is waiting for more data or there is a backlog. The correct answer is to decrease the buffer interval to 60 seconds (already) and also decrease the buffer size? Actually, option B says decrease buffer size to 256 KB? That would cause more frequent deliveries and reduce latency. Option A is wrong because increasing buffer interval would increase latency. Option C is wrong because changing compression format does not affect latency. Option D is wrong because using Lambda adds processing time.

Answer

Enable Lambda transformation to process data faster.

Answer

Increase the buffer interval to 300 seconds.

Answer

Change the compression format from GZIP to Snappy.

Question 17

A company runs a data ingestion pipeline that uses AWS Glue to read 500 GB of JSON files from an S3 bucket (s3://raw-data/) every hour. The Glue ETL job transforms the data and writes Parquet files to another S3 bucket (s3://processed-data/). The job is triggered by a time-based CloudWatch Events rule. Recently, the job has started taking over 2 hours to complete, causing delays in downstream processes. The data volume has been consistent, and no changes have been made to the job code or infrastructure. The S3 bucket 's3://raw-data/' receives new files continuously, but the Glue job reads all files in the bucket each run (no incremental processing). The engineer suspects that the job is reprocessing old data. Which action should the engineer take FIRST to reduce the job duration?

Accepted Answer

Enable Glue job bookmarking and configure the job to process only new data.. Option C is correct because enabling job bookmarking in Glue allows the job to process only new files since the last run, dramatically reducing processing time. Option A (increasing DPUs) would help with resource constraints but does not address the root cause of reprocessing old data. Option B (increasing parallelism) may help but not as much as eliminating reprocessing. Option D (using partition pruning) assumes the data is partitioned, but the stem says all files are read; partitioning might not help if files are not organized by time. The most impactful first step is to enable bookmarking.

Answer

Increase the parallelism of the Spark job by repartitioning the data.

Answer

Add partition pruning by modifying the S3 path to include date-based partitions.

Answer

Increase the number of DPUs for the Glue job to 100.

Question 18

A company runs a nightly batch ETL job using AWS Glue to transform data from Amazon RDS for MySQL to Amazon S3. The job reads 100 tables and writes Parquet files partitioned by date. Recently, the job started failing with 'ThrottlingException' from the RDS database. The data volume has increased, and the Glue job is reading large tables without any filtering. The job uses a single Glue job with multiple Spark executors. The engineer needs to reduce the load on the RDS database while maintaining the same processing time. What should the engineer do?

Accepted Answer

Modify the Glue job to use a JDBC connection with a WHERE clause to read only the latest partition.. Option D is correct because using a JDBC connection with a WHERE clause to filter data by date reduces the amount of data read from RDS, reducing throttling. Option A is wrong because increasing DPUs would increase parallelism and potentially increase load on RDS. Option B is wrong because DMS is a migration tool, not for regular ETL. Option C is wrong because reading all data into S3 first does not reduce load on RDS; the initial read still causes throttling.

Answer

Use AWS DMS to continuously replicate data to S3 and then run Glue on the S3 data.

Answer

Change the job to read all data from RDS into a staging table in S3 first, then transform.

Answer

Increase the number of DPUs to process data faster.

Question 19

A company is using Amazon Kinesis Data Firehose to ingest log data from web servers into an Amazon S3 bucket. The data is then queried by Amazon Athena. The company has noticed that the Athena queries are slow and expensive. The data engineer wants to optimize the storage format to improve query performance and reduce costs. Which configuration change should the data engineer make to the Firehose delivery stream?

Accepted Answer

Enable data transformation in Firehose to convert JSON to Parquet format with Snappy compression.. Option B is correct because converting data to Parquet format and compressing it reduces storage space and improves query performance in Athena. Option A is wrong because storing as ORC is also good but Parquet is more common with Athena. Option C is wrong because increasing buffer size delays delivery. Option D is wrong because enabling S3 server access logs adds cost and does not help query performance.

Answer

Increase the buffer interval to 600 seconds and buffer size to 128 MB to create larger files.

Answer

Change the output format to ORC and enable GZIP compression.

Answer

Enable S3 server access logs to track query patterns.

Question 20

A company is using AWS Glue to run ETL jobs that transform data from Amazon DynamoDB to Amazon S3. The DynamoDB table has a large number of items (over 10 million) and is heavily used by production applications. The Glue job reads the entire DynamoDB table each time it runs, causing increased read capacity consumption and affecting production performance. The team wants to reduce the impact on the source DynamoDB table while still keeping the S3 data up-to-date. What should the team do?

Accepted Answer

Use DynamoDB Streams and AWS Lambda to capture changes and write them to S3, then run incremental Glue jobs.. Option B is correct because using DynamoDB Streams with AWS Lambda allows incremental change data capture (CDC), reducing the need to read the entire table. Option A is wrong because increasing read capacity might help but still reads the whole table. Option C is wrong because reducing parallelism increases job duration but still reads everything. Option D is wrong because exporting to S3 via the console is a one-time operation, not incremental.

Answer

Increase the DynamoDB read capacity units to handle the Glue job's read load.

Answer

Use the DynamoDB console to export the table to S3 in Parquet format.

Answer

Reduce the parallelism of the Glue job to lower the read throughput.