DEA-C01 Data Ingestion and Transformation Practice Test 8 — 15 Questions

Question 1

Refer to the exhibit. A data engineer is configuring an IAM policy for an AWS Glue ETL job that reads data from the 'my-data-bucket' S3 bucket, transforms it, and writes the output back to the same bucket. The engineer wants to prevent accidental deletion of objects. Based on the policy, which statement is true about the Glue job's permissions?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The job can read and write objects, but cannot delete objects.. The IAM policy explicitly denies the `s3:DeleteObject` action, which prevents the Glue job from deleting objects in the 'my-data-bucket' S3 bucket. The policy allows `s3:GetObject` and `s3:PutObject` actions, enabling the job to read and write objects as required for the ETL process. This ensures the job can perform its transformation tasks without the risk of accidental deletion.

Answer

The job can write objects but cannot read objects.

Answer

The job can read objects but cannot write objects.

Answer

The job can read and write, but may also delete objects.

Question 2

Refer to the exhibit. A data engineer has configured an S3 event notification to send an event to an SQS queue when objects are created in the 'incoming/' prefix. The engineer wants to trigger an AWS Lambda function to process the object. However, the Lambda function is not being invoked. What is the most likely cause?

Accepted Answer

Lambda is not configured as an event source for the SQS queue.. The Lambda function is not being invoked because the SQS queue is not configured as an event source for Lambda. Even though S3 sends events to SQS, Lambda will only poll and process messages from the queue if an event source mapping (e.g., via CreateEventSourceMapping) is established. Without this mapping, the messages sit in the queue and Lambda remains idle.

Answer

The Lambda function lacks permission to read from the S3 bucket.

Answer

The SQS queue does not exist or is in a different account.

Answer

The S3 event notification filter prefix is incorrect.

Question 3

Refer to the exhibit. An AWS Glue ETL job is failing with an OutOfMemoryError. The job reads from Amazon S3 and performs a GROUP BY on a large dataset. Which change should the data engineer make to resolve this error?

Accepted Answer

Increase the number of DPUs allocated to the Glue job.. The OutOfMemoryError in an AWS Glue ETL job performing a GROUP BY on a large dataset indicates that the executors do not have enough memory to handle the shuffle operations required for aggregation. Increasing the number of DPUs (Data Processing Units) allocated to the Glue job increases the total memory and compute resources available, allowing the job to process larger partitions without running out of memory.

Answer

Use coalesce to reduce the number of partitions.

Answer

Increase the number of partitions in the DataFrame.

Answer

Use repartition to increase the number of partitions.

Question 4

A data engineer needs to ingest streaming data from thousands of IoT devices into AWS for real-time processing. The data volume peaks at 5 GB/min. Which AWS service should be used as the ingestion endpoint?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is designed for real-time data ingestion at scale, supporting throughput of up to 1 MB/s or 1,000 records/s per shard. With a peak of 5 GB/min (~83 MB/s), you can horizontally scale by adding shards to meet the required throughput, making it the ideal ingestion endpoint for high-volume streaming IoT data.

Answer

AWS Glue

Answer

Amazon S3

Answer

AWS Lambda

Question 5

A company uses AWS DMS to migrate an on-premises PostgreSQL database to Amazon RDS for PostgreSQL. After initial load, ongoing replication is set up. The replication task shows 'Task status: failed with error: The specified LSN is not available in the source database logs.' What is the most likely cause?

Accepted Answer

The source database's WAL retention period is too short, and required logs have been purged.. The error 'The specified LSN is not available in the source database logs' indicates that AWS DMS is trying to read a Write-Ahead Log (WAL) position that has already been purged. PostgreSQL sources require sufficient WAL retention to allow DMS to catch up during ongoing replication (CDC). If the WAL segments are recycled or removed before DMS reads them, the replication task fails with this specific LSN error.

Answer

DMS does not support PostgreSQL as a source for ongoing replication.

Answer

The source database's network security group blocks outbound traffic to DMS.

Answer

The full load was incomplete, preventing CDC from starting.

Question 6

A data engineer is designing a data ingestion pipeline for clickstream data that arrives in bursts, up to 100 MB/s, and must be processed with exactly-once semantics. The data must be stored in Amazon S3 partitioned by event date and hour. Which combination of services should the engineer use?

Accepted Answer

Amazon Kinesis Data Firehose with S3 destination and dynamic partitioning.. Amazon Kinesis Data Firehose with dynamic partitioning can directly ingest high-velocity clickstream data (up to 100 MB/s bursts) and automatically partition it by event date and hour in S3 with no custom code. It supports exactly-once delivery to S3 when configured with the `S3DestinationConfiguration` and appropriate error handling, meeting the burst throughput and partitioning requirements without managing consumers.

Answer

Amazon Kinesis Data Streams with AWS Lambda consumer writing to S3.

Answer

AWS Glue streaming ETL job reading from Amazon MSK and writing to S3.

Answer

Amazon Kinesis Data Streams with KCL application writing to S3.

Question 7

A company needs to ingest CSV files from an FTP server into Amazon S3 daily. The files are typically 50 MB each, and the process should be fully managed with minimal operational overhead. Which AWS service should be used?

Accepted Answer

AWS Transfer Family. AWS Transfer Family is the correct choice because it provides a fully managed, serverless solution for transferring files to and from Amazon S3 using FTP, FTPS, or SFTP protocols. It eliminates the need to manage any FTP infrastructure, directly integrates with S3 as a destination, and handles the daily ingestion of 50 MB CSV files with minimal operational overhead, aligning perfectly with the requirement for a fully managed service.

Answer

AWS Lambda with FTP library

Answer

AWS DataSync

Answer

Amazon AppFlow

Question 8

A data engineer is using AWS Glue ETL to transform a large dataset in S3. The job processes 2 TB of data daily and currently runs for 6 hours. The engineer wants to reduce runtime without changing the transformation logic. What is the best approach?

Accepted Answer

Increase the number of Glue DPUs or enable auto-scaling.. Increasing the number of DPUs or enabling auto-scaling directly allocates more distributed processing capacity to the AWS Glue job, which reduces runtime for large datasets by parallelizing the workload across more resources. Since the transformation logic is fixed and the job is already running on Spark, adding compute capacity is the most straightforward way to speed up processing without code changes.

Answer

Reduce the number of DPUs to minimize overhead.

Answer

Use the Spark UI to analyze bottlenecks and rewrite code.

Answer

Switch from Spark to Python shell.

Question 9

A company uses Amazon Kinesis Data Analytics (now Managed Service for Apache Flink) to run a Flink application on streaming data. The application fails with 'OutOfMemoryError: Java heap space'. The data volume is 10 MB/s. What is the most likely cause and solution?

Accepted Answer

The application's Parallelism is too low; increase the number of Parallelism and KPUs.. The OutOfMemoryError in a Flink application on Amazon Kinesis Data Analytics (Managed Service for Apache Flink) is most likely due to insufficient parallelism to handle the 10 MB/s data volume. Increasing parallelism distributes the workload across more KPUs (Kinesis Processing Units), reducing memory pressure per operator and preventing heap exhaustion. Option D directly addresses this by scaling resources to match throughput.

Answer

The data contains records larger than 1 MB; split records into smaller chunks.

Answer

Checkpointing is enabled too frequently; reduce checkpoint interval.

Answer

The Flink application is not suitable for 10 MB/s throughput; use Kinesis Data Firehose instead.

Question 10

A data engineer needs to transform JSON data from Amazon S3 into Parquet format using AWS Glue. The source files are in a bucket with thousands of small files. What is the best practice to optimize the Glue job performance?

Accepted Answer

Enable 'Group small files' in the Glue job or use a DynamicFrame with coalesce.. Enabling 'Group small files' in AWS Glue automatically coalesces thousands of small input files into larger partitions, reducing the number of tasks and minimizing overhead from task scheduling and S3 list operations. This is the recommended best practice for handling small files in Glue ETL jobs, as it optimizes read performance without requiring manual coalesce or repartitioning.

Answer

Convert the JSON files to CSV before processing with Glue.

Answer

Use an AWS Lambda function to pre-process the files.

Answer

Increase the number of DPUs to the maximum.

Question 11

A company uses AWS DMS to migrate data from Oracle to Aurora MySQL. During the ongoing replication, the target table shows duplicate primary key errors. What is the most likely cause?

Accepted Answer

The target table has an auto-increment column, and DMS is inserting explicit values that conflict.. When DMS replicates into a target table with an auto-increment column, it attempts to insert explicit values from the source. If the target's auto-increment counter has already generated a value matching one of those explicit inserts, a duplicate primary key error occurs. This is the most likely cause because DMS does not automatically skip or re-map auto-increment columns unless explicitly configured.

Answer

DMS is using 'Limited LOB mode' and truncating LOB data, causing row mismatches.

Answer

The source table has a trigger that inserts additional rows.

Answer

The DMS task is configured with 'Parallel apply' threads that cause race conditions.

Question 12

A data engineer is building a real-time data pipeline using Amazon Kinesis Data Streams with a Lambda consumer. The data volume is 2 MB/s with average record size of 5 KB. The Lambda function processes records and writes to DynamoDB. Occasionally, the Lambda function fails with 'ProvisionedThroughputExceededException' on DynamoDB. What is the best way to handle this?

Accepted Answer

Configure a Lambda destination on failure to send records to an SQS dead-letter queue, and implement retry logic in Lambda.. Configuring a Lambda destination on failure to send unprocessed records to an SQS dead-letter queue, combined with retry logic in the Lambda function, provides a robust mechanism to handle DynamoDB throttling exceptions. This approach decouples the retry handling from the Kinesis stream, preventing the Lambda function from blocking the shard iterator and allowing the pipeline to continue processing other records while failed records are retried or investigated.

Answer

Replace Lambda with the Kinesis Client Library (KCL) running on EC2.

Answer

Increase the Lambda function's reserved concurrency to process more records in parallel.

Answer

Use DynamoDB Streams to capture the records and process them asynchronously.

Question 13

A data engineer needs to ingest data from a SaaS application (Salesforce) into Amazon S3 on a daily basis. Which TWO AWS services can be used for this purpose? (Choose TWO.)

Accepted Answer

AWS Glue. AWS Glue is correct because it provides a fully managed ETL service that can connect to Salesforce via JDBC and extract data into Amazon S3 on a scheduled basis. Glue's crawlers and jobs can handle incremental loads and schema evolution, making it suitable for daily ingestion from SaaS sources.

Answer

AWS DataSync

Answer

Amazon Kinesis Data Streams

Answer

AWS Transfer Family

Question 14

A company is using AWS Glue ETL to process data from Amazon RDS for MySQL to Amazon S3. The job runs daily and takes 2 hours to complete. The engineer wants to improve performance without increasing cost significantly. Which TWO actions should the engineer take? (Choose TWO.)

Accepted Answer

Add a partition column to the source table based on a date column.. Adding a partition column (e.g., based on a date column) to the source table enables AWS Glue to use partition pruning during the read phase. This reduces the amount of data scanned and processed by the ETL job, directly improving performance without increasing cost. Partitioning is a common optimization for large datasets in RDS or S3-based sources.

Answer

Switch to a smaller worker type (e.g., G.1X instead of G.2X).

Answer

Use Spark DataFrames instead of DynamicFrames.

Answer

Enable 'Auto Scaling' in the Glue job configuration.

Question 15

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams with multiple consumers. The data must be processed by a Lambda function for real-time alerts and also stored in Amazon S3 for historical analysis. Which THREE components are needed to implement this architecture? (Choose THREE.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct component because it is the fully managed service designed to load streaming data into Amazon S3 without requiring custom code. It can directly subscribe to a Kinesis Data Stream as its source and automatically buffer, batch, compress, and deliver records to S3 for historical analysis, making it the ideal choice for the storage leg of this architecture.

Answer

Amazon Kinesis Data Analytics

Answer

Amazon SQS queue