DEA-C01 Data Ingestion and Transformation Practice Test 4 — 15 Questions

Question 1

A data engineer is troubleshooting a Kinesis Data Firehose delivery stream that is failing to deliver data to an Amazon S3 bucket. The stream is configured with a Lambda transformation function. The CloudWatch logs show that the Lambda function is timing out. Which action should the engineer take to resolve the issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the Lambda function timeout setting.. The CloudWatch logs indicate the Lambda function is timing out. The default Lambda timeout for a Firehose transformation is 60 seconds, and if the function's processing exceeds this limit, it will fail. Increasing the Lambda function timeout setting (Option B) directly addresses this by allowing the function more time to complete its execution before being terminated.

Answer

Reduce the Firehose buffer interval.

Answer

Decrease the Lambda function's batch size in Firehose.

Answer

Increase the memory allocated to the Lambda function.

Question 2

A data engineer is designing a data ingestion pipeline for streaming social media data. The data must be ingested with low latency (seconds) and stored in Amazon S3 for long-term analytics. The engineer also needs to perform real-time aggregations. Which TWO services should the engineer use? (Choose two.)

Accepted Answer

Amazon Kinesis Data Analytics. Amazon Kinesis Data Streams (E) is correct because it ingests streaming data with low latency (sub-second to seconds) and allows multiple consumers, which is essential for both real-time processing and delivery to storage. Amazon Kinesis Data Analytics (D) is correct because it runs SQL or Apache Flink queries continuously over the stream to produce real-time aggregations, satisfying the real-time analytics requirement. Together, Data Streams captures the social media feed and Data Analytics computes rolling metrics on it. AWS Glue ETL (B) is a batch-oriented, scheduled service and is not designed for second-level streaming latency. Amazon S3 (C) is the required long-term storage destination but is not an ingestion or real-time aggregation service. Amazon Kinesis Data Firehose (A) can load streams into S3, but it does not itself perform real-time aggregations, so it does not meet the stated analytics need.

Answer

Amazon Kinesis Data Firehose

Answer

AWS Glue ETL

Answer

Amazon S3

Question 3

A company is building a data lake on Amazon S3. The data sources include relational databases, streaming data, and log files. The data engineer needs to ensure that the data ingestion pipeline can handle schema evolution, support both batch and streaming, and provide a unified metadata catalog. Which THREE services should the engineer use? (Choose three.)

Accepted Answer

AWS Glue. AWS Glue (A) is correct because it provides a serverless ETL engine and the Glue Data Catalog, a unified metadata catalog that stores table definitions and schemas, and it supports schema evolution through crawlers and ETL jobs that can update table schemas as source structures change. Amazon S3 (D) is correct because it is the foundational storage layer for the data lake, holding the ingested relational, streaming, and log data in its original or transformed form while decoupling storage from compute. Amazon Kinesis Data Firehose (E) is correct because it handles streaming ingestion, buffering and delivering streaming data into S3 (and other destinations) with optional transformation via Lambda, satisfying the streaming requirement alongside batch ingestion. Amazon DynamoDB (B) is not appropriate because it is a NoSQL key-value database, not a data lake ingestion or cataloging service. Amazon Athena (C) is a query service that reads data from S3 using the Glue Data Catalog; it is used for analysis rather than ingestion, so it does not fulfill the ingestion pipeline requirement.

Answer

Amazon DynamoDB

Answer

Amazon Athena

Question 4

A data engineer is troubleshooting a slow-running AWS Glue ETL job that reads from Amazon S3 and writes to Amazon Redshift. The job processes 500 GB of CSV data daily. The engineer wants to improve performance. Which THREE actions should the engineer take? (Choose three.)

Accepted Answer

Use a JDBC connection with a higher batch size for writing to Redshift.. Option A is correct because increasing the JDBC batch size (e.g., via the Redshift connection's batchsize parameter) reduces the number of round trips and COPY/INSERT operations, improving write throughput to Redshift. Option B is correct because partitioning the S3 input by date or category enables partition pruning, so Glue reads only the relevant data and avoids scanning the full 500 GB. Option D is correct because adding more DPUs increases the number of executors and parallel tasks, allowing the Spark-based Glue job to process data faster. Option C is not appropriate because a single-node Redshift cluster reduces compute and storage capacity, worsening performance rather than improving it. Option E is not ideal because combining files into larger files can reduce parallelism and may not address the underlying bottleneck; Glue generally benefits from appropriately sized, partitioned files rather than simply fewer, larger ones.

Answer

Switch to a single-node Redshift cluster to reduce latency.

Answer

Reduce the number of input files by combining them into larger files.

Question 5

Refer to the exhibit. A data engineer is configuring an IAM policy for an AWS Glue ETL job that reads data from the 'my-data-bucket' S3 bucket, transforms it, and writes the output back to the same bucket. The engineer wants to prevent accidental deletion of objects. Based on the policy, which statement is true about the Glue job's permissions?

Accepted Answer

The job can read and write objects, but cannot delete objects.. The IAM policy explicitly denies the `s3:DeleteObject` action, which prevents the Glue job from deleting objects in the 'my-data-bucket' S3 bucket. The policy allows `s3:GetObject` and `s3:PutObject` actions, enabling the job to read and write objects as required for the ETL process. This ensures the job can perform its transformation tasks without the risk of accidental deletion.

Answer

The job can write objects but cannot read objects.

Answer

The job can read objects but cannot write objects.

Answer

The job can read and write, but may also delete objects.

Question 6

A data engineer is designing a data ingestion pipeline for clickstream data that arrives in bursts, up to 100 MB/s, and must be processed with exactly-once semantics. The data must be stored in Amazon S3 partitioned by event date and hour. Which combination of services should the engineer use?

Accepted Answer

Amazon Kinesis Data Firehose with S3 destination and dynamic partitioning.. Amazon Kinesis Data Firehose with dynamic partitioning can directly ingest high-velocity clickstream data (up to 100 MB/s bursts) and automatically partition it by event date and hour in S3 with no custom code. It supports exactly-once delivery to S3 when configured with the `S3DestinationConfiguration` and appropriate error handling, meeting the burst throughput and partitioning requirements without managing consumers.

Answer

Amazon Kinesis Data Streams with AWS Lambda consumer writing to S3.

Answer

AWS Glue streaming ETL job reading from Amazon MSK and writing to S3.

Answer

Amazon Kinesis Data Streams with KCL application writing to S3.

Question 7

A company needs to ingest CSV files from an FTP server into Amazon S3 daily. The files are typically 50 MB each, and the process should be fully managed with minimal operational overhead. Which AWS service should be used?

Accepted Answer

AWS Transfer Family. AWS Transfer Family is the correct choice because it provides a fully managed, serverless solution for transferring files to and from Amazon S3 using FTP, FTPS, or SFTP protocols. It eliminates the need to manage any FTP infrastructure, directly integrates with S3 as a destination, and handles the daily ingestion of 50 MB CSV files with minimal operational overhead, aligning perfectly with the requirement for a fully managed service.

Answer

AWS Lambda with FTP library

Answer

AWS DataSync

Answer

Amazon AppFlow

Question 8

A company uses Amazon Kinesis Data Analytics (now Managed Service for Apache Flink) to run a Flink application on streaming data. The application fails with 'OutOfMemoryError: Java heap space'. The data volume is 10 MB/s. What is the most likely cause and solution?

Accepted Answer

The application's Parallelism is too low; increase the number of Parallelism and KPUs.. The OutOfMemoryError in a Flink application on Amazon Kinesis Data Analytics (Managed Service for Apache Flink) is most likely due to insufficient parallelism to handle the 10 MB/s data volume. Increasing parallelism distributes the workload across more KPUs (Kinesis Processing Units), reducing memory pressure per operator and preventing heap exhaustion. Option D directly addresses this by scaling resources to match throughput.

Answer

The data contains records larger than 1 MB; split records into smaller chunks.

Answer

Checkpointing is enabled too frequently; reduce checkpoint interval.

Answer

The Flink application is not suitable for 10 MB/s throughput; use Kinesis Data Firehose instead.

Question 9

A company uses AWS DMS to migrate data from Oracle to Aurora MySQL. During the ongoing replication, the target table shows duplicate primary key errors. What is the most likely cause?

Accepted Answer

The target table has an auto-increment column, and DMS is inserting explicit values that conflict.. When DMS replicates into a target table with an auto-increment column, it attempts to insert explicit values from the source. If the target's auto-increment counter has already generated a value matching one of those explicit inserts, a duplicate primary key error occurs. This is the most likely cause because DMS does not automatically skip or re-map auto-increment columns unless explicitly configured.

Answer

DMS is using 'Limited LOB mode' and truncating LOB data, causing row mismatches.

Answer

The source table has a trigger that inserts additional rows.

Answer

The DMS task is configured with 'Parallel apply' threads that cause race conditions.

Question 10

A data engineer is building a real-time data pipeline using Amazon Kinesis Data Streams with a Lambda consumer. The data volume is 2 MB/s with average record size of 5 KB. The Lambda function processes records and writes to DynamoDB. Occasionally, the Lambda function fails with 'ProvisionedThroughputExceededException' on DynamoDB. What is the best way to handle this?

Accepted Answer

Configure a Lambda destination on failure to send records to an SQS dead-letter queue, and implement retry logic in Lambda.. Configuring a Lambda destination on failure to send unprocessed records to an SQS dead-letter queue, combined with retry logic in the Lambda function, provides a robust mechanism to handle DynamoDB throttling exceptions. This approach decouples the retry handling from the Kinesis stream, preventing the Lambda function from blocking the shard iterator and allowing the pipeline to continue processing other records while failed records are retried or investigated.

Answer

Replace Lambda with the Kinesis Client Library (KCL) running on EC2.

Answer

Increase the Lambda function's reserved concurrency to process more records in parallel.

Answer

Use DynamoDB Streams to capture the records and process them asynchronously.

Question 11

A company is using AWS Glue ETL to process data from Amazon RDS for MySQL to Amazon S3. The job runs daily and takes 2 hours to complete. The engineer wants to improve performance without increasing cost significantly. Which TWO actions should the engineer take? (Choose TWO.)

Accepted Answer

Add a partition column to the source table based on a date column.. Option D is correct because adding a partition column based on a date column allows AWS Glue to read only the relevant partitions (partition pruning) instead of scanning the entire source table, which reduces I/O and shuffle volume and speeds up the daily job without materially raising cost. Option E is correct because increasing the number of Glue DPUs adds parallel executors, so the Spark job can process partitions concurrently and finish the 2-hour run faster; the cost scales with DPU-hours, but since the job finishes sooner, the total cost increase is modest and often offset by the shorter runtime. Option A is not appropriate because switching to a smaller worker type (G.1X instead of G.2X) reduces memory and compute per worker, which typically slows the job and can cause spills or OOM errors rather than improving performance. Option B is not the best choice because DynamicFrames are already optimized for Glue ETL and converting to Spark DataFrames does not by itself improve performance for this RDS-to-S3 pipeline. Option C is not correct because Glue Auto Scaling adjusts workers based on workload but does not guarantee the performance improvement requested here and can still incur additional DPU cost, making it less directly effective than partitioning and adding DPUs.

Answer

Switch to a smaller worker type (e.g., G.1X instead of G.2X).

Answer

Use Spark DataFrames instead of DynamicFrames.

Answer

Enable 'Auto Scaling' in the Glue job configuration.

Question 12

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams with multiple consumers. The data must be processed by a Lambda function for real-time alerts and also stored in Amazon S3 for historical analysis. Which THREE components are needed to implement this architecture? (Choose THREE.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct component because it is the fully managed service designed to load streaming data into Amazon S3 without requiring custom code. It can directly subscribe to a Kinesis Data Stream as its source and automatically buffer, batch, compress, and deliver records to S3 for historical analysis, making it the ideal choice for the storage leg of this architecture.

Answer

Amazon Kinesis Data Analytics

Answer

Amazon SQS queue

Question 13

A data engineer has an IAM policy attached to an IAM role used by an AWS Glue job. The Glue job needs to read from S3 bucket 'data-bucket' and write to the same bucket. The job fails with an access denied error when trying to write to S3. What is the issue?

Accepted Answer

The policy does not include s3:ListBucket permission, which Glue may need.. AWS Glue jobs require the `s3:ListBucket` permission on the bucket to perform operations like listing objects, even when reading and writing specific keys. Without this permission, the job fails with an access denied error when trying to write, as Glue internally uses `ListBucket` to verify bucket existence and access patterns. The policy attached to the IAM role likely includes `s3:GetObject` and `s3:PutObject` but omits the bucket-level `s3:ListBucket` action, which is necessary for the Glue job to interact with S3 successfully.

Answer

The Glue job cannot assume the IAM role because of trust policy.

Answer

The resource ARN for S3 is missing the bucket-level permission.

Answer

The actions for S3 are incorrect; s3:PutObject is not sufficient.

Question 14

A data engineering team needs to load data from an on-premises Oracle database to Amazon S3 daily. The data volume is about 50 GB per day, and the network bandwidth is 100 Mbps. The team wants to minimize operational overhead and use AWS managed services. Which solution should they choose?

Accepted Answer

Use AWS Database Migration Service (DMS) to migrate the data to S3.. AWS DMS is the correct choice because it is purpose-built for migrating databases to AWS with minimal operational overhead. It can connect to an on-premises Oracle database via a JDBC or native Oracle connection, perform a full load, and then continuously replicate changes to an S3 target in Parquet or CSV format. With 50 GB/day over 100 Mbps (about 10.8 GB/hour theoretical max), the full load can complete in under 5 hours, and ongoing replication handles daily increments without manual intervention.

Answer

Use AWS DataSync to copy the database files directly to S3.

Answer

Use AWS Glue with a JDBC connection and schedule a crawler to load data into S3.

Answer

Use Amazon Kinesis Data Firehose to stream data from Oracle to S3.

Question 15

A company uses Amazon Kinesis Data Streams with a shard count of 10 to ingest clickstream data. The data is consumed by a Lambda function that transforms the records and writes to Amazon S3. Recently, the Lambda function started failing with 'ProvisionedThroughputExceededException' errors. The average record size is 5 KB, and the incoming data rate is 15 MB/s. What is the most likely cause and solution?

Accepted Answer

Increase the number of shards in the Kinesis data stream to 15.. The ProvisionedThroughputExceededException indicates that the total write throughput to the Kinesis stream is exceeding the provisioned capacity. With 10 shards, the maximum write throughput is 10 MB/s (1 MB/s per shard). The incoming data rate is 15 MB/s, which exceeds this limit, causing the error. Increasing the shard count to 15 raises the write throughput to 15 MB/s, matching the incoming rate and resolving the issue.

Answer

Decrease the batch size of the Lambda event source mapping.

Answer

Increase the Lambda function's memory allocation to 3008 MB.

Answer

Increase the Lambda function's reserved concurrency.