DEA-C01 Data Ingestion and Transformation Practice Test 10 — 15 Questions

Question 1

A company uses Kinesis Data Streams to ingest IoT data. The data volume varies, and occasionally the shard write throughput is exceeded, causing ProvisionedThroughputExceeded exceptions. The data engineer needs to handle these spikes without losing data. Which approach is most cost-effective and requires minimal code changes?

Accepted Answer

Use Kinesis Data Firehose as a consumer with retries and buffer settings. Option B (Use Kinesis Data Firehose as a consumer with retries and buffer settings) is correct because Firehose can buffer data and retry on failures, handling spikes. Option A (Increase shard count) is costly and requires manual scaling. Option C (Use Kinesis Client Library with exponential backoff) is a good practice but still requires custom code; Firehose is more managed. Option D (Send data to SQS and then to Kinesis) adds complexity and cost.

Answer

Implement custom retry logic using the Kinesis Client Library with exponential backoff

Answer

Increase the number of shards to handle peak throughput

Answer

Send data to an SQS queue first, then have a Lambda function write to Kinesis

Question 2

A company wants to migrate on-premises data to Amazon S3 using AWS DataSync. The data is 10 TB and the network bandwidth is 1 Gbps. The migration must be completed within 48 hours. What should the data engineer do to meet the deadline?

Accepted Answer

Use AWS DataSync with multiple agents and enable data compression. Option D (Use AWS DataSync with multiple agents and enable data compression) is correct because it maximizes throughput. Option A (Use Snowball Edge) is not necessary for 10 TB with 1 Gbps; it would take about 22 hours theoretically, but compression and multiple agents can help. Option B (Increase bandwidth) is not always feasible. Option C (Use S3 Transfer Acceleration) improves upload speed but not as much as multiple agents and compression.

Answer

Use S3 Transfer Acceleration to speed up the transfer

Answer

Use AWS Snowball Edge to transfer the data physically

Answer

Request a bandwidth increase from the ISP

Question 3

A data engineer is designing a data pipeline that ingests streaming data from Kinesis Data Streams, transforms it using AWS Lambda, and writes to S3. The Lambda function sometimes fails due to transient errors, and the engineer wants to ensure no data is lost. Which approach should be used?

Accepted Answer

Use Kinesis Data Firehose as the delivery stream with Lambda for transformation and configure error handling with retries and a backup S3 bucket. Option D (Use Kinesis Data Firehose as the delivery stream with Lambda for transformation and configure error handling with retries and a backup S3 bucket) is correct because Firehose provides built-in retry and error handling. Option A (Increase Lambda timeout) does not solve transient errors. Option B (Use a DLQ) helps capture failures but does not automatically retry. Option C (Use Kinesis Client Library) requires more custom code.

Answer

Use the Kinesis Client Library to process records with checkpointing

Answer

Increase the Lambda function's timeout and memory

Answer

Configure a dead-letter queue (DLQ) on Lambda to capture failed records

Question 4

A company needs to ingest data from multiple SaaS applications (e.g., Salesforce, Marketo) into Amazon S3 for centralized analytics. The data volume is several GB per day. Which AWS service is most suitable for this ingestion?

Accepted Answer

AWS Glue. Option B (AWS Glue) is correct because Glue can connect to various data sources using crawlers and ETL jobs, and it supports JDBC connections to SaaS databases. Option A (Amazon Athena) is for querying, not ingestion. Option C (Amazon Kinesis) is for streaming, not batch from SaaS. Option D (AWS Data Pipeline) is older and less flexible than Glue.

Answer

Amazon Kinesis Data Firehose

Answer

Amazon Athena

Answer

AWS Data Pipeline

Question 5

A company is ingesting log files from EC2 instances into CloudWatch Logs and then wants to deliver them to S3 for long-term storage and analysis. The data engineer needs to ensure the logs are delivered to S3 within 5 minutes of being generated. Which approach meets this requirement?

Accepted Answer

Use CloudWatch Logs subscription filter with Kinesis Data Firehose to deliver to S3. Option A (Use CloudWatch Logs subscription filter with Kinesis Data Firehose to deliver to S3) is correct because Firehose can deliver to S3 with low latency (typically <1 minute). Option B (CloudWatch Logs export to S3) is a batch process that can take hours. Option C (Use a Lambda function to read from CloudWatch Logs and write to S3) is slower and less reliable. Option D (Use CloudWatch Logs Insights) is for querying, not delivery.

Answer

Configure a CloudWatch Logs metric filter and invoke a Lambda function to write to S3

Answer

Use CloudWatch Logs Insights to query logs and save results to S3

Answer

Use the CloudWatch Logs export to S3 feature

Question 6

A company is building a data lake on S3 and needs to ingest data from on-premises Oracle database. The data is 5 TB and changes incrementally. The ingestion must capture changes in near real-time (less than 1 minute latency) and be cost-effective. Which approach should be used?

Accepted Answer

Use AWS Database Migration Service (DMS) with ongoing replication to S3. Option B (Use AWS DMS with ongoing replication from Oracle to S3) is correct because DMS supports change data capture (CDC) with low latency. Option A (Full export using AWS Glue) does not capture incremental changes in near real-time. Option C (Use Kinesis Data Firehose with Oracle as source) is not directly possible without a connector. Option D (Use AWS DataSync) is for file transfers, not database CDC.

Answer

Use Amazon Kinesis Data Firehose with an Oracle JDBC connector

Answer

Use AWS Glue to perform a full table export daily

Answer

Use AWS DataSync to sync the Oracle data files to S3

Question 7

A data engineer needs to transform CSV files arriving in S3 into Parquet format and partition them by date. The transformation should be event-driven and run immediately after each file is uploaded. Which approach is most efficient?

Accepted Answer

Use S3 event notification to trigger an AWS Glue job. Option D (Use S3 event notification to trigger an AWS Glue job) is correct because Glue can be triggered by S3 events and supports Parquet conversion and partitioning. Option A (AWS Lambda) has a 15-minute timeout and may not handle large files. Option B (Amazon Athena CTAS) is not event-driven and does not automatically run on upload. Option C (Amazon EMR) is overkill for this use case.

Answer

Use an S3 event notification to invoke a Lambda function that converts the file

Answer

Use an Amazon EMR cluster running Spark to process files as they arrive

Answer

Use Amazon Athena CREATE TABLE AS SELECT (CTAS) on a schedule

Question 8

A data engineer is designing a data ingestion pipeline that uses AWS Lambda to process records from a Kinesis Data Stream and write to DynamoDB. Which TWO strategies can help handle increased throughput and prevent data loss? (Choose TWO.)

Accepted Answer

Configure the Lambda event source mapping with a batch window and set the number of concurrent batches per shard. Options B and D are correct. B: Increasing the number of shards allows higher throughput and Lambda concurrency. D: Configuring the Lambda event source mapping with a batch window and concurrent batches helps manage load. A: Increasing Lambda timeout does not increase throughput. C: Using a DLQ captures failures but does not prevent loss; retries should be configured. E: Using synchronous invocation would block and cause throttling.

Answer

Use synchronous invocation of Lambda from the producer

Answer

Configure a dead-letter queue (DLQ) for the Lambda function

Answer

Increase the Lambda function timeout

Question 9

A company is using AWS Glue to run ETL jobs that transform data from S3 to Redshift. The jobs are failing intermittently with out-of-memory errors. Which THREE actions can help resolve this issue? (Choose THREE.)

Accepted Answer

Increase the number of DPUs allocated to the Glue job. Options A, B, and D are correct. A: Increasing the number of DPUs provides more memory. B: Using a larger worker type (e.g., G.1X or G.2X) increases memory per worker. D: Optimizing the transformation logic to reduce memory usage helps. C: Using Spark's 'coalesce' reduces partitions but may not solve memory issues. E: Using S3 Select pushes down filtering but does not address memory.

Answer

Use S3 Select to filter data before reading into the Glue job

Answer

Use Spark's 'coalesce' function to reduce the number of partitions

Question 10

A data engineer needs to set up a data ingestion pipeline that reads from Amazon MSK (Managed Streaming for Kafka) and writes to Amazon S3 with transformations. The data is in Avro format and must be converted to Parquet. Which THREE components should be used together? (Choose THREE.)

Accepted Answer

AWS Lambda function to convert Avro to Parquet as a Firehose transformation. Options A, B, and D are correct. A: MSK is the source. B: Kinesis Data Firehose can consume from MSK via a custom endpoint and deliver to S3, with Lambda for transformation. D: Lambda can be used as a transformation function within Firehose to convert Avro to Parquet. C: Glue is not directly integrated with MSK as a Firehose source. E: Athena is for querying, not part of the ingestion pipeline.

Answer

Amazon Athena to convert the data format

Answer

AWS Glue ETL job to read from MSK

Question 11

A data engineer needs to ingest streaming data from thousands of IoT devices into Amazon S3 in near real-time. The data must be processed with minimal latency and stored in a columnar format for analytics. Which service should the engineer use to ingest the data?

Accepted Answer

Amazon Kinesis Data Firehose. Option B is correct because Amazon Kinesis Data Firehose is a fully managed service for loading streaming data into S3, and it can convert data to columnar formats like Parquet and ORC. Option A (Kinesis Data Streams) requires custom consumers. Option C (Kinesis Data Analytics) processes data but does not load to S3 directly. Option D (SQS) is a message queue, not designed for streaming ingestion.

Answer

Amazon Kinesis Data Analytics

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon Kinesis Data Streams with a Lambda consumer

Question 12

A company uses AWS Glue ETL jobs to transform data from Amazon RDS to Amazon S3 daily. The job recently started failing with memory errors. The data volume has grown 3x in the past month. Which change should the data engineer make to resolve the issue?

Accepted Answer

Increase the number of DPUs allocated to the Glue job. Option D is correct because increasing the number of DPUs (Data Processing Units) in the AWS Glue job provides more memory and compute resources, which can handle larger data volumes. Option A (increasing RDS instance size) may not help if the bottleneck is the Glue job. Option B (switching to Spark) is not directly relevant; Glue already uses Spark. Option C (partitioning S3 output) improves query performance but does not fix memory errors during transformation.

Answer

Increase the size of the Amazon RDS instance

Answer

Switch the Glue job type from Python Shell to Spark

Answer

Partition the output data in Amazon S3 by date

Question 13

A data engineer needs to design a data ingestion pipeline that captures change data capture (CDC) events from an on-premises SQL Server database to Amazon S3 with low latency. The pipeline must handle schema changes and ensure exactly-once delivery semantics. Which combination of AWS services should the engineer use?

Accepted Answer

AWS Database Migration Service (DMS) with Amazon Kinesis Data Firehose to Amazon S3. Option C is correct because AWS Database Migration Service (DMS) with ongoing replication captures CDC events from on-premises SQL Server and can write to S3. Amazon Kinesis Data Firehose then delivers the data to S3 with buffering, and using a custom prefix with partition keys can help handle schema changes. Option A (Kinesis Data Streams) is not directly compatible with SQL Server CDC. Option B (AppFlow) is for SaaS applications, not on-premises databases. Option D (Lambda direct to S3) would require custom CDC implementation.

Answer

AWS AppFlow with SQL Server connector to Amazon S3

Answer

AWS Glue ETL job with JDBC connection to SQL Server and writing to Amazon S3

Answer

Amazon Kinesis Data Streams with AWS Lambda consumer writing to Amazon S3

Question 14

A data engineer is troubleshooting an AWS Glue ETL job that fails with the error: 'An error occurred while calling o137.pyWriteDynamicFrame. No such file or directory: s3://bucket/output/part-00000.parquet'. The job reads from a JDBC source and writes to S3. What is the most likely cause?

Accepted Answer

The output S3 path does not exist and the Glue job does not have permission to create it. Option A is correct because the error indicates the output path does not exist; the Glue job may not have permission to create the directory if the bucket or prefix does not exist. Option B (IAM role) would cause a permissions error, not 'No such file or directory'. Option C (schema mismatch) would cause a different error. Option D (memory) would cause out-of-memory or timeout errors.

Answer

The schema of the source data has changed, causing a mismatch during write

Answer

The Glue job ran out of memory during the transformation phase

Answer

The IAM role attached to the Glue job lacks permissions to read from the JDBC source

Question 15

A company ingests JSON logs into Amazon S3 using Kinesis Data Firehose. The logs contain a timestamp field, but the delivery to S3 is delayed by up to 15 minutes during peak hours. The business requires near-real-time availability (under 2 minutes). Which configuration change should the data engineer make?

Accepted Answer

Decrease the buffer interval to 60 seconds. Option B is correct because reducing the buffer interval to 60 seconds (minimum 60s for Firehose) forces Firehose to deliver data more frequently, reducing latency. Option A (increasing buffer size) would increase latency. Option C (increasing shards) does not apply to Firehose directly; shards are for Kinesis Data Streams. Option D (buffering hints) is not a direct configuration in Firehose.

Answer

Increase the number of shards in the Kinesis Data Firehose stream

Answer

Increase the buffer size to 128 MB

Answer

Enable buffering hints in the Firehose delivery stream