DEA-C01 Data Ingestion and Transformation Practice Test 33 — 15 Questions

Question 1

A company uses AWS Lambda to process events from an S3 bucket. The Lambda function writes transformed data to another S3 bucket. Occasionally, the Lambda invocation fails with 'ResourceNotFoundException'. What is the MOST likely cause?

Accepted Answer

The destination S3 bucket does not exist or the Lambda function's IAM role lacks permissions.. Option B is correct. The destination bucket may not exist or the Lambda function's IAM role lacks permissions to write to it. Option A is wrong because Lambda timeouts would cause 'Timeout' error. Option C is wrong because S3 event notifications are reliable. Option D is wrong because the source bucket exists since it triggered the event.

Answer

The Lambda function timed out.

Answer

The S3 event notification is misconfigured.

Answer

The source S3 bucket has versioning disabled.

Question 2

A company uses AWS Glue to process large datasets. The Glue job occasionally fails with 'DiskFull' errors. Which TWO actions should the engineer take to resolve this issue? (Choose two.)

Accepted Answer

Increase the number of workers for the Glue job.

Answer

Store intermediate results in Amazon S3 instead of local disk.

Answer

Enable job bookmark to skip already processed data.

Answer

Use a Python shell job instead of Spark.

Question 3

A company needs to ingest streaming data from an existing Amazon Kinesis Data Streams into Amazon S3 with partitioning by date. Which TWO services can accomplish this with minimal coding? (Choose two.)

Accepted Answer

Amazon Kinesis Data Firehose

Answer

AWS Glue Streaming

Answer

Amazon Kinesis Data Analytics

Answer

Amazon S3 Transfer Acceleration

Question 4

A company uses AWS DMS to continuously replicate data from an on-premises SQL Server to Amazon Aurora MySQL. The replication lag is increasing. Which THREE actions can reduce the lag? (Choose three.)

Accepted Answer

Use parallel apply on the target endpoint.

Answer

Enable DMS validation.

Answer

Enable Multi-AZ for the DMS replication instance.

Question 5

A company ingests streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed in real-time using custom Python code before being stored in Amazon S3. Which AWS service should be used to perform this transformation?

Accepted Answer

Amazon Kinesis Data Analytics for Apache Flink. Option C is correct because Amazon Kinesis Data Analytics for Apache Flink allows running Apache Flink applications, which can include custom Python transformations via Flink's Python API. Option A is wrong because AWS Glue is a batch ETL service, not optimized for real-time stream processing. Option B is wrong because Amazon Kinesis Data Firehose can perform simple transformations via Lambda, but the question specifies custom Python code; Firehose's transformation capability is limited to Lambda functions, which is possible but not the best practice for complex streaming transformations; however, the better answer is Kinesis Data Analytics for Apache Flink as it is purpose-built for stream processing with Flink. Option D is wrong because Amazon EMR is a big data platform that can process streams but requires more setup and is not the simplest managed service for this use case.

Answer

Amazon EMR

Answer

Amazon Kinesis Data Firehose

Answer

AWS Glue

Question 6

A data engineer is troubleshooting a daily batch ingestion pipeline that uses AWS Glue to read CSV files from Amazon S3 and write Parquet files to another S3 bucket. The job runs successfully but takes significantly longer than expected. The engineer notices that the input data is highly skewed with many small files. Which is the most effective optimization to reduce job duration?

Accepted Answer

Enable the 'groupFiles' option in the S3 source configuration. Option D is correct because grouping small files into larger splits reduces the overhead of task creation and improves Spark performance. Option A is wrong because increasing the number of DPUs can increase parallelism but may not help if the issue is file count overhead; it could even be wasteful. Option B is wrong because using a different file format is not the issue; the output is Parquet, which is already efficient. Option C is wrong because changing the shuffle behavior does not directly address the small files problem.

Answer

Change the output format to JSON

Answer

Increase the number of DPUs allocated to the job

Answer

Enable the 'use_glue_schema_registry' option

Question 7

A company wants to ingest real-time clickstream data from a website into Amazon S3 with minimal code. The data should be delivered within 60 seconds of generation. Which AWS service should be used?

Accepted Answer

Amazon Kinesis Data Firehose. Option B is correct because Amazon Kinesis Data Firehose is a fully managed service that can ingest streaming data and deliver it to S3 with near-real-time latency (typically under 60 seconds). Option A is wrong because AWS Database Migration Service is for database migration, not streaming ingestion. Option C is wrong because Amazon Kinesis Data Streams requires custom consumers and does not directly deliver to S3. Option D is wrong because Amazon S3 Transfer Acceleration speeds up uploads but does not provide streaming ingestion capabilities.

Answer

AWS Database Migration Service (DMS)

Answer

Amazon Kinesis Data Streams

Answer

Amazon S3 Transfer Acceleration

Question 8

A data engineer is designing a data ingestion pipeline to load data from an on-premises Oracle database into Amazon Redshift. The pipeline must capture changes (inserts, updates, deletes) with low latency and minimal impact on the source database. Which combination of AWS services should the engineer use?

Accepted Answer

AWS Database Migration Service (DMS) to Amazon S3, then COPY into Redshift. Option A is correct because AWS DMS with ongoing replication (change data capture) can capture changes from Oracle with minimal impact and replicate to S3, then COPY into Redshift. Option B is wrong because AWS Glue is batch-oriented and does not support real-time CDC natively. Option C is wrong because Lambda can process events but is not designed for continuous CDC from a database. Option D is wrong because Kinesis Data Streams requires custom producers and does not directly integrate with Oracle for CDC.

Answer

Amazon Kinesis Data Streams with a custom producer on Oracle

Answer

AWS Glue with JDBC connection to Oracle, writing to Redshift

Answer

AWS Lambda reading from Oracle logs and writing to Redshift

Question 9

A company is using Amazon Kinesis Data Firehose to ingest data into Amazon S3. The data must be transformed from JSON to Parquet format before delivery. Which feature should be enabled on the Firehose delivery stream?

Accepted Answer

Format conversion (Parquet/ORC). Option D is correct because Firehose can convert the input data format to Parquet or ORC using its built-in format conversion feature. Option A is wrong because Lambda transformation is for custom code, not format conversion. Option B is wrong because S3 events are for notifications, not transformation. Option C is wrong because Kinesis Data Analytics is for stream processing, not directly tied to Firehose.

Answer

Amazon Kinesis Data Analytics

Answer

Amazon S3 event notifications

Answer

AWS Lambda transformation

Question 10

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams. The stream has 10 shards, and the data volume is expected to grow by 50% over the next month. The engineer needs to ensure that the pipeline can scale without manual intervention. Which approach should be used?

Accepted Answer

Switch the Kinesis stream to on-demand capacity mode. Option B is correct because Kinesis Data Streams supports automatic scaling using the 'On-demand' capacity mode, which adjusts shards based on traffic. Option A is wrong because Auto Scaling groups are for EC2, not Kinesis. Option C is wrong because CloudWatch Alarms can trigger auto scaling, but scaling a Kinesis stream via API requires custom code; on-demand mode is simpler. Option D is wrong because Lambda does not natively scale the stream.

Answer

Set up a CloudWatch Alarm to trigger a Lambda function to add shards

Answer

Use an Auto Scaling group to add more shards

Answer

Configure the stream to use a Lambda function that scales shards

Question 11

A company needs to ingest data from multiple SaaS applications (e.g., Salesforce, Marketo) into Amazon S3 for analytics. The data sources have different schemas and update frequencies. Which AWS service should be used to build this ingestion pipeline with minimal code?

Accepted Answer

Amazon AppFlow. Option C is correct because AWS Glue has built-in connectors for many SaaS applications and can schedule crawlers to discover schema and extract data. Option A is wrong because Kinesis Data Firehose is for streaming data, not for batch extraction from SaaS APIs. Option B is wrong because AppFlow is specifically designed for SaaS data ingestion with minimal code. Actually, AppFlow is the correct answer; AWS Glue also has connectors but requires more setup. Correcting: Amazon AppFlow is purpose-built for ingesting data from SaaS applications to S3. Option C is wrong because Glue has connectors but AppFlow is simpler. The correct answer is B.

Answer

AWS Data Pipeline

Answer

AWS Glue

Answer

Amazon Kinesis Data Firehose

Question 12

A data engineer is using AWS Glue to run an ETL job that reads data from Amazon DynamoDB and writes to Amazon Redshift. The job fails with a 'ThroughputExceededException' error. What is the most likely cause?

Accepted Answer

The DynamoDB table's read capacity is insufficient for the Glue job's read rate. Option A is correct because DynamoDB throttles requests when read/write capacity is exceeded. The Glue job may be reading too fast for the table's provisioned capacity. Option B is wrong because Redshift concurrency scaling does not cause this error. Option C is wrong because Glue job timeout would be a different error. Option D is wrong because S3 permissions would be a different error.

Answer

The Glue job has a timeout setting that is too low

Answer

The Redshift cluster's concurrency scaling is insufficient

Answer

The S3 bucket where Glue writes temporary data does not have proper permissions

Question 13

A company is using Amazon Kinesis Data Streams with a Lambda consumer to process real-time events. The Lambda function is triggered by a DynamoDB stream to update a counter. Recently, the counter has been inaccurate due to duplicate processing. What is the most likely cause?

Accepted Answer

The Lambda function is not idempotent and is being retried on failures. Option D is correct because Lambda functions may be invoked multiple times for the same record if they fail or time out, leading to duplicates if the operation is not idempotent. Option A is wrong because DynamoDB streams guarantee at-least-once delivery, not exactly-once. Option B is wrong because Kinesis shard rebalancing does not cause duplicates. Option C is wrong because Lambda concurrency limits would cause throttling, not duplicates.

Answer

The DynamoDB stream is configured with 'TRIM_HORIZON' iterator type

Answer

The Lambda function's reserved concurrency is too low

Answer

The Kinesis stream has undergone a shard rebalance

Question 14

A data engineer is designing a data ingestion pipeline that uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The engineer wants to ensure that the data is organized in a directory structure by year, month, day, and hour. Which TWO configurations should the engineer set on the Firehose delivery stream? (Choose TWO.)

Accepted Answer

Enable dynamic partitioning. Correct options: A and D. Option A is correct because enabling dynamic partitioning allows Firehose to use partition keys from the data. Option D is correct because custom prefix configuration allows specifying the directory structure with expressions like '!{timestamp:yyyy}/!{timestamp:MM}/...'. Option B is wrong because S3 objects have keys, not directories; the prefix defines the path. Option C is wrong because Lambda transformation is not required for partitioning. Option E is wrong because data format conversion is separate from partitioning.

Answer

Use an AWS Lambda function to write to S3 with the desired prefix

Answer

Enable format conversion to Parquet

Answer

Configure an S3 bucket with versioning enabled

Question 15

A company is ingesting data from multiple sources into Amazon S3 using AWS Glue. The data is then transformed using Apache Spark on Amazon EMR. The data engineer wants to reduce the cost of storing and processing data by compressing the ingested files. Which THREE file formats support compression and are commonly used with Spark? (Choose THREE.)

Accepted Answer

ORC. Correct options: A, C, and D. Parquet, ORC, and Avro all support compression and are commonly used with Spark. Option B is wrong because JSON can be compressed but is not a columnar format and is less efficient; however, it does support compression, but the question asks for formats commonly used with Spark for analytics; Parquet, ORC, and Avro are typical. Option E is wrong because CSV can be compressed but is not efficient for Spark processing. The answer is A, C, D.

Answer

JSON

Answer

CSV