DEA-C01 Data Ingestion and Transformation Practice Test 28 — 15 Questions

Question 1

A company uses AWS Database Migration Service (DMS) to continuously replicate data from an on-premises Oracle database to Amazon S3 in Parquet format. The replication is used for near-real-time analytics. Recently, the DMS task started failing with an error indicating insufficient memory. The source database is large (2 TB). What should a data engineer do to resolve this issue while minimizing changes to the existing architecture?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the DMS replication instance size.. The error indicates the DMS replication instance is running out of memory during continuous replication of a 2 TB Oracle database to S3 in Parquet format. Increasing the replication instance size (Option D) directly addresses the memory constraint by providing more RAM and processing capacity, which is necessary for handling large volumes of Change Data Capture (CDC) data and Parquet conversion overhead. This solution requires minimal architectural changes, as it only involves modifying the instance class in the DMS task settings.

Answer

Change the target format to JSON to reduce memory usage.

Answer

Split the DMS task into multiple smaller tasks.

Answer

Use Change Data Capture (CDC) only, without full load.

Question 2

A data engineer needs to ingest data from multiple SaaS applications (Salesforce, Marketo) into Amazon S3 for a data lake. The data volumes are moderate and the sync needs to be scheduled daily. Which AWS service is most appropriate for this task?

Accepted Answer

Amazon AppFlow. Amazon AppFlow is purpose-built for securely transferring data between SaaS applications (like Salesforce and Marketo) and AWS services (like S3). It supports scheduled, incremental data syncs with built-in connectors, making it the most appropriate choice for moderate-volume daily ingestion into a data lake.

Answer

AWS Glue

Answer

AWS Database Migration Service (DMS)

Answer

Amazon Kinesis Data Firehose

Question 3

A data engineer is designing a data ingestion pipeline for a social media analytics platform. The pipeline must ingest tweets in real-time, perform sentiment analysis, and store results in Amazon S3. The sentiment analysis is compute-intensive and must be done as the data arrives. The estimated throughput is 10,000 tweets per second. Which architecture is most suitable?

Accepted Answer

Amazon Kinesis Data Streams with Amazon Kinesis Data Analytics for sentiment analysis, then Kinesis Data Firehose to S3.. The most suitable because Amazon Kinesis Data Streams can ingest up to 10,000 records per second per shard (with shard-level scaling), and Kinesis Data Analytics provides built-in, low-latency stream processing for compute-intensive sentiment analysis using SQL or Apache Flink. Kinesis Data Firehose then reliably buffers and writes the processed results to Amazon S3 without custom code, ensuring near-real-time delivery.

Answer

Amazon SQS with AWS Lambda pollers to process tweets and store in S3.

Answer

Amazon EMR with Spark Streaming to process tweets and write to S3.

Answer

Amazon API Gateway with AWS Lambda to process each tweet and store in S3.

Question 4

A data engineer needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The network bandwidth is limited to 100 Mbps. The transfer must be completed within one week. Which service should be used?

Accepted Answer

AWS Snowball. AWS Snowball is the correct choice because transferring 50 TB over a 100 Mbps network would take approximately 46 days (50 TB * 8 bits/byte / 100 Mbps / 86400 seconds/day), far exceeding the one-week deadline. Snowball provides a physical storage device that can be shipped to the on-premises location, allowing data to be loaded locally and shipped to AWS, bypassing network bandwidth constraints entirely.

Answer

AWS DataSync

Answer

Amazon CloudFront

Answer

AWS Database Migration Service (DMS)

Question 5

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format. The delivery stream is configured with a buffer size of 5 MB and a buffer interval of 60 seconds. However, the data engineer notices that S3 objects are being created with sizes much smaller than 5 MB. What is a likely cause?

Accepted Answer

The incoming data rate is too low, causing the buffer interval to trigger before reaching the buffer size.. Kinesis Data Firehose delivers data to S3 when either the buffer size (5 MB) or the buffer interval (60 seconds) is reached, whichever occurs first. If the incoming data rate is low, the buffer interval will expire before accumulating 5 MB of data, resulting in smaller S3 objects.

Answer

The data is being compressed before delivery, reducing object size.

Answer

The data transformation lambda is splitting records into smaller ones.

Answer

The S3 bucket is configured with a lifecycle policy that splits objects.

Question 6

An e-commerce company uses AWS Glue to run ETL jobs that transform clickstream data from Amazon S3. The job reads Parquet files, performs aggregations, and writes the results to Amazon Redshift. The job runs successfully but takes longer than expected. The data volume is increasing. Which design change would MOST improve the job's performance?

Accepted Answer

Increase the number of Glue worker nodes (DPUs) for the job.. Increasing the number of Glue worker nodes (DPUs) directly scales the distributed processing capacity of the ETL job, allowing it to process larger volumes of Parquet data in parallel. This is the most straightforward way to reduce execution time when data volume is growing, as AWS Glue automatically partitions the workload across the additional workers.

Answer

Write the aggregated results to a single large file instead of multiple partitions.

Answer

Convert the Parquet files to CSV to simplify the schema.

Answer

Replace the Redshift target with Amazon Redshift Spectrum.

Question 7

A data engineer needs to ingest JSON data from an on-premises relational database into Amazon S3 every hour. Which AWS service should be used to set up a scheduled, incremental data transfer?

Accepted Answer

AWS Database Migration Service (DMS) with S3 as target.. AWS DMS is purpose-built for migrating databases to AWS targets, including Amazon S3. It supports ongoing replication (change data capture) and scheduled full-load tasks, making it ideal for hourly incremental transfers from an on-premises relational database to S3 without custom scripting.

Answer

Amazon S3 Transfer Acceleration with a cron job.

Answer

AWS Glue with a JDBC connection and a scheduled crawler.

Answer

Amazon Kinesis Data Firehose with a database source.

Question 8

A data engineer is designing a serverless data ingestion pipeline that uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data must be transformed using AWS Lambda before being written to S3. Which two steps are required to enable this transformation? (Select TWO.)

Accepted Answer

Configure a Lambda function as a data transformation source in the Firehose delivery stream.. Option B is correct because Firehose supports Lambda-based data transformation by letting you specify a Lambda function as the processor in the delivery stream's transform configuration (via the console or the ProcessingConfiguration/TransformParameters in the API), which Firehose then invokes synchronously for each buffered batch. Option C is correct because the Lambda function must return the transformed records in the exact structure Firehose expects — a JSON object containing records with recordId, result (Ok, Dropped, or ProcessingFailed), and base64-encoded data — otherwise Firehose cannot continue delivery. Option A is wrong because S3 event notifications trigger actions on object creation and play no role in Firehose's inline transformation; Firehose itself invokes the Lambda function. Option D is wrong because subscribing Lambda to the Firehose CloudWatch Logs log group is not a configuration step for transformation and would not enable it. Option E is wrong because the Lambda function must return transformed data to Firehose, not write directly to S3; Firehose remains responsible for delivering the records to the destination bucket.

Answer

Set up an S3 event notification to trigger the Lambda function on object creation.

Answer

Subscribe the Lambda function to the CloudWatch Logs log group for the Firehose stream.

Answer

Have the Lambda function write the transformed data directly to the S3 bucket.

Question 9

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3. The data volume is about 500 GB initially, with daily incremental updates of 10 GB. The pipeline must minimize operational overhead. Which AWS service should be used for the initial and incremental loads?

Accepted Answer

AWS Database Migration Service (DMS) with change data capture (CDC) to Amazon S3.. AWS DMS with CDC is the correct choice because it supports continuous replication from Oracle to Amazon S3 with minimal overhead. It handles both the initial 500 GB full load and ongoing 10 GB daily increments via change data capture, without requiring custom code or complex pipeline management.

Answer

AWS Glue with a JDBC connection and incremental crawl.

Answer

Amazon Kinesis Data Firehose with a custom producer.

Answer

AWS Data Pipeline with a SQL activity and HiveCopyActivity.

Question 10

A company is building a data lake on Amazon S3. Data arrives from multiple sources in JSON, CSV, and Avro formats. The data must be transformed to Parquet and partitioned by date and source. Which TWO services can perform this transformation with minimal custom code? (Choose TWO.)

Accepted Answer

Amazon EMR with Spark. Amazon EMR with Spark (A) is correct because Spark natively reads JSON, CSV, and Avro, and can write Parquet with partitionBy('date','source') using built-in DataFrame APIs, requiring only a small script rather than a custom transformation engine. AWS Glue ETL jobs (D) are correct because Glue provides managed, serverless Spark with built-in DynamicFrame readers/writers for JSON, CSV, and Avro, plus automatic schema inference and native Parquet output with partition keys, minimizing custom code. AWS Lake Formation (B) is a permission and metadata/catalog governance layer, not a data transformation engine. Amazon Athena CTAS (C) can convert formats and partition results, but it is query-oriented and less suited to general multi-format ETL pipelines. Amazon Kinesis Data Firehose (E) is a streaming delivery service that can convert to Parquet via Glue schema, but it does not perform the required multi-source batch transformation and partitioning logic.

Answer

AWS Lake Formation

Answer

Amazon Athena CTAS queries

Answer

Amazon Kinesis Data Firehose

Question 11

A data engineer is troubleshooting an AWS Glue job that reads from Amazon S3 and writes to Amazon Redshift. The job runs successfully but 5% of records are missing after the load. The engineer suspects data consistency issues. Which THREE actions could help diagnose and resolve the problem? (Choose THREE.)

Accepted Answer

Use the Redshift COPY command with a manifest file to load data.. Option A is correct because using the Redshift COPY command with a manifest file explicitly lists every S3 object to be loaded, preventing silent omissions of files that can occur with prefix-based loads and making missing records easier to detect. Option C is correct because Glue job bookmarks track which S3 files have already been processed, so enabling them prevents files from being skipped or reprocessed and helps identify gaps in the input data that cause missing records. Option E is correct because reviewing the job's CloudWatch Logs surfaces errors, warnings, and skipped-record messages from Glue and Redshift, which is the primary diagnostic step for understanding why 5% of records are missing. Option B does not belong because increasing DPUs only scales compute capacity and does not address data consistency or missing records. Option D does not belong because a staging table with a transaction ensures atomicity of the load but does not by itself diagnose or resolve records being dropped during extraction or copy.

Answer

Increase the number of DPUs for the Glue job.

Answer

Use a staging table in Redshift with a transaction to commit.

Question 12

A company uses AWS Data Pipeline to copy data from DynamoDB to S3 daily. Recently, the pipeline started failing with 'ThrottlingException' errors. The DynamoDB table has on-demand capacity. Which action should be taken to resolve the issue?

Accepted Answer

Configure the pipeline to use a retry strategy with exponential backoff.. ThrottlingException errors in AWS Data Pipeline when reading from DynamoDB indicate that the pipeline's read requests are exceeding the table's available throughput. Since the table uses on-demand capacity, which can handle spikes but has a per-second throughput limit, implementing exponential backoff in the pipeline's retry strategy allows it to reduce request rate upon throttling, aligning with AWS SDK best practices for handling DynamoDB throttling.

Answer

Increase the write capacity units of the DynamoDB table.

Answer

Replace Data Pipeline with AWS Glue using a DynamoDB connector.

Answer

Disable the pipeline's retry logic and increase the timeout.

Question 13

A data engineer needs to transform JSON data from an S3 bucket using AWS Glue. The JSON contains nested arrays and objects. Which Glue transform is best suited for flattening nested structures?

Accepted Answer

Relationalize. The Relationalize transform is specifically designed to flatten nested JSON structures (arrays and objects) into a set of related tables, making it ideal for this use case. It automatically handles complex nesting by creating separate DataFrames for each nested level and linking them via foreign keys, which is exactly what is needed when ingesting JSON with nested arrays and objects into a relational format.

Answer

Unnest

Answer

ResolveChoice

Answer

Map

Question 14

A data engineer is troubleshooting a Kinesis Data Firehose delivery stream that ingests JSON log data from web servers. The stream is configured to transform records with an AWS Lambda function and deliver to an Amazon S3 bucket. Recently, the stream has been failing with 'InvalidData' errors. Which action should the engineer take to resolve the issue?

Accepted Answer

Check the CloudWatch Logs for the Lambda function to identify transformation errors.. The 'InvalidData' error in Kinesis Data Firehose typically indicates that the Lambda function used for data transformation is failing or returning malformed records. By checking the CloudWatch Logs for the Lambda function, the engineer can identify specific transformation errors, such as incorrect JSON parsing, missing fields, or exceptions, which cause Firehose to reject the records. This is the most direct troubleshooting step because Firehose relies on the Lambda function to return valid transformed data in the expected format.

Answer

Verify the S3 bucket policy allows Firehose to write.

Answer

Increase the buffer size and interval in the Firehose delivery stream.

Answer

Change the data format to CSV in the Firehose configuration.

Question 15

A data engineer is setting up an Amazon Kinesis Data Analytics application to process streaming data from a Kinesis data stream named "input-stream". The application uses a reference data source from an S3 bucket. The engineer has attached the IAM policy shown in the exhibit to the application's IAM role. When starting the application, the engineer receives an 'AccessDeniedException' error. Which additional permission is required?

Accepted Answer

s3:GetObject on the S3 bucket containing the reference data. The Kinesis Data Analytics application needs to read reference data from the S3 bucket, which requires the s3:GetObject permission on the bucket and its objects. The error 'AccessDeniedException' indicates the IAM role lacks this specific permission to retrieve the reference data file. Option B correctly adds the missing s3:GetObject action to allow the application to fetch the reference data from S3.

Answer

kinesis:PutRecord on the input stream

Answer

kinesis:CreateStream on the input stream

Answer

kinesis:PutRecords on the input stream