DEA-C01 Data Ingestion and Transformation Practice Test 23 — 15 Questions

Question 1

A data engineering team is ingesting data from multiple sources into Amazon S3 using AWS Glue ETL jobs. The jobs are failing intermittently with the error: 'Task ran out of memory'. The input data size varies widely from 100 MB to 10 GB per job. Which configuration change would best mitigate this issue?

Accepted Answer

Enable job bookmarking to process only incremental data. Option B is correct because enabling job bookmarking in Glue allows the job to process only new data, reducing memory pressure. Option A is wrong because increasing the number of workers adds parallelism but may not address memory per worker. Option C is wrong because switching to Python shell would reduce capability. Option D is wrong because decreasing batch size may help but could slow down processing.

Answer

Increase the number of workers in the Glue job

Answer

Reduce the batch size in the S3 source node

Answer

Change the job type from Spark to Python shell

Question 2

A company wants to ingest real-time clickstream data from a website into Amazon S3 with a maximum latency of 60 seconds. The data volume peaks at 500 MB/s. Which service should they use to buffer and deliver the data to S3?

Accepted Answer

Amazon Kinesis Data Firehose. Option A is correct because Kinesis Data Firehose can buffer incoming data for up to 60 seconds and deliver to S3. Option B is wrong because Kinesis Data Streams stores data for up to 365 days but requires a consumer to write to S3. Option C is wrong because SQS is a message queue, not designed for streaming ingestion. Option D is wrong because Lambda is not a buffer service.

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon Kinesis Data Streams

Answer

AWS Lambda

Question 3

A data engineer is designing a pipeline that ingests JSON logs from an application into Amazon S3. The logs contain a timestamp field. The pipeline must partition the data by date in S3 (e.g., year=2024/month=10/day=01). Which approach minimizes transformation effort?

Accepted Answer

Use Amazon Kinesis Data Firehose with dynamic partitioning. Option D is correct because S3 Batch Operations can copy objects and apply prefix changes, but for real-time partitioning, Kinesis Data Firehose with dynamic partitioning is the best approach. The correct answer is actually Kinesis Data Firehose. However, since the options are limited, the correct answer is D (Kinesis Data Firehose) as it can automatically partition data based on the timestamp. Option A is wrong because Lambda would require custom code. Option B is wrong because Athena is a query service. Option C is wrong because Glue crawlers create metadata, not partitions.

Answer

Use AWS Glue crawlers to infer schema and create partitions

Answer

Use AWS Lambda to process each object and copy to the appropriate prefix

Answer

Use Amazon Athena to create partitions on the existing data

Question 4

A company is using AWS DMS to replicate data from an on-premises Oracle database to Amazon RDS for MySQL. The replication is working, but the target table has a different schema. Which DMS feature should be used to transform the source schema to match the target?

Accepted Answer

Use DMS transformation rules. Option A is correct because DMS supports transformation rules that can change table names, schemas, and columns during migration. Option B is wrong because DMS does not have a built-in schema conversion feature (that's AWS SCT). Option C is wrong because Lambda can be used for custom transformations but adds complexity. Option D is wrong because Glue is a separate ETL service, not integrated with DMS.

Answer

Use AWS Schema Conversion Tool (SCT)

Answer

Use AWS Glue ETL jobs

Answer

Use AWS Lambda triggers

Question 5

A data engineer needs to ingest data from an Amazon S3 bucket into Amazon Redshift for analytics. The data is in CSV format and the Redshift table already exists. Which service can be used to perform this ingestion with minimal configuration?

Accepted Answer

Amazon Redshift COPY command. Option C is correct because Redshift COPY command can directly load data from S3. Option A is wrong because Kinesis Data Firehose is for streaming, not batch. Option B is wrong because Glue is overkill. Option D is wrong because DMS is for database migration.

Answer

AWS Glue

Answer

Amazon Kinesis Data Firehose

Answer

AWS Database Migration Service (DMS)

Question 6

A streaming application sends data to Amazon Kinesis Data Streams. The data must be enriched with reference data from an Amazon DynamoDB table in real-time. Which AWS service can be used to perform this enrichment with minimal latency?

Accepted Answer

Amazon Kinesis Data Analytics for Apache Flink. Option B is correct because Kinesis Data Analytics for Apache Flink can process streaming data and join with DynamoDB in real-time. Option A is wrong because Lambda can enrich but may add latency due to cold starts. Option C is wrong because Kinesis Data Firehose does not support real-time enrichment from DynamoDB. Option D is wrong because Glue is batch-oriented.

Answer

Amazon Kinesis Data Firehose with Lambda transformation

Answer

AWS Lambda function triggered by Kinesis Data Streams

Answer

AWS Glue streaming ETL

Question 7

A data engineering team is designing a data ingestion pipeline for a social media analytics platform. The pipeline must handle up to 100,000 events per second with less than 1 second processing latency. Which TWO services should be used together to meet these requirements?

Accepted Answer

Amazon Kinesis Data Analytics for Apache Flink. A (Kinesis Data Streams) provides the high-throughput ingestion layer, and C (Kinesis Data Analytics for Apache Flink) provides low-latency stream processing. B (SQS) is a queue, not optimized for high-throughput streams. D (Firehose) has higher latency (buffering). E (Glue) is batch-oriented.

Answer

AWS Glue streaming ETL

Answer

Amazon Kinesis Data Firehose

Answer

Amazon SQS

Question 8

A company is migrating its data warehouse from on-premises to Amazon Redshift. The migration involves copying 50 TB of data from an S3 bucket to Redshift. The network bandwidth is limited to 1 Gbps. Which TWO approaches should the team use to complete the transfer within 7 days?

Accepted Answer

Use Amazon S3 Transfer Acceleration. A (Snowball Edge) allows physical transfer of the data, bypassing network limitations. D (S3 Transfer Acceleration) optimizes network transfer but may not be sufficient alone. B (Direct Connect) helps but still limited by 1 Gbps. C (Kinesis Firehose) is for streaming. E (Lambda) is not for bulk transfer.

Answer

Use AWS Direct Connect with 10 Gbps

Answer

Use AWS Lambda to copy data in parallel

Answer

Use Amazon Kinesis Data Firehose

Question 9

A data engineer needs to ingest JSON files from an Amazon S3 bucket into an Amazon DynamoDB table. The files are uploaded every hour. Which THREE services can be used together to build this ingestion pipeline?

Accepted Answer

Amazon SQS. Amazon SQS is correct because it decouples the ingestion pipeline, allowing S3 Event Notifications to send messages to an SQS queue when new JSON files arrive. AWS Lambda can then poll the SQS queue to process the files and write to DynamoDB, ensuring reliable, asynchronous ingestion without data loss.

Answer

AWS Step Functions

Answer

Amazon DynamoDB Streams

Question 10

Refer to the exhibit. A data engineer runs an AWS Glue ETL job that writes output to an S3 bucket. The job fails with the error shown. What is the most likely cause?

Accepted Answer

The IAM role used by the Glue job lacks the s3:PutObject permission for the output bucket. Option A is correct because the error message indicates the GlueServiceRole does not have s3:PutObject permission on the bucket. Option B is wrong because the error is about permissions, not the file name. Option C is wrong because Parquet format is not the issue. Option D is wrong because the bucket exists (error shows it in the ARN).

Answer

The Glue job attempted to write data in an unsupported format

Answer

The S3 bucket does not exist

Answer

The output file name contains invalid characters

Question 11

Refer to the exhibit. A data engineer is troubleshooting an AWS Lambda function that reads from an S3 bucket and writes to a Kinesis Data Stream. The Lambda function fails with an AccessDeniedException when calling the kinesis:PutRecords API. Which change is needed to the IAM policy?

Accepted Answer

Change the resource ARN for Kinesis to a wildcard. Option B is correct because the policy allows kinesis:PutRecord and kinesis:PutRecords, but the resource ARN is missing the stream name in the correct format. The ARN should be arn:aws:kinesis:us-east-1:123456789012:stream/input-stream, which is correct. However, the error might be due to missing permissions for the stream. Actually, the policy looks correct. The most likely issue is that the Lambda execution role does not have the kinesis:PutRecords permission. The exhibit allows it, but maybe the role is different. The correct answer is C: Add the kinesis:PutRecords permission, but it's already there. Wait, the error is AccessDeniedException, so the IAM policy is missing the action. The policy includes kinesis:PutRecords, so the error might be due to resource constraints. The correct answer might be D: Change the resource to a wildcard. But the resource is specific. I think the issue is that the stream name is wrong. But the question says "Refer to the exhibit." and the policy seems to allow PutRecords. Perhaps the Lambda function is trying to write to a different stream. The most straightforward answer is that the Lambda execution role does not have the kinesis:PutRecords permission. But the exhibit shows it does. Hmm. Let me re-read. The policy allows kinesis:PutRecord and kinesis:PutRecords. The resource is a specific stream ARN. If the Lambda is using a different stream, it would fail. So the fix is to update the resource ARN to match the stream. Option B: Add the kinesis:PutRecords permission to the policy (but it's already there). Option D: Change the resource to a wildcard. That would be a common fix. I'll go with D.

Answer

Add s3:PutObject permission to the policy

Answer

Change the resource ARN for Kinesis to include the correct stream name

Answer

Add kinesis:PutRecords permission to the policy

Question 12

Refer to the exhibit. A data engineer is using a Kinesis Data Stream with 2 shards. The producer uses a partition key that is the user ID (a UUID). The consumer is falling behind. Which change would improve throughput?

Accepted Answer

Increase the number of shards. Option B is correct because increasing the number of shards increases the ingestion capacity. Option A is wrong because Kinesis Data Firehose would add latency. Option C is wrong because the partition key is already random (UUID), which distributes data well. Option D is wrong because increasing the retention period does not affect throughput.

Answer

Switch to Kinesis Data Firehose

Answer

Increase the retention period

Answer

Change the partition key to a constant value

Question 13

A company receives streaming clickstream data from its website. The data must be ingested with low latency and transformed in real time before being stored in Amazon S3. Which AWS service combination is most suitable for this use case?

Accepted Answer

Amazon Kinesis Data Streams with Amazon Kinesis Data Analytics. Option B is correct because Amazon Kinesis Data Streams ingests streaming data with low latency, and Kinesis Data Analytics can perform real-time transformations using SQL or Apache Flink. Option A is wrong because Kinesis Data Firehose is a delivery service that can transform data but with higher latency. Option C is wrong because AWS Glue is a batch ETL service. Option D is wrong because Amazon S3 is a storage service, not for real-time transformation.

Answer

Amazon S3 with S3 Object Lambda

Answer

Amazon Kinesis Data Firehose with AWS Lambda for transformation

Answer

AWS Glue jobs triggered by Amazon S3 events

Question 14

A data engineer is designing a data ingestion pipeline to load millions of small JSON files from an on-premises FTP server into Amazon S3. The pipeline should minimize cost and operational overhead. Which approach is most suitable?

Accepted Answer

Deploy AWS DataSync to transfer files from the FTP server to S3. Option C is correct because AWS DataSync can transfer data from on-premises to S3 efficiently, handling many small files. Option A is wrong because S3 Transfer Acceleration speeds up transfers over long distances but requires S3 API access. Option B is wrong because Direct Connect provides dedicated network but still needs a transfer mechanism. Option D is wrong because Snowball Edge is for large data volumes and incurs shipping delays.

Answer

Use S3 Transfer Acceleration to upload files directly from the FTP server

Answer

Use AWS Snowball Edge to ship the data to AWS

Answer

Set up an AWS Direct Connect connection and use AWS CLI to copy files

Question 15

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is transformed using an AWS Lambda function. Recently, the transformation errors have increased due to Lambda timeouts. The data engineer needs to diagnose and resolve the issue without losing data. What should the engineer do?

Accepted Answer

Increase the Lambda function timeout and ensure that failed records are sent to a backup S3 bucket. Option A is correct because increasing the Lambda timeout gives the function more time to complete, and Kinesis Firehose retries failures; data is not lost if the Lambda fails because Firehose can retry or send failed records to a backup S3 bucket. Option B is wrong because Lambda never stores data. Option C is wrong because CloudWatch Logs captures logs but does not store failed records. Option D is wrong because Lambda cannot write to SQS directly from Firehose transformation; Firehose handles failed records internally.

Answer

Enable Amazon CloudWatch Logs for the Lambda function to capture errors and store failed records in CloudWatch

Answer

Configure the Lambda function to write failed records to an Amazon SQS queue for later reprocessing

Answer

Modify the Lambda function to store failed records in Amazon S3 before processing