DEA-C01 Data Ingestion and Transformation Practice Test 7 — 15 Questions

Question 1

Refer to the exhibit. An AWS Glue ETL job is failing with an OutOfMemoryError. The job reads from Amazon S3 and performs a GROUP BY on a large dataset. Which change should the data engineer make to resolve this error?

Accepted Answer

Increase the number of DPUs allocated to the Glue job.. Option C is correct. Increasing the number of DPUs provides more memory per executor. Option A is wrong because increasing parallelism may help but requires more cores, not just partitions. Option B is wrong because using coalesce reduces partitions but may cause data skew. Option D is wrong because increasing partition count may worsen memory issues.

Answer

Use coalesce to reduce the number of partitions.

Answer

Increase the number of partitions in the DataFrame.

Answer

Use repartition to increase the number of partitions.

Question 2

A data engineer needs to ingest streaming data from thousands of IoT devices into AWS for real-time processing. The data volume peaks at 5 GB/min. Which AWS service should be used as the ingestion endpoint?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is designed for real-time streaming data ingestion. Option A is wrong because S3 is for object storage, not real-time streaming. Option B is wrong because Lambda is compute, not an ingestion endpoint. Option D is wrong because Glue is for ETL jobs.

Answer

AWS Glue

Answer

Amazon S3

Answer

AWS Lambda

Question 3

A company uses AWS DMS to migrate an on-premises PostgreSQL database to Amazon RDS for PostgreSQL. After initial load, ongoing replication is set up. The replication task shows 'Task status: failed with error: The specified LSN is not available in the source database logs.' What is the most likely cause?

Accepted Answer

The source database's WAL retention period is too short, and required logs have been purged.. The error indicates that the required WAL logs have been removed or are not accessible. Option A is correct because DMS needs WAL logs for CDC. Option B is wrong because network connectivity would cause a different error. Option C is wrong because DMS supports PostgreSQL. Option D is wrong because ongoing replication does not need a full load.

Answer

DMS does not support PostgreSQL as a source for ongoing replication.

Answer

The source database's network security group blocks outbound traffic to DMS.

Answer

The full load was incomplete, preventing CDC from starting.

Question 4

A data engineer is designing a data ingestion pipeline for clickstream data that arrives in bursts, up to 100 MB/s, and must be processed with exactly-once semantics. The data must be stored in Amazon S3 partitioned by event date and hour. Which combination of services should the engineer use?

Accepted Answer

Amazon Kinesis Data Firehose with S3 destination and dynamic partitioning.. Amazon Kinesis Data Firehose can buffer and batch data to S3 with partitioning. Option A is correct. Option B is wrong because Lambda cold starts can cause latency. Option C is wrong because Glue is for batch ETL, not real-time. Option D is wrong because KCL requires custom application.

Answer

Amazon Kinesis Data Streams with AWS Lambda consumer writing to S3.

Answer

AWS Glue streaming ETL job reading from Amazon MSK and writing to S3.

Answer

Amazon Kinesis Data Streams with KCL application writing to S3.

Question 5

A company needs to ingest CSV files from an FTP server into Amazon S3 daily. The files are typically 50 MB each, and the process should be fully managed with minimal operational overhead. Which AWS service should be used?

Accepted Answer

AWS Transfer Family. AWS Transfer Family provides fully managed SFTP/FTP to S3 ingestion. Option A is correct. Option B is wrong because DataSync is for server-to-server transfers, not FTP. Option C is wrong because AppFlow is for SaaS applications. Option D is wrong because Lambda is not a managed ingestion service.

Answer

AWS Lambda with FTP library

Answer

AWS DataSync

Answer

Amazon AppFlow

Question 6

A data engineer is using AWS Glue ETL to transform a large dataset in S3. The job processes 2 TB of data daily and currently runs for 6 hours. The engineer wants to reduce runtime without changing the transformation logic. What is the best approach?

Accepted Answer

Increase the number of Glue DPUs or enable auto-scaling.. Increasing DPUs or enabling auto-scaling can improve performance. Option A is correct. Option B is wrong because reducing DPUs would slow the job. Option C is wrong because using a different engine may not be compatible. Option D is wrong because Spark UI is for monitoring, not performance improvement.

Answer

Reduce the number of DPUs to minimize overhead.

Answer

Use the Spark UI to analyze bottlenecks and rewrite code.

Answer

Switch from Spark to Python shell.

Question 7

A company uses Amazon Kinesis Data Analytics (now Managed Service for Apache Flink) to run a Flink application on streaming data. The application fails with 'OutOfMemoryError: Java heap space'. The data volume is 10 MB/s. What is the most likely cause and solution?

Accepted Answer

The application's Parallelism is too low; increase the number of Parallelism and KPUs.. Insufficient Parallelism or KPU allocation leads to OOM. Option A is correct. Option B is wrong because checkpointing actually helps. Option C is wrong because Flink can handle 10 MB/s with proper resources. Option D is wrong because data format does not cause OOM.

Answer

The data contains records larger than 1 MB; split records into smaller chunks.

Answer

Checkpointing is enabled too frequently; reduce checkpoint interval.

Answer

The Flink application is not suitable for 10 MB/s throughput; use Kinesis Data Firehose instead.

Question 8

A data engineer needs to transform JSON data from Amazon S3 into Parquet format using AWS Glue. The source files are in a bucket with thousands of small files. What is the best practice to optimize the Glue job performance?

Accepted Answer

Enable 'Group small files' in the Glue job or use a DynamicFrame with coalesce.. Grouping small files into fewer partitions reduces overhead. Option A is correct. Option B is wrong because increasing DPUs may not help with small files. Option C is wrong because converting to CSV adds extra step. Option D is wrong because Parquet is more efficient for Glue.

Answer

Convert the JSON files to CSV before processing with Glue.

Answer

Use an AWS Lambda function to pre-process the files.

Answer

Increase the number of DPUs to the maximum.

Question 9

A company uses AWS DMS to migrate data from Oracle to Aurora MySQL. During the ongoing replication, the target table shows duplicate primary key errors. What is the most likely cause?

Accepted Answer

The target table has an auto-increment column, and DMS is inserting explicit values that conflict.. Duplicate PK errors often occur when the target table has an auto-increment column that conflicts with DMS inserts. Option A is correct. Option B is wrong because LOB mode doesn't cause duplicates. Option C is wrong because full LOB mode is for large objects. Option D is wrong because parallel threads can cause duplicates if not properly configured.

Answer

DMS is using 'Limited LOB mode' and truncating LOB data, causing row mismatches.

Answer

The source table has a trigger that inserts additional rows.

Answer

The DMS task is configured with 'Parallel apply' threads that cause race conditions.

Question 10

A data engineer is building a real-time data pipeline using Amazon Kinesis Data Streams with a Lambda consumer. The data volume is 2 MB/s with average record size of 5 KB. The Lambda function processes records and writes to DynamoDB. Occasionally, the Lambda function fails with 'ProvisionedThroughputExceededException' on DynamoDB. What is the best way to handle this?

Accepted Answer

Configure a Lambda destination on failure to send records to an SQS dead-letter queue, and implement retry logic in Lambda.. Implementing a dead-letter queue and retry logic with exponential backoff is best practice. Option A is correct. Option B is wrong because increasing Lambda concurrency may worsen throttling. Option C is wrong because KCL adds complexity. Option D is wrong because DynamoDB Streams is for change data capture, not for throttling errors.

Answer

Replace Lambda with the Kinesis Client Library (KCL) running on EC2.

Answer

Increase the Lambda function's reserved concurrency to process more records in parallel.

Answer

Use DynamoDB Streams to capture the records and process them asynchronously.

Question 11

A data engineer needs to ingest data from a SaaS application (Salesforce) into Amazon S3 on a daily basis. Which TWO AWS services can be used for this purpose? (Choose TWO.)

Accepted Answer

AWS Glue. Amazon AppFlow natively integrates with Salesforce and can write to S3. AWS Glue can also connect via JDBC. Option A is correct. Option B is wrong because Kinesis Data Streams is for real-time streaming, not batch from SaaS. Option C is correct. Option D is wrong because Transfer Family is for FTP. Option E is wrong because DataSync is for file/object storage, not SaaS.

Answer

AWS DataSync

Answer

Amazon Kinesis Data Streams

Answer

AWS Transfer Family

Question 12

A company is using AWS Glue ETL to process data from Amazon RDS for MySQL to Amazon S3. The job runs daily and takes 2 hours to complete. The engineer wants to improve performance without increasing cost significantly. Which TWO actions should the engineer take? (Choose TWO.)

Accepted Answer

Add a partition column to the source table based on a date column.. Increasing the number of DPUs improves performance but increases cost; however, it's a common approach. Partitioning the source table helps parallel reads. Option A is correct. Option B is correct. Option C is wrong because reducing worker type would slow performance. Option D is wrong because DynamicFrame is recommended for Glue. Option E is wrong because AWS Glue does not support auto-scaling by default without additional configuration.

Answer

Switch to a smaller worker type (e.g., G.1X instead of G.2X).

Answer

Use Spark DataFrames instead of DynamicFrames.

Answer

Enable 'Auto Scaling' in the Glue job configuration.

Question 13

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams with multiple consumers. The data must be processed by a Lambda function for real-time alerts and also stored in Amazon S3 for historical analysis. Which THREE components are needed to implement this architecture? (Choose THREE.)

Accepted Answer

Amazon Kinesis Data Firehose. Kinesis Data Streams is the ingestion layer. Lambda can be a consumer for real-time alerts. Kinesis Data Firehose can read from the stream and write to S3. Option A is correct. Option B is correct. Option D is correct. Option C is wrong because Kinesis Data Analytics is for running SQL or Flink, not needed here. Option E is wrong because SQS is for decoupling, not needed for this pattern.

Answer

Amazon Kinesis Data Analytics

Answer

Amazon SQS queue

Question 14

A data engineer has an IAM policy attached to an IAM role used by an AWS Glue job. The Glue job needs to read from S3 bucket 'data-bucket' and write to the same bucket. The job fails with an access denied error when trying to write to S3. What is the issue?

Accepted Answer

The policy does not include s3:ListBucket permission, which Glue may need.. The policy allows s3:PutObject on the bucket, but Glue jobs also need permissions on the bucket itself (s3:ListBucket) for certain operations. Also, the resource is correct. However, the most common issue is that the IAM role does not have permission to decrypt the KMS key if the bucket uses SSE-KMS. But the exhibit does not mention KMS. Option A is wrong because the resource is correct. Option B is wrong because the actions are correct. Option C is correct because the policy does not include s3:ListBucket, which may be required by Glue to list objects. Option D is wrong because Glue can assume roles.

Answer

The Glue job cannot assume the IAM role because of trust policy.

Answer

The resource ARN for S3 is missing the bucket-level permission.

Answer

The actions for S3 are incorrect; s3:PutObject is not sufficient.

Question 15

A data engineer is troubleshooting a Lambda function that reads from the Kinesis stream 'my-data-stream'. The Lambda function is able to read data but occasionally fails with 'KMS.AccessDeniedException'. What is the most likely cause?

Accepted Answer

The Lambda function's execution role does not have kms:Decrypt permission for the KMS key.. The stream uses KMS encryption. The Lambda function's IAM role likely lacks permission to decrypt using the KMS key. Option A is correct. Option B is wrong because the stream has 1 shard, which is fine. Option C is wrong because Lambda can read from streams. Option D is wrong because retention period is 24 hours, which is fine.

Answer

The retention period is too short; increase it.

Answer

The stream has too few shards; increase shard count.

Answer

The Lambda function is not authorized to consume from Kinesis streams.

DEA-C01 Data Ingestion and Transformation Practice Test 7 — 15 Questions

Exhibit

DEA-C01 Data Ingestion and Transformation Practice Test 7 — 15 Questions

Exhibit