DEA-C01 Data Ingestion and Transformation Practice Test 16 — 15 Questions

Question 1

A company ingests millions of small files (1-10 KB) into Amazon S3 every hour. These files are then processed by AWS Glue ETL jobs. The Glue jobs are slow because of the overhead of reading many small files. Which strategy will most effectively improve Glue job performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use a Lambda function to merge small files into larger ones before Glue processes them.. The most effective strategy is to use a Lambda function to merge small files into larger ones before Glue processes them. AWS Glue, which uses Apache Spark, incurs significant overhead when reading many small files because each file requires a separate read operation, metadata lookup, and task scheduling. Merging small files into larger ones (e.g., 128 MB or larger) reduces the number of files, thereby decreasing overhead and improving job performance. This approach is a common best practice for optimizing ETL jobs on S3.

Answer

Enable Glue job bookmark.

Answer

Increase the number of DPUs for the Glue job.

Answer

Use S3 Select to filter data before Glue reads it.

Question 2

A data pipeline uses Amazon Kinesis Data Streams with enhanced fan-out consumers. The team notices that one consumer falls behind and data accumulates. Which action will help this consumer catch up without affecting other consumers?

Accepted Answer

Register an enhanced fan-out consumer for the slow consumer.. Enhanced fan-out gives each registered consumer its own dedicated 2 MB/sec read throughput per shard over HTTP/2, so registering the slow consumer as an enhanced fan-out consumer isolates its read capacity from other consumers and lets it catch up without stealing throughput from them. This is the only option that directly addresses per-consumer throughput contention.

Answer

Increase the retention period of the stream.

Answer

Increase the number of shards in the stream.

Answer

Use a Lambda consumer instead of an enhanced fan-out consumer.

Question 3

A company uses AWS Glue to transform data in an S3 data lake. The transformation logic requires joining two large datasets that are each hundreds of gigabytes. The Glue job runs out of memory. Which configuration change will most likely resolve this issue?

Accepted Answer

Increase the number of DPUs for the Glue job.. Increasing the number of DPUs for the Glue job is the most likely configuration change to resolve an out-of-memory error when joining two large datasets. AWS Glue DPUs (Data Processing Units) provide compute resources and memory for the job. When a job runs out of memory, it indicates that the current allocation is insufficient for the data volume and transformation complexity. Adding more DPUs increases the total memory and processing power available to the job, allowing it to handle larger joins without running out of memory.

Answer

Repartition the data before the join.

Answer

Use a different file format like Parquet with compression.

Answer

Use the 'spark.sql.autoBroadcastJoinThreshold' setting to broadcast the smaller table.

Question 4

A company wants to ingest streaming data from social media feeds into AWS for real-time analytics. Which TWO services can directly ingest streaming data without writing custom code? (Choose TWO.)

Accepted Answer

Amazon AppFlow. Amazon Kinesis Data Firehose can directly ingest streaming data and deliver to destinations. AWS Glue can stream from Kafka but not directly ingest from social media without custom connectors. Kinesis Data Streams requires producers to send data, not direct ingestion. AppFlow can ingest from SaaS applications including social media.

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Answer

Amazon S3 Transfer Acceleration

Question 5

A data engineer needs to schedule a nightly ETL job that reads from an Amazon RDS database and writes to Amazon S3 in Parquet format. The solution must be serverless and minimize cost. Which TWO AWS services should be used? (Choose TWO.)

Accepted Answer

Amazon S3. Option E (AWS Glue) is correct because it is a fully serverless, managed ETL service that can run nightly jobs via triggers/schedules, read from Amazon RDS using JDBC connections, and write output to Amazon S3 in Parquet format using its built-in transforms and crawlers, with pay-as-you-go pricing that minimizes cost for periodic jobs. Option D (Amazon S3) is correct because it is the required serverless, low-cost destination storage for the Parquet files produced by the ETL job. Option A (AWS Data Pipeline) is not appropriate because it is a legacy orchestration service that relies on EC2 instances and is not the preferred serverless, cost-minimizing choice. Option B (AWS Lambda) is not suitable as the primary ETL engine because its 15-minute execution limit and memory constraints make it a poor fit for large RDS-to-S3 Parquet transformations. Option C (Amazon Athena) is a serverless query service for analyzing data in S3, not an ETL scheduler or transformation engine for writing Parquet output.

Answer

AWS Data Pipeline

Answer

AWS Lambda

Answer

Amazon Athena

Question 6

A data engineer attached this IAM policy to a Lambda function used to transform data in S3. The function is unable to write output to the bucket. What is the most likely reason?

Accepted Answer

The policy does not allow the s3:ListBucket action on the bucket.. The policy allows GetObject and PutObject on objects, but not the s3:ListBucket action required to check existence or list objects. The function likely needs ListBucket to write or verify.

Answer

The resource ARN is missing the bucket-level ARN.

Answer

The policy does not allow the s3:DeleteObject action.

Answer

The policy does not allow the s3:PutObjectAcl action.

Question 7

A data engineer runs the command shown. The consumer application is unable to read data older than 24 hours. What is the most likely cause?

Accepted Answer

The retention period is set to 24 hours, so data older than 24 hours is deleted.. Kinesis Data Streams stores records for a configurable retention period, defaulting to 24 hours; once that window passes, records are automatically aged out and consumers cannot retrieve them regardless of shard state or encryption. The symptom — inability to read data older than 24 hours — maps directly to the default retention setting.

Answer

The shard has reached its maximum sequence number.

Answer

The stream is encrypted with KMS, preventing access.

Answer

The stream is in ACTIVE status but not processing data.

Question 8

A data engineering team needs to ingest streaming data from thousands of IoT devices. The data must be processed in near real-time and stored in Amazon S3 in Apache Parquet format partitioned by device_id and timestamp. Which combination of services should the team use to minimize operational overhead and cost?

Accepted Answer

Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, and optionally AWS Lambda.. Amazon Kinesis Data Streams can ingest data from thousands of IoT devices, and Kinesis Data Firehose can deliver that data to Amazon S3 in Parquet format with partitioning by device_id and timestamp. Firehose handles the conversion to Parquet and partitioning automatically, minimizing operational overhead. Optionally, AWS Lambda can be used for custom processing. This combination is serverless and cost-effective.

Answer

Amazon Kinesis Data Streams, Amazon EC2 for processing, and Amazon S3 with lifecycle policies.

Answer

Amazon MSK (Kafka), AWS Glue Streaming, and Amazon S3.

Answer

Amazon S3 Transfer Acceleration and AWS Lambda for event-driven transformation.

Question 9

A company uses AWS Glue ETL jobs to transform data and load it into Amazon Redshift. The jobs are failing with 'Out of Memory' errors. What is the most cost-effective way to resolve this issue without changing the transformation logic?

Accepted Answer

Increase the number of G.1X workers in the Glue job configuration.. Increasing the number of G.1X workers (DPUs) adds parallelism, allowing the job to handle more data in memory without changing logic, and is cost-effective since G.1X workers are cheaper than G.2X. Option B is wrong: Redshift Spectrum is for querying data directly from S3, not for fixing memory issues in Glue ETL jobs. Option C is wrong: Changing to G.2X workers increases memory per worker but is more expensive than adding more G.1X workers; the goal is cost-effective. Option D is wrong: Switching to Scala does not directly address memory issues and may require code changes.

Answer

Use Amazon Redshift Spectrum to query data directly from S3 without transformation.

Answer

Change the worker type to G.2X and keep the same number of workers.

Answer

Switch the job from Python to Scala.

Question 10

A company needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The network bandwidth is limited to 1 Gbps, and the transfer must complete within 10 days. The data is compressible. Which solution is MOST appropriate?

Accepted Answer

Use AWS DataSync over a Direct Connect connection.. With a 1 Gbps link, transferring 50 TB would take approximately 111 hours (about 4.6 days) theoretically, well within the 10-day window. AWS DataSync over a Direct Connect connection provides a high-speed, secure, and reliable method to transfer large datasets online. DataSync automates the HDFS-to-S3 transfer, handles compression, and can saturate the 1 Gbps link, making it the most appropriate solution. Snowball Edge involves physical shipment and logistics, which would likely exceed the 10-day deadline. Options B and C also rely on internet bandwidth and are less efficient than DataSync.

Answer

Use Amazon S3 Transfer Acceleration with multipart uploads.

Answer

Use multiple parallel AWS CLI sync commands over the internet.

Answer

Use AWS Snowball Edge to physically ship the data.

Question 11

A company uses Amazon Kinesis Data Firehose to deliver streaming data to an Amazon S3 bucket. The delivery occasionally fails due to 'ThrottlingException' from S3. What should the team do to resolve this issue without losing data?

Accepted Answer

Increase the Firehose buffer size and buffer interval to reduce the number of S3 PUT requests.. Increasing the Firehose buffer size and buffer interval allows Firehose to accumulate more data before flushing to S3, which reduces the frequency of S3 PUT requests and therefore lowers the chance of hitting S3 request-rate throttling. This is the recommended tuning approach because Firehose automatically retries failed deliveries, so no data is lost as long as the destination isn't persistently throttling. The larger buffer smooths out the request rate while preserving delivery guarantees.

Answer

Enable S3 Transfer Acceleration on the destination bucket.

Answer

Disable error logging in Firehose to reduce API calls.

Answer

Configure Firehose to deliver data to Amazon DynamoDB instead.

Question 12

A company uses AWS DMS to migrate an on-premises Oracle database to Amazon RDS for PostgreSQL. The migration completes, but the target table has more rows than the source. Which is the MOST likely cause?

Accepted Answer

Target tables lack unique constraints, causing DMS to insert duplicate rows.. When DMS performs a full load into a target table that lacks a primary key or unique constraint, it cannot detect or prevent duplicate rows, so re-running the task or overlapping CDC can insert the same source row multiple times, inflating the target row count. Adding a primary key or unique index on the target (and enabling the appropriate DMS task settings) prevents this.

Answer

DMS used binary replication which included extra metadata rows.

Answer

Oracle and PostgreSQL handle case sensitivity differently.

Answer

DMS performed a full load instead of ongoing replication.

Question 13

A company uses AWS Lambda to process events from an S3 bucket. The Lambda function writes transformed data to another S3 bucket. Occasionally, the Lambda invocation fails with 'ResourceNotFoundException'. What is the MOST likely cause?

Accepted Answer

The destination S3 bucket does not exist or the Lambda function's IAM role lacks permissions.. The destination bucket may not exist or the Lambda function's IAM role lacks permissions to write to it. Option A is wrong because Lambda timeouts would cause 'Timeout' error. Option C is wrong because S3 event notifications are reliable. Option D is wrong because the source bucket exists since it triggered the event.

Answer

The Lambda function timed out.

Answer

The S3 event notification is misconfigured.

Answer

The source S3 bucket has versioning disabled.

Question 14

A company uses AWS Glue to process large datasets. The Glue job occasionally fails with 'DiskFull' errors. Which TWO actions should the engineer take to resolve this issue? (Choose two.)

Accepted Answer

Increase the number of workers for the Glue job.. Option A is correct because adding more workers to an AWS Glue job distributes the data processing across additional executors, so each worker handles a smaller partition of the dataset and consumes less local scratch disk space, directly mitigating 'DiskFull' errors. Option E is correct because switching to the G.2X worker type gives each worker more memory and, critically, more local EBS/scratch disk capacity than the standard G.1X worker, allowing shuffle and spill operations to complete without exhausting disk. Option B is not the right fix because Glue/Spark already spills shuffle data to Amazon S3 via the Spark shuffle manager, and manually redirecting intermediate results is not a supported configuration knob for resolving DiskFull. Option C is incorrect because job bookmarks only track previously processed data to avoid reprocessing; they reduce runtime and cost but do not increase available disk space. Option D is incorrect because Python shell jobs run single-node scripts without the distributed Spark engine, so they cannot process large datasets and would not solve the underlying disk capacity problem.

Answer

Store intermediate results in Amazon S3 instead of local disk.

Answer

Enable job bookmark to skip already processed data.

Answer

Use a Python shell job instead of Spark.

Question 15

A company needs to ingest streaming data from an existing Amazon Kinesis Data Streams into Amazon S3 with partitioning by date. Which TWO services can accomplish this with minimal coding? (Choose two.)

Accepted Answer

Amazon Kinesis Data Firehose

Answer

AWS Glue Streaming

Answer

Amazon Kinesis Data Analytics

Answer

Amazon S3 Transfer Acceleration