DEA-C01 Data Ingestion and Transformation Practice Test 32 — 15 Questions

Question 1

Refer to the exhibit. A data engineer runs this CLI command on an S3 bucket. The data is ingested from multiple sources. Which AWS service would be best to process these files in a single batch transformation?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS Glue. AWS Glue is designed for batch ETL processing, capable of handling multiple files of varying sizes in a single transformation job. Option A is wrong because Lambda has limitations on execution time (15 minutes) and memory (10 GB), making it unsuitable for large-scale batch transforms. Option B is wrong because Kinesis Data Analytics is for real-time stream processing, not batch. Option C is wrong because Athena is an interactive query service for ad-hoc analysis of data in S3, not for batch transformations.

Answer

AWS Lambda

Answer

Amazon Kinesis Data Analytics

Answer

Amazon Athena

Question 2

A data engineer needs to ingest streaming data from thousands of IoT devices and immediately process each record with minimal latency. Which AWS service should be used as the ingestion point?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is designed for real-time streaming data ingestion with low latency. AWS Glue is for batch ETL, S3 is object storage, and Lambda is compute but not an ingestion endpoint itself.

Answer

AWS Lambda

Answer

Amazon S3

Answer

AWS Glue

Question 3

A company wants to schedule a nightly batch job to copy data from an on-premises PostgreSQL database to Amazon S3. The solution must minimize operational overhead. Which AWS service should be used?

Accepted Answer

AWS Glue. AWS Glue is the correct choice because it provides a fully managed ETL service that can connect to on-premises PostgreSQL via JDBC, extract data, and write it to Amazon S3 with minimal configuration. Glue's built-in scheduler can run the job nightly, eliminating the need to manage servers or orchestration infrastructure, which directly meets the requirement to minimize operational overhead.

Answer

AWS Data Pipeline

Answer

Amazon EMR

Answer

AWS Database Migration Service (AWS DMS) with ongoing replication

Question 4

A company is ingesting streaming data into Kinesis Data Streams. The consumer application experiences high latency due to a single shard bottleneck. What is the most effective way to reduce latency?

Accepted Answer

Increase the number of shards in the data stream.. Increasing the number of shards increases throughput and reduces latency. Waiting for autoscaling is passive, using KCL is for processing, and switching to Firehose changes the architecture.

Answer

Wait for automatic scaling to add shards.

Answer

Use the Kinesis Client Library (KCL) to process records.

Answer

Switch to Amazon Kinesis Data Firehose.

Question 5

A data engineering team uses AWS Glue ETL jobs to process data daily. They notice that job run times are increasing as data volume grows. Which action will most effectively improve performance without changing the code?

Accepted Answer

Increase the number of DPUs for the Glue job.. Increasing the number of DPUs (Data Processing Units) allocated to the job provides more parallelism and memory, speeding up processing without code changes.

Answer

Use a smaller instance type for the Glue job.

Answer

Enable job bookmark to skip previously processed data.

Answer

Split the data into more files in S3.

Question 6

A company uses Amazon Kinesis Data Firehose to deliver data to an S3 bucket. The data must be delivered within 60 seconds of ingestion. Currently, the delivery takes 3 minutes due to large buffer sizes. How should the engineer adjust the Firehose configuration?

Accepted Answer

Decrease the buffer interval to 60 seconds.. Amazon Kinesis Data Firehose delivers data to S3 based on either a buffer size threshold or a buffer interval (in seconds), whichever is reached first. To ensure delivery within 60 seconds, you must decrease the buffer interval to 60 seconds, which forces Firehose to flush data to S3 every 60 seconds regardless of buffer size. The current 3-minute delay is caused by the buffer interval being larger than 60 seconds, so reducing it directly meets the requirement.

Answer

Increase the buffer interval to 120 seconds.

Answer

Increase the buffer size to 128 MB.

Answer

Decrease the buffer size to 1 MB.

Question 7

A company ingests millions of small files (1-10 KB) into Amazon S3 every hour. These files are then processed by AWS Glue ETL jobs. The Glue jobs are slow because of the overhead of reading many small files. Which strategy will most effectively improve Glue job performance?

Accepted Answer

Use a Lambda function to merge small files into larger ones before Glue processes them.. Grouping small files into larger ones (e.g., by merging in a preprocessing step) reduces the number of file read operations and improves Glue's efficiency. Using S3 Select or increasing DPUs helps but doesn't address the root cause.

Answer

Enable Glue job bookmark.

Answer

Increase the number of DPUs for the Glue job.

Answer

Use S3 Select to filter data before Glue reads it.

Question 8

A data pipeline uses Amazon Kinesis Data Streams with enhanced fan-out consumers. The team notices that one consumer falls behind and data accumulates. Which action will help this consumer catch up without affecting other consumers?

Accepted Answer

Register an enhanced fan-out consumer for the slow consumer.. Registering a new enhanced fan-out consumer with its own dedicated read throughput allows it to catch up independently. Increasing shards affects all consumers, and increasing iterator age may help but doesn't increase throughput.

Answer

Increase the retention period of the stream.

Answer

Increase the number of shards in the stream.

Answer

Use a Lambda consumer instead of an enhanced fan-out consumer.

Question 9

A company uses AWS Glue to transform data in an S3 data lake. The transformation logic requires joining two large datasets that are each hundreds of gigabytes. The Glue job runs out of memory. Which configuration change will most likely resolve this issue?

Accepted Answer

Increase the number of DPUs for the Glue job.. Increasing the number of DPUs provides more memory for the join operation. Glue automatically distributes data across workers, so more workers mean more total memory.

Answer

Repartition the data before the join.

Answer

Use a different file format like Parquet with compression.

Answer

Use the 'spark.sql.autoBroadcastJoinThreshold' setting to broadcast the smaller table.

Question 10

A company wants to ingest streaming data from social media feeds into AWS for real-time analytics. Which TWO services can directly ingest streaming data without writing custom code? (Choose TWO.)

Accepted Answer

Amazon AppFlow. Amazon Kinesis Data Firehose can directly ingest streaming data and deliver to destinations. AWS Glue can stream from Kafka but not directly ingest from social media without custom connectors. Kinesis Data Streams requires producers to send data, not direct ingestion. AppFlow can ingest from SaaS applications including social media.

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Answer

Amazon S3 Transfer Acceleration

Question 11

A data engineer needs to schedule a nightly ETL job that reads from an Amazon RDS database and writes to Amazon S3 in Parquet format. The solution must be serverless and minimize cost. Which TWO AWS services should be used? (Choose TWO.)

Accepted Answer

Amazon S3. AWS Glue can run serverless ETL jobs. Amazon S3 is the destination. Lambda could trigger but not run the ETL itself; Data Pipeline is not serverless; Athena is query-only.

Answer

AWS Data Pipeline

Answer

AWS Lambda

Answer

Amazon Athena

Question 12

A data engineer attached this IAM policy to a Lambda function used to transform data in S3. The function is unable to write output to the bucket. What is the most likely reason?

Accepted Answer

The policy does not allow the s3:ListBucket action on the bucket.. The policy allows GetObject and PutObject on objects, but not the s3:ListBucket action required to check existence or list objects. The function likely needs ListBucket to write or verify.

Answer

The resource ARN is missing the bucket-level ARN.

Answer

The policy does not allow the s3:DeleteObject action.

Answer

The policy does not allow the s3:PutObjectAcl action.

Question 13

A data engineer runs the command shown. The consumer application is unable to read data older than 24 hours. What is the most likely cause?

Accepted Answer

The retention period is set to 24 hours, so data older than 24 hours is deleted.. The stream's retention period is 24 hours, meaning data is automatically deleted after 24 hours. The consumer tries to read data older than 24 hours, which is no longer available.

Answer

The shard has reached its maximum sequence number.

Answer

The stream is encrypted with KMS, preventing access.

Answer

The stream is in ACTIVE status but not processing data.

Question 14

A data engineering team needs to ingest streaming data from thousands of IoT devices. The data must be processed in near real-time and stored in Amazon S3 in Apache Parquet format partitioned by device_id and timestamp. Which combination of services should the team use to minimize operational overhead and cost?

Accepted Answer

Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, and optionally AWS Lambda.. Amazon Kinesis Data Streams ingests streaming data from thousands of IoT devices in near real-time. Kinesis Data Firehose delivers the data directly to Amazon S3 with built-in conversion to Apache Parquet and automatic partitioning by device_id and timestamp, minimizing operational overhead. AWS Lambda can optionally be used for lightweight transformations if needed. Option A is incorrect because using Amazon EC2 for processing adds significant operational overhead for scaling and management. Option B is incorrect because Amazon MSK (Kafka) and AWS Glue Streaming require more operational overhead and are not as cost-effective for this simple near-real-time use case. Option D is incorrect because S3 Transfer Acceleration is designed for large file transfers over long distances, not for streaming data ingestion, and AWS Lambda alone cannot handle continuous streaming without a dedicated ingestion service like Kinesis.

Answer

Amazon Kinesis Data Streams, Amazon EC2 for processing, and Amazon S3 with lifecycle policies.

Answer

Amazon MSK (Kafka), AWS Glue Streaming, and Amazon S3.

Answer

Amazon S3 Transfer Acceleration and AWS Lambda for event-driven transformation.

Question 15

A company uses AWS Glue ETL jobs to transform data and load it into Amazon Redshift. The jobs are failing with 'Out of Memory' errors. What is the most cost-effective way to resolve this issue without changing the transformation logic?

Accepted Answer

Increase the number of G.1X workers in the Glue job configuration.. Increasing the number of G.1X workers (DPUs) adds parallelism, allowing the job to handle more data in memory without changing logic, and is cost-effective since G.1X workers are cheaper than G.2X. Option B is wrong: Redshift Spectrum is for querying data directly from S3, not for fixing memory issues in Glue ETL jobs. Option C is wrong: Changing to G.2X workers increases memory per worker but is more expensive than adding more G.1X workers; the goal is cost-effective. Option D is wrong: Switching to Scala does not directly address memory issues and may require code changes.

Answer

Use Amazon Redshift Spectrum to query data directly from S3 without transformation.

Answer

Change the worker type to G.2X and keep the same number of workers.

Answer

Switch the job from Python to Scala.