DEA-C01 Data Ingestion and Transformation Practice Test 20 — 15 Questions

Question 1

A data engineer is troubleshooting a Lambda function that reads from a Kinesis Data Stream, processes records, and writes to a Kinesis Data Firehose delivery stream. The Firehose delivery stream is configured to deliver data to an S3 bucket. The Lambda function is failing with an access denied error. The IAM policy attached to the Lambda execution role is shown in the exhibit. Which permission is missing?

Accepted Answer

firehose:DescribeDeliveryStream on the Firehose delivery stream. Option B is correct because the Lambda function needs permission to describe the Firehose delivery stream to get the endpoint, and kinesis:DescribeStream is already allowed for the Kinesis stream, but firehose:DescribeDeliveryStream is missing for the Firehose resource. Option A is wrong because Kinesis actions are allowed. Option C is wrong because s3:PutObject is allowed. Option D is wrong because the function is not writing to CloudWatch Logs; the error is about access to Firehose.

Answer

kinesis:PutRecord on the Kinesis stream

Answer

logs:CreateLogGroup and logs:CreateLogStream on the CloudWatch log group

Answer

s3:PutObjectAcl on the S3 bucket

Question 2

A company wants to migrate on-premises data to Amazon S3 using AWS DataSync. The data is stored on an NFS file server and the total volume is 50 TB. The network bandwidth between the on-premises data center and AWS is 1 Gbps (gigabit per second). What is the primary factor that will determine the total time required for the initial data transfer?

Accepted Answer

The available network bandwidth between on-premises and AWS. Option C is correct because the network bandwidth is the bottleneck for transferring 50 TB over a 1 Gbps link; it will take approximately 4.7 days to transfer if bandwidth is fully utilized, but file metadata operations and number of files also matter. Option A is wrong because DataSync can handle large files efficiently. Option B is wrong because the NFS server performance is usually not the bottleneck compared to network. Option D is wrong because S3 object storage can accept data at high throughput.

Answer

The number of S3 buckets used as the destination

Answer

The average file size in the dataset

Answer

The IOPS (I/O operations per second) of the on-premises NFS server

Question 3

A data engineer runs an AWS Glue ETL job that reads from a large Amazon S3 source (several terabytes of CSV files) and writes transformed data to an S3 bucket in Parquet format. The job fails with the error shown in the exhibit. The job uses the Standard worker type with 10 workers (G.1X). The engineer needs to resolve the failure with minimal cost increase. What should the engineer do?

Accepted Answer

Change the worker type to G.2X with 10 workers.. Option D is correct because the OutOfMemoryError indicates that the Spark executors do not have enough memory; switching to G.2X workers doubles memory per worker (from 16 GB to 32 GB) without increasing the number of workers, which is more cost-effective than increasing the number of workers with G.1X. Option A is wrong because increasing the number of workers with G.1X may not resolve the memory issue per executor; it spreads the data across more executors but each still has limited memory. Option B is wrong because reducing the number of files (coalesce) may not help if the issue is per-task memory. Option C is wrong because using the G.2X worker type with more memory per worker is likely sufficient; G.4X may be unnecessary and more expensive.

Answer

Increase the number of workers to 20 while keeping G.1X worker type.

Answer

Change the worker type to G.4X with 10 workers.

Answer

Set the 'coalesce' parameter to reduce the number of output files.

Question 4

A company wants to ingest data from a SaaS application into Amazon S3. The SaaS application supports streaming data via HTTP POST requests. The data volume is approximately 100 MB per hour, and the company needs to store the raw data in S3 for archival and later analysis. Which approach is the most cost-effective and operationally efficient?

Accepted Answer

Use Amazon Kinesis Data Firehose with HTTP endpoint as the source, and configure S3 as the destination.. Option C is correct because Kinesis Data Firehose can directly accept HTTP POST requests via the Kinesis Agent or direct API calls, and it automatically delivers data to S3 with buffering, requiring no servers to manage. Option A is wrong because running an EC2 instance adds operational overhead and cost even if it's small. Option B is wrong because SQS also requires a consumer (e.g., Lambda) to move data to S3, adding complexity. Option D is wrong because API Gateway and Lambda add unnecessary layers and cost for simple ingestion.

Answer

Launch a t3.nano EC2 instance that runs a script to receive HTTP POST requests and write to S3.

Answer

Use Amazon Simple Queue Service (SQS) to queue the HTTP POST data and have an AWS Lambda function read from SQS and write to S3.

Answer

Use Amazon API Gateway to create a REST API that receives the data and triggers an AWS Lambda function to store it in S3.

Question 5

A data engineer needs to transform JSON data from a Kinesis Data Stream into Parquet format and store it in an S3 data lake. The transformation includes simple field mapping and data type conversions. Which AWS service is the most cost-effective for performing this transformation in near-real-time?

Accepted Answer

AWS Lambda function triggered by Kinesis Data Firehose. Option D is correct because Kinesis Data Firehose can perform data transformation using a Lambda function before delivering to S3, and it supports converting JSON to Parquet using its built-in capability or Lambda. Option A is wrong because AWS Glue is a serverless Spark ETL service, but it is designed for batch processing, not streaming. Option B is wrong because Amazon EMR is a managed Hadoop cluster that is more complex and costly for simple transformations. Option C is wrong because Amazon Athena is an interactive query service, not a transformation engine.

Answer

Amazon Athena with CTAS (CREATE TABLE AS SELECT)

Answer

AWS Glue ETL job

Answer

Amazon EMR with Spark Streaming

Question 6

A data engineer runs an AWS Glue crawler that is configured to crawl an S3 bucket named 'my-data-lake' and update the Glue Data Catalog. The crawler fails with an access denied error. The IAM role attached to the crawler has the policy shown in the exhibit. What is the likely cause of the failure?

Accepted Answer

The policy does not allow logging to CloudWatch Logs.. Option B is correct because the crawler needs permission to write logs to CloudWatch Logs; the policy does not include logs:CreateLogGroup, logs:CreateLogStream, and logs:PutLogEvents. Option A is wrong because the policy includes s3:ListBucket on the bucket and s3:GetObject on the bucket contents. Option C is wrong because the policy includes glue:CreateTable and glue:UpdateTable. Option D is wrong because the crawler does not need to write to S3.

Answer

The policy does not allow glue:CreateTable on the 'my-data-lake' database.

Answer

The policy does not allow s3:PutObject on the 'my-data-lake' bucket.

Answer

The policy does not allow s3:ListBucket on the 'my-data-lake' bucket.

Question 7

A company is using AWS Database Migration Service (DMS) to migrate a 2 TB Oracle database to Amazon Aurora PostgreSQL. The migration must have minimal downtime. The source database is highly active with continuous writes. Which DMS migration type and additional configuration should the engineer use?

Accepted Answer

Use a full load migration task followed by ongoing replication (CDC).. Option D is correct because full load + change data capture (CDC) allows initial bulk load and then continuously replicates ongoing changes to keep the target in sync, minimizing downtime. Option A is wrong because full load only will not capture changes during migration, causing data loss. Option B is wrong because CDC only requires an initial snapshot anyway; starting with CDC assumes existing data is already present, which isn't the case. Option C is wrong because task restart does not address ongoing changes.

Answer

Use a CDC-only migration task to capture changes from the source.

Answer

Use a full load migration task and stop the source database before starting.

Answer

Use a full load migration task with task restart enabled.

Question 8

A data engineer needs to schedule an AWS Glue ETL job to run every hour and process new data that arrives in an S3 bucket. The job should only process files that have been added since the last run. Which approach should the engineer use to track which files have been processed?

Accepted Answer

Enable job bookmarks in the Glue ETL job.. Option B is correct because Glue job bookmarks automatically track the last processed files in S3 and only process new data in subsequent runs. Option A is wrong because S3 Event Notifications can trigger the job but do not track processed files; the job would need to handle deduplication. Option C is wrong because a DynamoDB table would require custom code to track state. Option D is wrong because checking last modified timestamp manually is error-prone and not built-in.

Answer

Configure S3 Event Notifications to trigger the Glue job on each new object creation.

Answer

Store the last processed timestamp in a DynamoDB table and query it at the start of the job.

Answer

Use S3 Inventory to list all objects and filter by last modified date in the job.

Question 9

A data engineer is designing a data ingestion pipeline for real-time clickstream data from a website. The data must be ingested with low latency (seconds) and made available for multiple consumer applications, including a dashboard that refreshes every minute and a machine learning model that processes data in near-real-time. The engineer needs to choose a streaming ingestion service. Which TWO services meet these requirements? (Select TWO.)

Accepted Answer

Amazon Managed Streaming for Apache Kafka (Amazon MSK). Option A (Kinesis Data Streams) provides low-latency ingestion and allows multiple consumers to process data independently via enhanced fan-out. Option D (Amazon MSK) is also a streaming platform that supports multiple consumers with low latency. Option B (Kinesis Data Firehose) is designed for loading data into destinations with buffering, not for multiple consumers. Option C (SQS) is a message queue with at-least-once delivery but is not a streaming platform and does not support replay efficiently. Option E (S3) is object storage, not a streaming ingestion service.

Answer

Amazon Kinesis Data Firehose

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon S3

Question 10

A data engineer is troubleshooting a Kinesis Data Streams consumer application that is falling behind. The stream has 10 shards and is receiving 5 MB/s of data. The consumer uses the Kinesis Client Library (KCL) with a single worker. The worker is processing all 10 shards but is experiencing high latency and checkpointing delays. Which THREE actions should the engineer take to improve consumer performance? (Select THREE.)

Accepted Answer

Increase the number of KCL workers to match the number of shards.. Option A (increase number of workers) allows shard distribution across multiple workers, improving parallelism. Option D (increase KCL maxRecords per call) reduces the number of API calls, improving throughput. Option E (enable enhanced fan-out) dedicates throughput to the consumer, reducing read throttling. Option B is wrong because increasing shards would increase the load on the consumer. Option C is wrong because reducing the checkpoint interval would cause more frequent checkpointing, potentially increasing delays.

Answer

Decrease the checkpoint interval to reduce checkpointing overhead.

Answer

Increase the number of shards in the stream.

Question 11

A data engineer needs to transform data in an S3 data lake using AWS Glue ETL. The data is in CSV format and needs to be converted to Parquet with partitioning by date. The engineer wants to minimize the number of files written to S3 to improve query performance. Which TWO configuration options should the engineer use? (Select TWO.)

Accepted Answer

Use the coalesce method to reduce the number of output partitions.. Option A (use coalesce) reduces the number of output files by merging partitions. Option C (enable partition pruning) helps queries skip irrelevant partitions, improving performance. Option B is wrong because increasing the number of workers may increase the number of output files. Option D is wrong because disabling compression increases file size. Option E is wrong because using dynamic frame with larger partition size does not directly minimize file count.

Answer

Increase the number of workers in the Glue job to increase parallelism.

Answer

Disable compression in the Parquet output.

Answer

Set the 'groupFiles' option to 'inPartition' in the DynamicFrame writer.

Question 12

A company runs an e-commerce platform that generates clickstream data from user interactions on their website. The data is sent as JSON objects via HTTP POST to an API Gateway endpoint, which triggers a Lambda function that writes each record to a Kinesis Data Stream (100 shards). A second Lambda function consumes the stream, transforms the data (enriches with geolocation from a DynamoDB table), and writes to a Kinesis Data Firehose delivery stream that delivers Parquet files to an S3 data lake every 5 minutes. The system has been working for months, but recently the Firehose delivery stream started showing 'DeliveryFailed' errors for a subset of records. The errors point to 'InvalidData' from the Lambda transformation. The engineer reviews the Lambda transformation code and notices that the geolocation lookup occasionally fails because the DynamoDB table has a throttling issue. The engineer needs to handle these failures gracefully so that records that fail enrichment are still delivered to S3 with a null geolocation field, without blocking other records. Which course of action should the engineer take?

Accepted Answer

Modify the Lambda function to catch exceptions during the geolocation lookup, set the geolocation field to null, and continue processing the record.. Option D is correct because modifying the Lambda to catch exceptions, set geolocation to null, and continue processing ensures that failed records are still delivered. Option A is wrong because configuring a dead-letter queue for Firehose is not directly supported; Firehose can send failed records to an S3 bucket for failed data, but that would not include the enriched data. Option B is wrong because increasing DynamoDB read capacity might reduce throttling but does not guarantee no failures; also it increases cost. Option C is wrong because sending failed records to a separate stream adds complexity and does not ensure they are delivered with null geolocation.

Answer

Configure the Kinesis Data Firehose delivery stream to send failed records to a dead-letter queue (DLQ) for later reprocessing.

Answer

Modify the Lambda function to send failed records to a separate Kinesis Data Stream for manual processing.

Answer

Increase the read capacity units (RCUs) on the DynamoDB table to eliminate throttling.

Question 13

A data engineer is responsible for ingesting log files from a fleet of on-premises servers into Amazon S3 for central analysis. Each server generates log files that are rotated every hour, resulting in files of about 500 MB each. The total daily data volume is approximately 1 TB. The network connection between the on-premises data center and AWS is a 100 Mbps VPN. The engineer needs to ensure that all log files are transferred to S3 within 24 hours of generation without data loss. The engineer is considering using AWS DataSync. However, the initial setup shows that the transfer speed is insufficient to meet the 24-hour SLA. What should the engineer do to meet the requirement?

Accepted Answer

Configure AWS DataSync to run on a schedule with incremental transfers and enable data compression.. Option C is correct because AWS DataSync supports scheduling tasks incrementally; after the initial full sync, only new and changed files are transferred, which will fit within the bandwidth. Option A is wrong because ordering a Snowball Edge would take days to arrive and is not suitable for ongoing daily transfers. Option B is wrong because increasing bandwidth may not be feasible or cost-effective. Option D is wrong because compressing files may reduce size but adds complexity and does not solve the network bottleneck; incremental sync is more effective.

Answer

Use AWS CLI with multipart uploads and parallel threads to maximize throughput.

Answer

Contact the network team to upgrade the VPN bandwidth to at least 1 Gbps.

Answer

Order an AWS Snowball Edge device to transfer the initial data and then use DataSync for incremental changes.

Question 14

A data engineer is setting up a data pipeline to ingest data from an Amazon RDS for MySQL database into Amazon S3 using AWS Glue ETL. The Glue job uses a JDBC connection to read from the MySQL database. The job runs successfully, but the engineer notices that the job is taking longer than expected. The MySQL database is 500 GB in size and the Glue job uses 10 workers of type G.1X. The engineer wants to improve the performance of the extraction phase. The database is actively used by other applications, so the engineer must minimize the impact on the source database. Which approach should the engineer take?

Accepted Answer

Partition the table by a numeric column, such as the primary key, and use the 'hashex' or 'hashpar' partitioning option in the Glue JDBC connection.. Option A is correct because partitioning the table on a key column (e.g., primary key) allows Glue to read in parallel from multiple partitions, reducing the load on the database and improving performance. Option B is wrong because increasing workers without partitioning may cause the database to be overwhelmed with connections. Option C is wrong because using a read replica offloads the read traffic but still requires efficient partitioning; also, setting up a read replica adds cost and complexity. Option D is wrong because full table scans are inefficient; incremental loads are for ongoing changes, not initial extraction.

Answer

Use an incremental extraction strategy with a watermark column to reduce the amount of data read each time.

Answer

Create a read replica of the MySQL database and configure the Glue job to read from the replica.

Answer

Increase the number of Glue workers to 20 to increase parallelism.

Question 15

A company uses AWS Glue to process data from multiple sources. The data is stored in an Amazon S3 data lake. The company needs to transform the data using a custom Python library that is not available in the default Glue environment. What is the MOST efficient way to make this library available to the Glue jobs?

Accepted Answer

Upload the library as a .whl file to Amazon S3 and reference it in the Glue job's --additional-python-modules parameter.. Option D is correct because AWS Glue supports Python shell jobs and allows installing additional Python modules via --additional-python-modules or by providing a .whl file in S3 and referencing it. Option A is wrong because Glue does not support installing packages via pip at runtime. Option B is wrong because creating a custom Docker image is for Glue for Ray or for ETL jobs with custom environment, but it's more complex than needed. Option C is wrong because installing the library on every node is not supported; Glue manages the environment.

Answer

Manually install the library on each node in the Glue cluster by editing the bootstrap script.

Answer

Create a custom Docker image with the library and use it in AWS Glue for Ray.

Answer

Use a shell command in the Glue job script to run 'pip install <library>' before the job runs.

DEA-C01 Data Ingestion and Transformation Practice Test 20 — 15 Questions

Exhibit

DEA-C01 Data Ingestion and Transformation Practice Test 20 — 15 Questions

Exhibit