DEA-C01 Data Ingestion and Transformation Practice Test 34 — 15 Questions

Question 1

A data engineer is building a data ingestion pipeline that uses AWS Lambda to process records from Amazon Kinesis Data Streams. The Lambda function writes the processed data to Amazon DynamoDB. Which TWO factors affect the maximum number of concurrent Lambda executions for this stream? (Choose TWO.)

Accepted Answer

Batch size configured for the Lambda event source mapping. Correct options: A and C. The number of shards determines the maximum number of concurrent batches (one per shard). The batch size (records per invocation) also affects concurrency because smaller batches lead to more invocations. Option B is wrong because DynamoDB table type does not affect Lambda concurrency. Option D is wrong because Lambda memory is not directly related to concurrency. Option E is wrong because stream name does not affect concurrency.

Answer

DynamoDB table's read capacity units

Answer

Lambda function's memory allocation

Answer

Kinesis stream name

Question 2

Refer to the exhibit. A data engineer created this IAM policy for a Lambda function that reads from a Kinesis stream and writes to an S3 bucket. The Lambda function fails with an 'AccessDenied' error when trying to write to S3. What is the missing permission?

Accepted Answer

s3:ListBucket on the bucket. Option C is correct because the Lambda function needs permission to list the bucket (s3:ListBucket) to verify the bucket exists and possibly to check for existing objects. The 's3:PutObject' action is allowed, but without 's3:ListBucket', the function may fail when trying to write if the key does not exist or for bucket-level operations. Option A is wrong because 's3:GetObject' is already present. Option B is wrong because 's3:DeleteObject' is not needed. Option D is wrong because 's3:PutObjectAcl' is not required.

Answer

s3:GetObject on the bucket

Answer

s3:PutObjectAcl on the bucket

Answer

s3:DeleteObject on the bucket

Question 3

Refer to the exhibit. A data engineer runs this AWS Glue job but it fails with an error that the table 'orders' does not exist in the 'sales_db' database. The engineer has verified that the table exists in the AWS Glue Data Catalog. What is the most likely cause of the error?

Accepted Answer

The IAM role used by the Glue job does not have permission to read the Data Catalog. Option B is correct because the Glue job needs permissions to access the Data Catalog. The IAM role associated with the job must have 'glue:GetTable' permission. Option A is wrong because the error is about the table not existing, not about the S3 path. Option C is wrong because the script is correct syntactically. Option D is wrong because the job bookmark setting does not affect table discovery.

Answer

The Glue job has job bookmark enabled and is skipping the table

Answer

The script uses 'create_dynamic_frame.from_catalog' incorrectly

Answer

The S3 path 's3://data-lake/raw/' does not exist

Question 4

Refer to the exhibit. A data engineer is configuring a Kinesis Data Firehose delivery stream. The stream is expected to receive bursts of 10 MB of data every 2 minutes. What is the maximum time it will take for data to be delivered to S3 during a burst?

Accepted Answer

300 seconds. Option B is correct. The buffer interval is 300 seconds (5 minutes) and the buffer size is 5 MB. During a burst of 10 MB every 2 minutes, the buffer will fill to 5 MB in 1 minute (since 10 MB/2 min = 5 MB/min), so the buffer will be flushed every minute due to size reaching 5 MB. However, the buffer interval is 300 seconds, which is the maximum time. The actual delivery time will be the minimum of the size and interval triggers. Since the size triggers first, data will be delivered within 1 minute of the burst. But the question asks for maximum time; if the burst is continuous, the buffer size will trigger every 1 minute. However, the interval is 300 seconds, so the maximum time could be up to 5 minutes if there is no data. But during a burst, the buffer will flush sooner. The correct answer is that the maximum time is 300 seconds (5 minutes) because that is the configured interval. But since the buffer size is small, data will be flushed sooner. However, the question asks for maximum time during a burst; if the burst is exactly 10 MB every 2 minutes, the buffer will fill to 5 MB in 1 minute, so delivery will occur within 1 minute. But the maximum possible is the interval of 300 seconds. Considering the wording, the maximum time is 300 seconds. Option B is correct.

Answer

60 seconds

Answer

1 second

Answer

600 seconds

Question 5

A company streams clickstream data from websites to Amazon Kinesis Data Streams. A Lambda function processes each record and writes it to Amazon S3. Recently, the function has been timing out under high load. Which solution should a data engineer implement to handle the increased throughput?

Accepted Answer

Increase the number of shards in the Kinesis data stream.. Option C is correct because increasing the number of shards in Kinesis Data Streams increases parallelism, allowing more Lambda invocations to process records concurrently. Option A is wrong because writing to S3 more frequently does not reduce Lambda processing time. Option B is wrong because Lambda already processes records; increasing memory may help but does not address the root cause of limited shard count. Option D is wrong because S3 does not support streaming directly.

Answer

Increase the Lambda function's timeout value.

Answer

Increase the memory allocated to the Lambda function.

Answer

Configure Amazon S3 Event Notifications to trigger Lambda directly.

Question 6

A data engineer needs to ingest JSON files from an on-premises SFTP server into Amazon S3. The files are uploaded daily and each file is up to 500 MB. The solution must be serverless and minimize cost. Which service should the engineer use?

Accepted Answer

AWS Transfer Family (SFTP) endpoint.. Option B is correct because AWS Transfer Family supports SFTP and can automatically transfer files to S3, with no need to manage servers. Option A is wrong because AWS DataSync requires an agent on-premises. Option C is wrong because S3 Transfer Acceleration is for speeding up transfers to S3, not for ingesting from SFTP. Option D is wrong because Kinesis Data Firehose is for streaming data, not file-based SFTP transfers.

Answer

Amazon Kinesis Data Firehose.

Answer

AWS DataSync with an on-premises agent.

Answer

Amazon S3 Transfer Acceleration.

Question 7

A company uses AWS Glue ETL to transform data from Amazon RDS for PostgreSQL to Amazon S3. The transformation includes joining several tables and aggregating millions of rows. The job runs successfully but takes over 2 hours. The data engineer wants to reduce runtime. Which action is MOST effective?

Accepted Answer

Increase the number of DPUs for the Glue job.. Option D is correct because increasing the number of DPUs (Data Processing Units) in AWS Glue ETL jobs can significantly reduce runtime by parallelizing the transformation. Option A is wrong because Auto Scaling is enabled by default and may not help if DPUs are already maxed. Option B is wrong because converting to Parquet can help with reading from S3, but the source is RDS, not S3. Option C is wrong because writing to S3 using dynamic frames does not directly improve performance of reading from RDS.

Answer

Enable Auto Scaling for the Glue job.

Answer

Use AWS Glue DynamicFrames instead of DataFrames.

Answer

Convert the source data to Parquet format.

Question 8

An organization uses AWS Lake Formation to manage a data lake in S3. A new data engineer needs to create a Glue ETL job that reads from a Lake Formation-managed table. The engineer has been granted SELECT permission on the table via Lake Formation. However, the job fails with an AccessDenied error. What is the MOST likely cause?

Accepted Answer

The IAM role used by the Glue job does not have Lake Formation permissions.. Option C is correct because Glue jobs need an IAM role that has Lake Formation permissions to access the table; the error indicates the job's IAM role lacks necessary permissions. Option A is wrong because S3 permissions are managed by Lake Formation, but access is granted via Lake Formation, not direct S3 bucket policies. Option B is wrong because the table already exists; there is no need to register. Option D is wrong because Lake Formation does not require a VPC endpoint.

Answer

The S3 bucket policy does not allow the Glue job to access the data.

Answer

The table has not been registered with Lake Formation.

Answer

The Glue job is not running in the same VPC as Lake Formation.

Question 9

A company needs to ingest real-time sensor data from thousands of IoT devices into Amazon S3, with a latency of less than 1 minute. The data must be transformed (e.g., convert to Parquet) before landing in S3. Which combination of services is MOST cost-effective?

Accepted Answer

Amazon Kinesis Data Streams to Amazon Kinesis Data Firehose to S3.. Option A is correct because Kinesis Data Streams ingests data in real-time, and Kinesis Data Firehose can buffer, transform (e.g., convert to Parquet), and deliver to S3 with low latency. Option B is wrong because Kinesis Data Analytics is for running SQL on streams, not for simple transformation. Option C is wrong because Glue is batch-oriented, not real-time streaming. Option D is wrong because Lambda can transform but may not scale cost-effectively for thousands of devices.

Answer

Amazon Kinesis Data Streams to AWS Lambda to S3.

Answer

Amazon Kinesis Data Streams to Amazon Kinesis Data Analytics to S3.

Answer

Amazon Kinesis Data Streams to AWS Glue streaming ETL to S3.

Question 10

A data engineer runs an AWS Glue ETL job that writes to a table in the AWS Glue Data Catalog. The job fails occasionally with the error 'Resource Not Found' for the table. The table exists. What is a likely cause?

Accepted Answer

The job is using an outdated version of the table schema.. Option B is correct because Glue jobs use a version of the table schema that may be stale if the schema was updated after the job started; the job may be referencing an old version that no longer exists. Option A is wrong because table location is irrelevant if the table exists. Option C is wrong because concurrent jobs do not cause 'Resource Not Found' errors. Option D is wrong because the job name does not affect table access.

Answer

Multiple Glue jobs are writing to the same table concurrently.

Answer

The table location in S3 is incorrect.

Answer

The Glue job name contains special characters.

Question 11

A company uses Amazon Kinesis Data Firehose to deliver data to an Amazon S3 bucket. The data delivery is delayed by up to 5 minutes. The engineer wants to reduce the delay to under 1 minute. Which parameter should be adjusted?

Accepted Answer

Decrease the buffer interval in Kinesis Data Firehose.. Option B is correct because lowering the buffer interval reduces the time Firehose waits before delivering a batch, thus reducing latency. Option A is wrong because increasing buffer size would increase delay. Option C is wrong because compression has no effect on delivery timing. Option D is wrong because error handling does not affect delivery delay.

Answer

Enable error logging to CloudWatch.

Answer

Increase the buffer size in Kinesis Data Firehose.

Answer

Enable data compression.

Question 12

A company has a large volume of CSV files in S3 that need to be transformed into Parquet using AWS Glue. The files are partitioned by date. The engineer wants to minimize costs by processing only new files each day. Which approach should be used?

Accepted Answer

Enable job bookmarks in the Glue job.. Option C is correct because using a Job Bookmark in Glue tracks processed data and skips already processed files, processing only new ones. Option A is wrong because it would reprocess all files. Option B is wrong because partitioning alone does not prevent reprocessing. Option D is wrong because an SQS event can trigger a job, but without bookmarks, it may still reprocess.

Answer

Use S3 partition discovery to automatically read new partitions.

Answer

Schedule the Glue job to run daily and process all files.

Answer

Use S3 Event Notifications to trigger the Glue job on each new file.

Question 13

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams. The data must be enriched with reference data from a DynamoDB table before being written to S3. The engineer wants to minimize latency. Which architecture is BEST?

Accepted Answer

Use Kinesis Data Analytics for Apache Flink to enrich and output to Firehose.. Option C is correct because Kinesis Data Analytics for Apache Flink supports enrichment using external sources like DynamoDB with low latency via asynchronous I/O. Option A is wrong because Kinesis Data Firehose does not support real-time enrichment. Option B is wrong because Lambda may have cold starts and limited concurrency. Option D is wrong because Glue streaming ETL is batch-oriented and higher latency.

Answer

Use AWS Glue streaming ETL to read from Kinesis, enrich, and write to S3.

Answer

Use Kinesis Data Firehose with a Lambda function for enrichment.

Answer

Use a Lambda function to poll the stream, enrich, and write to Firehose.

Question 14

A data engineer is designing a data ingestion pipeline for real-time clickstream data. Which TWO services can be used to ingest the data into Amazon Kinesis Data Streams?

Accepted Answer

Kinesis Producer Library (KPL). Options A and D are correct. The Kinesis Producer Library (KPL) is a library for producers to send data to Kinesis Data Streams. AWS SDK can also be used directly. Option B is wrong because Kinesis Data Firehose is a downstream consumer, not a producer. Option C is wrong because AWS Glue is an ETL service, not a producer. Option E is wrong because S3 is a destination, not a producer.

Answer

Amazon S3

Answer

Kinesis Data Firehose

Answer

AWS Glue

Question 15

A company uses AWS Glue to transform data in S3. The Glue job fails with memory errors. Which THREE actions can help resolve this?

Accepted Answer

Optimize the transformation to use pushdown predicates.. Options A, B, and E are correct. Increasing DPUs (A) adds more worker nodes. Using a larger worker type (B) increases memory per worker. Optimizing transformations like using pushdown predicates (E) reduces data scanned. Option C is wrong because reducing DPUs would make the problem worse. Option D is wrong because increasing timeout does not address memory issues.

Answer

Increase the job timeout.

Answer

Decrease the number of DPUs.