DEA-C01 Practice Test 40 — 15 Questions

Question 1

A financial services company ingests stock trade data from multiple exchanges into an Amazon S3 bucket (trade-bucket). Each exchange sends a CSV file every 5 minutes. The data must be transformed into Parquet format and partitioned by exchange and date (trade_date) for efficient querying using Amazon Athena. The pipeline must handle late-arriving data (files up to 2 hours late) and ensure exactly-once processing to avoid duplicates. Currently, a scheduled AWS Glue ETL job runs every hour, reads new CSV files, converts them to Parquet, and writes to an output bucket. However, the team is experiencing data duplication: if the job fails midway, upon retry it reprocesses all files in the input folder, causing duplicates in the output. Additionally, the job takes too long because it scans all files each run. The engineer must redesign the pipeline to eliminate duplicates and improve efficiency. What should the engineer do?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Set up an S3 event notification to invoke an AWS Lambda function that starts a Glue job with a parameter containing the S3 object key of the new file; modify the Glue job to process only that file and use the file key to avoid duplicates.. The best approach. By setting up an S3 event notification to invoke a Lambda function that triggers a Glue job with the new file's S3 key as a parameter, the job processes only that specific file. Using the file key in the job logic ensures idempotency—if the job fails and retries, it reprocesses the same file key, and deduplication can be handled (e.g., by checking if the output partition already contains that file's data or using a job bookmark on the file key). This achieves exactly-once processing and incremental processing (no full scans), improving efficiency. Option A (Glue Workflows) still processes all files each run and doesn't prevent duplicates if a file arrives after the job starts. Option C (moving CSV files to an archive folder) risks race conditions if late-arriving data comes while the job is running, and does not guarantee exactly-once if the job fails mid-way. Option D (EMR with Spark Structured Streaming) is overly complex and expensive for a 5-minute CSV batch ingestion; checkpointing can handle failures but adds significant operational overhead.

Answer

Use AWS Glue Workflows to orchestrate the job and add a condition to check for duplicates before writing.

Answer

Modify the Glue job to move processed CSV files to an archive folder after successful transformation, and process only unprocessed files.

Answer

Replace Glue with Amazon EMR and use Spark Structured Streaming with checkpointing to process files incrementally.

Question 2

A healthcare company uses AWS Glue to process patient data stored in Amazon S3. The data is encrypted at rest using SSE-KMS with a customer managed key. The Glue ETL job runs on a schedule and reads from an S3 bucket, transforms the data, and writes to another S3 bucket also encrypted with the same KMS key. Recently, the security team rotated the KMS key. After the rotation, the Glue job started failing with 'AccessDenied' errors when trying to read from the source bucket. The Glue job's IAM role has permissions to use the KMS key (kms:Decrypt, kms:GenerateDataKey). The S3 bucket policies allow the role to read/write. What is the MOST likely cause of the failure?

Accepted Answer

The KMS key rotation created a new backing key, but the Glue job's IAM role does not have permission to decrypt with the old backing key.. When you rotate a customer managed KMS key, AWS KMS retains the old backing key to allow decryption of data encrypted before the rotation. However, the Glue job's IAM role must have permission to use the old backing key via the kms:Decrypt action. If the key policy or IAM policy does not explicitly allow decryption with the old backing key (or if the key policy was inadvertently updated to remove access to the old key material), the Glue job will fail with AccessDenied when reading SSE-KMS encrypted objects that were encrypted with the previous key version.

Answer

The Glue job's IAM role is missing the kms:Encrypt permission on the KMS key.

Answer

The Glue job is using the wrong encryption context when calling KMS.

Answer

The S3 bucket policy has a condition that requires the request to use the latest version of the KMS key.

Question 3

A media company uses Amazon Kinesis Data Firehose to ingest log data from web servers into Amazon S3. The data is then processed by AWS Glue jobs. The company wants to ensure that data is delivered to S3 within 5 minutes of ingestion. Currently, the Firehose delivery stream is configured with a buffer interval of 300 seconds and a buffer size of 5 MB. The log data arrives at a rate of 2 MB per second. The data engineer notices that some log files are delayed by up to 10 minutes. The company cannot change the buffer size due to downstream requirements. What should the data engineer do to meet the 5-minute delivery requirement?

Accepted Answer

Decrease the buffer interval to 120 seconds.. Decreasing the buffer interval to 120 seconds ensures that data is delivered to S3 within 2 minutes, meeting the 5-minute requirement. The current buffer interval of 300 seconds (5 minutes) is the primary cause of delays up to 10 minutes. Option A is incorrect because increasing the buffer interval would increase delay. Option B is incorrect because increasing buffer size would also increase delay, as it takes longer to fill. Option C is incorrect because compression does not affect the buffer interval; data still waits up to 300 seconds.

Answer

Increase the buffer interval to 600 seconds to reduce the number of delivery attempts.

Answer

Increase the buffer size to 10 MB to ensure data is delivered in larger chunks.

Answer

Enable GZIP compression on the Firehose stream to reduce data size.

Question 4

A media company ingests video files from content partners into an Amazon S3 bucket. Each video file is 10-50 GB. Upon upload, an AWS Lambda function is triggered to extract metadata (e.g., resolution, duration) and store it in DynamoDB. The company now wants to also generate a thumbnail image for each video. The thumbnail generation is CPU-intensive and can take up to 5 minutes per video. The Lambda function has a maximum execution time of 15 minutes. The company has noticed that some thumbnail generation tasks are timing out. What should the data engineer do to reliably generate thumbnails for all videos?

Accepted Answer

Use AWS Batch to run a containerized thumbnail generation job triggered by S3 events. Use AWS Batch to run a containerized thumbnail generation job triggered by S3 events. AWS Batch is designed for long-running, compute-intensive workloads like video processing, and it can automatically scale resources based on the job queue. Option A (EC2 instance) requires manual management and scaling. Option B (Glue Python shell) is intended for ETL jobs, not CPU-intensive video processing. Option C (increasing Lambda timeout and memory) may still not be reliable for 5-minute thumbnail generation since Lambda has a maximum execution time of 15 minutes and is not optimized for heavy compute tasks; it could still time out or be cost-inefficient.

Answer

Provision an EC2 instance to run a script that polls S3 for new videos and generates thumbnails

Answer

Use AWS Glue with a Python shell job to generate thumbnails

Answer

Increase the Lambda timeout to 15 minutes and allocate more memory

Question 5

A financial services company ingests real-time stock trade data using Amazon Kinesis Data Streams with 10 shards. Each shard receives about 500 records per second, each record approximately 1 KB. The data is consumed by a single AWS Lambda function that transforms the data and writes to Amazon S3. The Lambda function is configured with 1024 MB memory and a timeout of 5 minutes. The company notices that the Lambda function is frequently throttled, and data ingestion lags behind. The Lambda function's CloudWatch metrics show that the iterator age is increasing, and the function's concurrency is maxed out at 1000. The data engineer needs to resolve the throttling issue without changing the Lambda function code. What should the data engineer do?

Accepted Answer

Increase the number of shards in the Kinesis data stream to increase parallelism.. Increasing the number of shards increases the number of Kinesis Data Streams processing units, which directly increases the concurrency limit for Lambda consumers. With more shards, more Lambda function instances can process records in parallel, reducing the iterator age and alleviating throttling. Option B is wrong because reducing memory would likely degrade performance and not increase concurrency limit (concurrency limit is independent of memory). Option C is wrong because decreasing batch size increases the number of invocations, potentially worsening throttling and overhead. Option D is wrong because increasing memory may improve per-record processing speed, but the core issue is concurrency limit being maxed out; increasing memory does not increase concurrency limit and may not resolve throttling if CPU is not the bottleneck.

Answer

Reduce the Lambda function memory to 512 MB to increase concurrency limit.

Answer

Decrease the batch size to 10 records to reduce processing time per invocation.

Answer

Increase the Lambda function memory to 2048 MB to improve processing speed.

Question 6

A company uses AWS KMS to encrypt sensitive data stored in S3. To meet compliance requirements, they need to ensure that the encryption keys are automatically rotated every year. Which type of KMS key should they use?

Accepted Answer

Customer managed key with automatic rotation enabled. Customer managed keys with automatic rotation enabled support automatic annual rotation, meeting the compliance requirement. AWS managed keys rotate automatically every year, but they cannot be controlled or customized by the customer, so they are not the best choice when the customer needs to manage the key policy or rotation schedule. Custom key stores (CloudHSM) do not support automatic rotation. Option A (customer managed key with manual rotation) requires manual intervention to rotate, not automatic. Therefore, D is the correct answer.

Answer

Customer managed key with manual rotation

Answer

AWS managed key

Answer

Custom key store (CloudHSM) key

Question 7

A company wants to grant cross-account access to an S3 bucket without using IAM roles. The data engineer needs to write a bucket policy that allows another AWS account to list objects. Which Principal should be specified in the bucket policy?

Accepted Answer

The AWS account ID of the other account. Specifying the AWS account ID of the other account as the Principal in the bucket policy grants cross-account access to all users and roles in that account, allowing them to list objects. Option A is incorrect because the owning account's ID would grant access to itself, not the other account. Option C is incorrect because specifying an IAM user ARN would restrict access to only that user, not the entire account. Option D is incorrect because the root user is a specific principal, not the account-wide access needed for cross-account delegation.

Answer

The AWS account ID that owns the bucket

Answer

The IAM user ARN in the other account

Answer

The root user of the other account

Question 8

Refer to the exhibit. A data engineer is troubleshooting a Glue job that reads objects from this S3 bucket. The job runs successfully but produces no output. The Glue catalog table points to the same S3 path. What is the most likely cause?

Accepted Answer

The S3 key does not follow Hive-style partitioning (e.g., year=2024/month=01).. AWS Glue Data Catalog relies on Hive-style partitioning (e.g., year=2024/month=01) to automatically discover and register partitions. The S3 key in the exhibit does not follow this pattern, so Glue cannot identify the partitions, resulting in no data being read even though the job runs. Option B is incorrect because object metadata size does not affect Glue's ability to read data. Option C is incorrect because the STANDARD storage class is fully supported by Glue. Option D is incorrect because ContentType is not a factor in Glue catalog table definitions.

Answer

The object metadata is too large.

Answer

The StorageClass is not supported by Glue.

Answer

The ContentType is not supported by Glue.

Question 9

A company wants to implement least privilege access for its data lake on S3. Which THREE practices should be followed? (Choose THREE.)

Accepted Answer

Use S3 bucket policies for cross-account access. The correct answers are B, C, and E. To implement least privilege for an S3 data lake, use S3 bucket policies for cross-account access (B), use S3 access points to enforce network policies (C), and use IAM policies to grant specific permissions (E). Option A is incorrect because granting s3:* to all users violates least privilege. Option D is incorrect because disabling S3 Block Public Access increases risk; it should be enabled.

Answer

Grant s3:* to all users for simplicity

Answer

Disable S3 Block Public Access to allow flexibility

Question 10

A data engineer needs to ensure that data stored in Amazon S3 is automatically deleted after 30 days. Which S3 feature should be used?

Accepted Answer

S3 Lifecycle policy. S3 Lifecycle policies can automatically delete objects after a specified time period, such as 30 days. Option B (MFA Delete) requires multi-factor authentication for deletion but does not automate deletion. Option C (Versioning) keeps multiple versions but does not delete. Option D (Object Lock) prevents deletion or modification but does not schedule automatic deletion.

Answer

S3 MFA Delete

Answer

S3 Versioning

Answer

S3 Object Lock

Question 11

A data engineer needs to ingest JSON files from an on-premises SFTP server into Amazon S3. The files are uploaded daily and each file is up to 500 MB. The solution must be serverless and minimize cost. Which service should the engineer use?

Accepted Answer

AWS Transfer Family (SFTP) endpoint.. AWS Transfer Family provides a fully managed SFTP endpoint that directly integrates with Amazon S3, enabling file transfers without managing servers. Option A (Kinesis Data Firehose) is designed for streaming data, not file-based SFTP ingestion. Option B (AWS DataSync with an on-premises agent) requires installing and managing an agent, which adds operational overhead and is not fully serverless. Option C (S3 Transfer Acceleration) accelerates transfers to S3 but does not support ingesting from SFTP sources.

Answer

Amazon Kinesis Data Firehose.

Answer

AWS DataSync with an on-premises agent.

Answer

Amazon S3 Transfer Acceleration.

Question 12

A company wants to encrypt data at rest in Amazon S3 using server-side encryption. They need to manage the encryption keys themselves and rotate them annually. Which S3 encryption option should they use?

Accepted Answer

SSE-C. SSE-C (Server-Side Encryption with Customer-Provided Keys) allows customers to provide their own encryption keys, manage them, and rotate them as needed. SSE-S3 uses AWS-managed keys, offering no customer control over key management. SSE-KMS uses AWS KMS keys, where AWS manages the key material, though customers can manage key policies and automatic rotation. Client-side encryption is not server-side and does not meet the requirement. Therefore, SSE-C is the correct option.

Answer

SSE-KMS

Answer

SSE-S3

Answer

Client-side encryption

Question 13

A company needs to ingest data from multiple SaaS applications (e.g., Salesforce, Marketo) into Amazon S3 for analytics. The data sources have different schemas and update frequencies. Which AWS service should be used to build this ingestion pipeline with minimal code?

Accepted Answer

Amazon AppFlow. Amazon AppFlow (Option D) is the correct answer because it is purpose-built for ingesting data from SaaS applications like Salesforce and Marketo into Amazon S3 with minimal code. It supports various source connectors, handles schema variations, and allows scheduling based on update frequencies. AWS Data Pipeline (A) requires more manual configuration and code for connectors. AWS Glue (B) has connectors but is more complex for this use case, often requiring additional ETL scripting. Amazon Kinesis Data Firehose (C) is designed for streaming data, not batch extraction from SaaS APIs.

Answer

AWS Data Pipeline

Answer

AWS Glue

Answer

Amazon Kinesis Data Firehose

Question 14

A data engineer needs to audit all changes to IAM policies in an AWS account. Which AWS service should be used?

Accepted Answer

AWS CloudTrail. WS CloudTrail because it records API calls made in the account, including changes to IAM policies. AWS Config tracks resource configuration changes, not API calls. AWS Organizations is for multi-account management. Amazon CloudWatch Logs is for log storage and monitoring, not auditing API calls.

Answer

AWS Config

Answer

AWS Organizations

Answer

Amazon CloudWatch Logs

Question 15

Refer to the exhibit. A data engineer runs this AWS Glue job but it fails with an error that the table 'orders' does not exist in the 'sales_db' database. The engineer has verified that the table exists in the AWS Glue Data Catalog. What is the most likely cause of the error?

Accepted Answer

The IAM role used by the Glue job does not have permission to read the Data Catalog. The error 'table orders does not exist in sales_db' occurs because the IAM role associated with the Glue job does not have the necessary permissions to read the AWS Glue Data Catalog. Even though the table exists, the job requires 'glue:GetTable' permission on the 'sales_db.orders' table to discover it. Option B is incorrect because job bookmark settings control data processing state, not table discovery. Option C is incorrect because the script using 'create_dynamic_frame.from_catalog' is syntactically correct. Option D is incorrect because the error specifically indicates the table is missing, not the S3 path.

Answer

The Glue job has job bookmark enabled and is skipping the table

Answer

The script uses 'create_dynamic_frame.from_catalog' incorrectly

Answer

The S3 path 's3://data-lake/raw/' does not exist