Courseiva

CCNA Data Ingestion Transformation Questions

75 of 447 questions · Page 4/6 · Data Ingestion Transformation topic · Answers revealed

226
MCQmedium

A company ingests application logs into Amazon S3 through Amazon Kinesis Data Firehose. The logs arrive as newline-delimited JSON, and analysts query them with Amazon Athena. Query performance is poor because the JSON files are small and uncompressed. The engineer must improve Athena query performance while keeping the raw JSON available for a downstream legacy system. Which change should the engineer make?

A.Enable S3 Transfer Acceleration on the target bucket to speed up Athena scans.
B.Configure the Athena table with a SerDe for JSON and add a WHERE clause on the timestamp column in every query.
C.Increase the Firehose buffer size and interval so larger objects are delivered to S3.
D.Add a Firehose record format conversion to Apache Parquet and keep the raw JSON in a separate S3 prefix using a second delivery stream.
AnswerD

Converting to Parquet with a Firehose record format conversion gives Athena a columnar, compressed format that scans far less data, directly improving query performance. Delivering the raw JSON to a separate prefix through a second stream preserves the legacy feed, satisfying both requirements without duplicating logic in the consumers.

Why this answer

Athena performance improves when it scans less data, and converting the stream to Parquet with a record format conversion makes the stored objects columnar and compressed. A second delivery stream that leaves the raw JSON in place satisfies the legacy consumer, so the engineer gets both performance and compatibility.

Exam trap

The trap here is assuming buffer tuning or query-side filters can substitute for changing the storage format, when only a columnar, compressed layout reduces the bytes Athena actually reads.

227
MCQmedium

A company uses AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job fails intermittently with a 'MemoryError'. What is the MOST likely cause?

A.The Glue job worker type is too small for the data volume
B.The Glue job uses too many DynamicFrames
C.The S3 output bucket is in a different region
D.The Kinesis stream has insufficient shards
AnswerA

MemoryError indicates the worker's heap is exhausted during processing. A too-small worker type provides insufficient memory for the Kinesis stream's volume, so the job fails intermittently as load spikes. Scaling the worker type directly addresses the memory constraint.

Why this answer

The 'MemoryError' in AWS Glue indicates that the worker type allocated to the job does not have sufficient memory to process the data volume. Glue workers (Standard, G.1X, G.2X) have fixed memory allocations (e.g., 16 GB for Standard), and if the streaming data from Kinesis exceeds this, the job fails. Increasing the worker type or the number of workers resolves this.

Exam trap

The trap here is that candidates confuse memory errors with throttling or connectivity issues, leading them to pick insufficient shards (Option D) or cross-region problems (Option C), when the root cause is almost always an undersized worker type for the data volume.

How to eliminate wrong answers

Option B is wrong because using too many DynamicFrames does not directly cause a MemoryError; DynamicFrames are lazy transformations and memory issues arise from data volume or worker size, not the number of frames. Option C is wrong because an S3 output bucket in a different region would cause a cross-region access error (e.g., AccessDenied or timeout), not a MemoryError. Option D is wrong because insufficient Kinesis shards cause throttling (ProvisionedThroughputExceededException) or data latency, not a memory exhaustion in the Glue job.

228
MCQhard

A data engineer is using AWS Glue to process a large dataset stored in Amazon S3 in Parquet format. The Glue job performs a join between two tables and writes the result back to S3. The engineer notices that the job is running slowly and consuming excessive DPU hours. The job has 10 workers of type G.1X. Which action should the engineer take to improve performance and reduce cost?

A.Use the 'ApplyMapping' transformation to rename columns before the join.
B.Enable AWS Glue job bookmarks to avoid reprocessing old data.
C.Increase the number of workers to 20 and keep the worker type as G.1X.
D.Partition the Parquet data by the join key and use broadcast join if one table is small.
AnswerD

Partitioning the data by the join key can reduce data shuffling during the join, and using a broadcast join for a small table avoids a full shuffle altogether. These optimizations can significantly improve performance and reduce DPU hours. This approach directly addresses the join bottleneck and is a best practice in AWS Glue.

Why this answer

The performance issue is likely due to inefficient join operations causing large data shuffles. Partitioning the Parquet data by the join key aligns data with the join operation, reducing shuffling. If one table is small enough to fit in memory, a broadcast join eliminates the shuffle entirely.

These optimizations reduce runtime and DPU consumption. Other options either do not address the join bottleneck or could increase cost without solving the problem.

Exam trap

The trap here is assuming that adding more workers always improves performance, when the real issue may be data skew or lack of partitioning.

229
MCQmedium

A data engineer needs to ingest data from an Amazon DynamoDB table into an Amazon S3 data lake. The table is updated frequently and the engineer must capture all item-level changes in near real-time without impacting table performance. The ingested data must be stored in a format that preserves the change type (INSERT, MODIFY, REMOVE). Which solution meets these requirements with the LEAST operational overhead?

A.Enable DynamoDB Accelerator (DAX) and use it to stream changes to Amazon S3.
B.Schedule an AWS Glue job to periodically scan the entire DynamoDB table and write the full dataset to Amazon S3.
C.Use AWS Database Migration Service (AWS DMS) with ongoing replication to read from DynamoDB and write to Amazon S3.
D.Enable DynamoDB Streams on the table and use AWS Lambda to read the stream and write the records to Amazon S3 in JSON format.
AnswerD

DynamoDB Streams captures item-level changes in near real-time and Lambda can process these records and write them to S3. This preserves the change type and requires no server management, meeting the low operational overhead requirement. It is the standard serverless pattern for DynamoDB change data capture to S3.

Why this answer

DynamoDB Streams provides a time-ordered sequence of item-level modifications in near real-time, and AWS Lambda can process these records and write them to S3. This serverless approach requires minimal operational overhead and preserves the change type. Other options either do not capture change type, are batch-oriented, or add unnecessary management overhead.

Exam trap

The trap here is assuming that AWS DMS captures change type for DynamoDB, but it does not; it replicates the current state without indicating the operation.

230
MCQhard

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is transformed using an AWS Lambda function. Some records fail transformation and are lost because the Lambda function throws an exception. The data engineer needs to capture the failed records for analysis without affecting the pipeline. What should the engineer do?

A.Configure the Firehose delivery stream to send failed records to a backup S3 bucket
B.Increase the buffer size of the Firehose stream
C.Disable the Lambda transformation and process all records in batch later
D.Modify the Lambda function to write failed records to Amazon DynamoDB
AnswerA

Firehose can be configured to send failed records to a backup S3 bucket.

Why this answer

Amazon Kinesis Data Firehose natively supports a 'backup' or 'error output' configuration: when a Lambda transformation fails (e.g., throws an exception), Firehose can automatically route the failed records to a designated Amazon S3 bucket for failed data. This allows the engineer to capture and analyze the failed records without blocking or altering the main delivery pipeline, as the stream continues processing successfully transformed records.

Exam trap

The trap here is that candidates often confuse Firehose's 'backup' mode (which copies all records) with the 'error output' configuration (which only captures failed records), or they assume that modifying the Lambda function to handle failures is the only option, missing the native Firehose feature that requires no code changes.

How to eliminate wrong answers

Option B is wrong because increasing the buffer size (e.g., MBs or interval) only affects how long Firehose waits before delivering a batch; it does not capture or handle Lambda transformation failures. Option C is wrong because disabling the Lambda transformation would stop all data transformation, which is a core requirement of the pipeline, and processing records in batch later would not recover records already lost during the stream. Option D is wrong because while writing failed records to DynamoDB from within the Lambda function is possible, it requires modifying the Lambda code and does not leverage Firehose's built-in error handling; moreover, DynamoDB has a 400 KB item size limit and is not designed for high-volume streaming failure capture, making it less suitable than the native S3 backup bucket.

231
MCQeasy

A company needs to transfer 20 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The network bandwidth is limited and the transfer must complete within one week. Which service should the company use?

A.Amazon S3 Transfer Acceleration
B.AWS Snowball Edge
C.AWS Direct Connect with DataSync
D.AWS DataSync over a VPN connection
AnswerB

AWS Snowball Edge ships a physical appliance to the on-premises site, bypassing the limited network bandwidth constraint. It transfers 20 TB within a week, whereas online transfer methods would be too slow over the constrained link.

Why this answer

AWS Snowball Edge is a petabyte-scale data transport solution that uses physical devices to transfer large amounts of data into and out of AWS, bypassing network bandwidth limitations. For 20 TB over a limited network within one week, Snowball Edge is the most practical and cost-effective choice. It supports up to 80 TB usable capacity per device and provides encryption and tamper-resistant security.

Exam trap

DEA-C01 often tests the threshold where network transfer becomes impractical, so candidates may overlook Snowball for 20 TB and incorrectly choose online transfer methods like DataSync or Transfer Acceleration.

How to eliminate wrong answers

Option A is wrong because S3 Transfer Acceleration speeds up transfers over the internet but still relies on available bandwidth, which is limited. Option C is wrong because AWS Direct Connect with DataSync requires a dedicated network connection that may not be feasible within the timeframe and does not solve bandwidth constraints. Option D is wrong because DataSync over VPN is also limited by network bandwidth and would likely not complete 20 TB within a week.

232
MCQmedium

A company uses AWS Glue to transform data in S3. The transformation job reads Parquet files, filters rows, and writes to another S3 bucket. The job takes longer than expected. Which change would MOST likely reduce the job execution time?

A.Use a single large file instead of multiple small files.
B.Reduce the number of partitions in the output data.
C.Convert the input files from Parquet to CSV format.
D.Increase the number of DPUs allocated to the Glue job.
AnswerD

Glue distributes work across DPUs, so adding DPUs increases parallel task capacity and shortens the filter-and-write job's runtime. This directly addresses the execution-time constraint, provided the job is not bottlenecked by a single small input partition.

Why this answer

Increasing the number of DPUs (Data Processing Units) allocated to the Glue job directly increases the parallelism of the Apache Spark-based execution environment. Since the job reads Parquet files, filters rows, and writes output, a bottleneck in compute capacity is the most likely cause of prolonged execution time. More DPUs allow Spark to distribute the workload across more executors, reducing overall runtime.

Exam trap

The trap here is that candidates often assume optimizing file format or output partitioning will always improve performance, but for a compute-bound transformation job, increasing parallelism via DPUs is the most direct solution.

How to eliminate wrong answers

Option A is wrong because using a single large file instead of multiple small files would reduce parallelism in Spark's file scanning, potentially increasing execution time due to less efficient task distribution. Option B is wrong because reducing the number of partitions in the output data affects the write phase but does not address the root cause of slow processing during the read and filter stages. Option C is wrong because converting from Parquet to CSV would increase I/O and CPU overhead due to CSV's lack of compression, schema enforcement, and columnar storage, making the job slower, not faster.

233
MCQhard

A company uses AWS Glue to process data from Amazon RDS MySQL into Amazon S3. The Glue job uses a JDBC connection and runs on a schedule. Recently, the job has been failing with a 'Communications link failure' error. The RDS instance is in a private subnet. Which troubleshooting step should the data engineer take FIRST?

A.Check the Glue job's DPU allocation; increase if too low.
B.Review the Glue job script for data type mismatches.
C.Verify that the Glue job's VPC subnet and security group allow outbound traffic to RDS.
D.Increase the RDS instance's max_connections parameter.
AnswerC

Glue jobs in a VPC need a subnet route and security group egress permitting TCP 3306 to the RDS instance; without it, the JDBC handshake fails with 'Communications link failure'. Verifying outbound connectivity from the Glue ENI directly addresses the private-subnet constraint before deeper checks.

Why this answer

The 'Communications link failure' error typically indicates a network connectivity issue between the Glue job and the RDS instance. Since the RDS instance is in a private subnet, the Glue job must be configured with a VPC subnet and security group that allows outbound traffic to the RDS instance's security group on port 3306 (MySQL). Without this network path, the JDBC connection cannot be established, making verifying the VPC and security group configuration the first logical troubleshooting step.

Exam trap

The trap here is that candidates often assume 'Communications link failure' is a database-side issue (like connection limits or timeouts) and jump to tuning RDS parameters, when in fact it is most commonly a network connectivity problem in a VPC environment.

How to eliminate wrong answers

Option A is wrong because DPU allocation affects job parallelism and memory, not network connectivity; increasing DPUs would not resolve a 'Communications link failure' caused by a missing network path. Option B is wrong because data type mismatches typically cause runtime errors during data conversion, not connection-level failures like 'Communications link failure'. Option D is wrong because increasing max_connections addresses connection limits on the RDS side, but the error indicates the Glue job cannot even reach the database, not that connections are being refused due to exhaustion.

234
MCQmedium

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The application reads from a Kinesis data stream and writes results to an Amazon S3 bucket. Recently, the application has been failing with 'ResourceNotFoundException' for the S3 bucket. What is the MOST likely cause?

A.The IAM role used by the application does not have s3:PutObject permission.
B.The S3 bucket ARN is incorrectly specified in the application configuration.
C.The Flink application code specifies the wrong AWS Region for the S3 bucket.
D.The S3 bucket has versioning disabled.
AnswerB

A malformed or incorrect S3 bucket ARN in the application configuration causes the Flink application's S3 sink to fail resolving the target bucket, producing ResourceNotFoundException. Correcting the ARN restores access without changing IAM or network settings.

Why this answer

The 'ResourceNotFoundException' for the S3 bucket indicates that the application cannot find the bucket. This is most likely due to an incorrectly specified bucket name or ARN in the application configuration. If the bucket name is misspelled or the ARN is malformed, the Kinesis Data Analytics application cannot resolve the bucket resource.

Option A is incorrect because missing 's3:PutObject' permission would cause an 'AccessDenied' error, not 'ResourceNotFoundException'. Option C is incorrect because the AWS Region for S3 is determined by the bucket's location, and specifying a wrong region in Flink code would typically result in a different error (e.g., 'IllegalArgumentException' or cross-region access issues) but not 'ResourceNotFoundException'. Option D is incorrect because versioning is unrelated to bucket existence; disabling versioning does not cause a resource not found error.

Exam trap

Candidates often confuse permission errors with resource-not-found errors. A missing IAM permission typically results in 'AccessDenied', not 'ResourceNotFoundException'.

235
MCQmedium

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The job needs to run daily and process only new data since the last run. The data in S3 is partitioned by date in the format `year=YYYY/month=MM/day=DD/`. Which feature should the data engineer use to track processed partitions and avoid reprocessing old data?

A.AWS Glue job bookmarks
B.AWS Glue crawler update schedules
C.AWS Glue triggers with a schedule
D.AWS Glue Data Catalog partition indexes
AnswerA

AWS Glue job bookmarks track the data that has already been processed in previous job runs. When reading from S3 with partitions, bookmarks keep track of the last processed partition and only process new partitions in subsequent runs. This is the recommended way to handle incremental processing without manual tracking.

Why this answer

AWS Glue job bookmarks are designed to track the state of data processed by a job. When reading from partitioned S3 data, bookmarks record the last processed partition and only process new partitions in subsequent runs. This enables incremental processing without manual intervention, making it the correct choice for avoiding reprocessing of old data.

Exam trap

The trap here is confusing scheduling or catalog maintenance features with state management, assuming that a trigger or crawler schedule would automatically handle incremental processing.

236
MCQeasy

Refer to the exhibit. A data engineer runs this CLI command on an S3 bucket. The data is ingested from multiple sources. Which AWS service would be best to process these files in a single batch transformation?

A.AWS Lambda
B.Amazon Kinesis Data Analytics
C.Amazon Athena
D.AWS Glue
AnswerD

AWS Glue is a serverless, managed extract-transform-load service that handles schema discovery via crawlers and runs distributed Spark jobs, so it can transform all files from multiple sources in one batch job without provisioning or managing servers.

Why this answer

AWS Glue is designed for batch ETL processing, capable of handling multiple files of varying sizes in a single transformation job. Option A is wrong because Lambda has limitations on execution time (15 minutes) and memory (10 GB), making it unsuitable for large-scale batch transforms. Option B is wrong because Kinesis Data Analytics is for real-time stream processing, not batch.

Option C is wrong because Athena is an interactive query service for ad-hoc analysis of data in S3, not for batch transformations.

237
MCQmedium

A data engineer must ingest data from an Amazon DynamoDB table into an Amazon S3 bucket for analytics. The table receives continuous writes, and the ingestion must capture changes with minimal latency. The engineer wants a fully managed solution that requires no server provisioning. Which approach should the engineer use?

A.Use AWS Database Migration Service (AWS DMS) with ongoing replication to replicate the DynamoDB table to Amazon S3.
B.Enable DynamoDB Streams on the table, and use AWS Lambda to read the stream and write the data to Amazon S3.
C.Use Amazon Kinesis Data Firehose to read directly from DynamoDB and deliver to Amazon S3.
D.Schedule an AWS Glue job to run every minute, reading the entire DynamoDB table and writing to Amazon S3.
AnswerB

DynamoDB Streams captures item-level changes in near real time, and Lambda can process these events without managing servers. This serverless pattern provides low-latency ingestion into S3. The engineer does not need to provision or scale any infrastructure, and the solution automatically scales with the stream volume, making it ideal for continuous writes.

Why this answer

DynamoDB Streams provides a time-ordered sequence of item-level changes, and Lambda can process these events in near real time. Writing to S3 from Lambda is a common serverless pattern that requires no server management and scales automatically. This combination meets the latency and managed-service requirements without the overhead of provisioning or full-table scans.

Exam trap

The trap here is assuming that AWS DMS or Kinesis Data Firehose can directly capture DynamoDB changes with low latency, when actually DynamoDB Streams is the native change capture mechanism.

238
Multi-Selecthard

A data engineering team is designing a batch processing workflow using AWS Glue. The job reads from an S3 bucket, transforms data, and writes to another S3 bucket. The job runs daily and processes new data incrementally. Which THREE features should they use to optimize performance and cost?

Select 3 answers
A.Convert all input data to Parquet format before processing.
B.Enable Glue job autoscaling.
C.Manually increase the number of DPUs for each run.
D.Use predicate pushdown and column pruning in the script.
E.Enable job bookmarks to process only new data.
AnswersB, D, E

Glue job autoscaling dynamically adds and removes workers based on the stage of the shuffle and partition workload, so the daily incremental job does not over-provision capacity for its peak. This directly cuts DPU-hour cost while meeting the performance requirement for variable-volume runs.

Why this answer

Option B is correct because Glue job autoscaling dynamically adjusts the number of workers (DPUs) based on the workload, which reduces cost during lighter stages and improves throughput during heavier ones for a daily incremental job. Option D is correct because predicate pushdown and column pruning let Glue read only the needed partitions, rows, and columns from Parquet/ORC on S3, cutting I/O, shuffle, and DPU-hours. Option E is correct because Glue job bookmarks persist state so the job processes only new or changed data since the last run, which is exactly the incremental daily pattern described and avoids reprocessing old S3 objects.

Option A is not marked correct because converting all input data to Parquet is a broad storage-format change, not a Glue job performance/cost feature, and it may add conversion overhead. Option C is not marked correct because manually increasing DPUs for every run is static over-provisioning that raises cost rather than optimizing it, unlike autoscaling.

239
Multi-Selecteasy

A company is designing a data ingestion pipeline for real-time IoT sensor data. The data volume peaks at 10,000 messages per second. The pipeline must process messages in order per sensor and persist raw data to Amazon S3 for archival. Which TWO services should be used together to meet these requirements? (Choose TWO.)

Select 2 answers
A.Amazon Kinesis Data Firehose
B.Amazon Kinesis Data Streams
C.AWS Lambda
D.Amazon AppFlow
E.Amazon Simple Queue Service (Amazon SQS)
AnswersA, B

Delivers streaming data to S3.

Why this answer

Amazon Kinesis Data Streams (KDS) provides the ordered, real-time ingestion layer required for per-sensor message ordering, as it partitions data by a partition key (e.g., sensor ID) and guarantees order within a shard. Amazon Kinesis Data Firehose then reliably reads from the KDS stream and delivers the raw data to Amazon S3 for archival, handling buffering, compression, and automatic retries without requiring custom code.

Exam trap

The trap here is that candidates often choose AWS Lambda as a processing step without realizing it is not required for the core ingestion and archival pipeline, and they may overlook that Kinesis Data Streams is needed for ordering while Firehose handles the S3 delivery.

240
MCQmedium

A data engineer is designing a pipeline to ingest change data capture (CDC) events from an Amazon RDS for PostgreSQL database into Amazon S3. The CDC events are captured using AWS DMS. The data must be available for querying within 5 minutes of the change. Which approach meets these requirements?

A.Export the database to S3 using pg_dump and then use AWS Glue to load into S3 in Parquet format.
B.Use AWS DMS to replicate data to Amazon Redshift, then unload to S3.
C.Use AWS DMS to replicate data directly to S3 in near real-time.
D.Use AWS DMS to replicate data to an SQS queue, then process with Lambda to write to S3.
AnswerC

AWS DMS continuous replication writes CDC changes straight to Amazon S3, avoiding intermediate staging that would add latency. This satisfies the five-minute availability constraint because DMS applies ongoing changes as they occur, rather than relying on scheduled batch exports or hourly snapshots. Target endpoint configuration handles the S3 delivery directly.

Why this answer

AWS DMS supports continuous replication (change data capture) directly to Amazon S3 as a target endpoint. DMS can write CDC events to S3 in near real-time (typically seconds to minutes), meeting the 5-minute latency requirement without intermediate services. The data is stored in comma-separated value (CSV) or Parquet format, ready for querying via Athena or Glue.

Exam trap

The trap here is that candidates may overcomplicate the solution by introducing intermediate services (like Redshift or SQS) when DMS’s native S3 target endpoint already provides near-real-time CDC replication, and they may confuse pg_dump (a batch export tool) with a CDC mechanism.

How to eliminate wrong answers

Option A is wrong because pg_dump is a one-time logical backup tool, not a CDC solution; it cannot capture ongoing changes and would require full exports, exceeding the 5-minute latency window. Option B is wrong because replicating to Redshift then unloading to S3 adds unnecessary complexity and latency; Redshift is optimized for analytics, not as a CDC staging area, and the unload operation introduces additional delay. Option D is wrong because DMS does not natively support SQS as a target endpoint; DMS can write to S3, Kinesis, or other targets, but not directly to SQS, and the proposed architecture would require custom integration, breaking the near-real-time requirement.

241
MCQmedium

A company uses AWS Glue DataBrew to clean and normalize data from an Amazon S3 bucket before loading it into Amazon Redshift. The data contains PII such as social security numbers. A compliance policy requires that PII be masked in the DataBrew output. Which DataBrew transformation should the engineer use to replace the last four digits of each SSN with asterisks?

A.Use the 'Encrypt' transformation with AWS KMS to encrypt the SSN column.
B.Use the 'Remove columns' transformation to drop the SSN column entirely.
C.Use the 'Mask value' transformation with a custom pattern to mask the last four characters.
D.Use the 'Replace value or pattern' transformation to replace the entire SSN with a fixed string.
AnswerC

DataBrew's 'Mask value' transformation supports custom patterns and can mask specific characters. By defining a pattern that targets the last four digits, the engineer can replace them with asterisks while preserving the rest of the SSN. This meets the compliance requirement precisely and is a built-in DataBrew feature.

Why this answer

AWS Glue DataBrew provides a 'Mask value' transformation that can apply custom masking patterns to specific characters. By configuring a pattern to mask the last four digits of the SSN with asterisks, the engineer preserves the first five digits for analytics while complying with the PII masking policy. Other transformations either over-mask, encrypt, or remove the data, failing the specific requirement.

Exam trap

The trap here is assuming that encryption is a form of masking; masking replaces characters with symbols, while encryption transforms data into ciphertext.

242
MCQeasy

An IAM policy includes the above resource ARN for CloudWatch Logs. A data engineer needs to allow a Lambda function to create log streams and put logs to the log group 'my-log-group'. However, the Lambda function is failing with access denied. What is the issue?

A.The region in the ARN does not match the Lambda function's region.
B.The Lambda function does not have an execution role.
C.The ARN does not include the log-stream portion.
D.The ARN is incorrectly formatted because of the wildcard.
AnswerC

CloudWatch Logs actions such as CreateLogStream and PutLogEvents require the ARN to include the log-stream wildcard suffix (log-group:my-log-group:log-stream:*). The policy's ARN stops at the log-group level, so the Lambda function lacks permission for stream-level operations.

Why this answer

The ARN provided in the policy is for the log group itself (arn:aws:logs:region:account-id:log-group:my-log-group), but CloudWatch Logs requires separate permissions for the log-stream subresource. To allow a Lambda function to create log streams and put logs, the ARN must include the log-stream portion, typically with a wildcard like arn:aws:logs:region:account-id:log-group:my-log-group:log-stream:*. Without this, the IAM policy does not grant access to the log-stream actions (CreateLogStream, PutLogEvents), causing an access denied error even if the log group ARN is correct.

Exam trap

The trap here is that candidates assume granting access to the log group ARN implicitly covers log streams, but AWS IAM requires explicit resource-level permissions for each subresource in the ARN hierarchy.

How to eliminate wrong answers

Option A is wrong because the region in the ARN must match the Lambda function's region for the policy to apply, but the question does not indicate a region mismatch; the error is specifically about log-stream permissions. Option B is wrong because a Lambda function must have an execution role to access AWS resources, and the question implies a role exists (the policy is attached), but the role lacks the necessary log-stream permissions. Option D is wrong because the wildcard in the ARN is correctly placed for the log group name (my-log-group) and is not the cause of the failure; the issue is the missing log-stream component, not the wildcard format.

243
MCQhard

A data engineer is using AWS Glue to transform data from an Amazon S3 bucket. The Glue job reads JSON files, applies complex transformations, and writes the output to another S3 bucket in Parquet format. The job runs daily and must complete within a 2-hour window. The engineer notices that the job is taking longer than expected and wants to optimize performance. The source data is partitioned by date, and the job uses a dynamic frame. Which optimization should the engineer implement to improve performance?

A.Increase the number of DPUs allocated to the Glue job.
B.Enable job bookmarks to track processed data and avoid reprocessing.
C.Convert the dynamic frame to a Spark DataFrame and use Spark SQL for transformations.
D.Use pushdown predicates to filter data at the source based on partition columns.
AnswerD

Pushdown predicates allow Glue to filter data at the source, reducing the amount of data read and processed. Since the data is partitioned by date, the engineer can specify a predicate to read only the relevant partitions. This minimizes I/O and speeds up the job, often more effectively than adding resources. It directly addresses the performance bottleneck by reducing data volume.

Why this answer

Pushdown predicates enable AWS Glue to filter data at the source based on partition columns, significantly reducing the amount of data read from S3. Since the data is partitioned by date, specifying a predicate for the desired date range ensures only relevant partitions are processed, leading to faster job completion and lower cost. This is a targeted optimization that addresses the performance bottleneck without unnecessary resource scaling.

Exam trap

The trap here is assuming that adding more DPUs is always the best way to speed up a Glue job, when in fact reducing the amount of data read via pushdown predicates is often more effective and cost-efficient.

244
MCQmedium

Refer to the exhibit. A data engineer is troubleshooting an AWS Glue ETL job that fails with an access denied error when writing to S3. The IAM role attached to the Glue job has the policy shown. What is the most likely cause of the error?

A.The Glue job is missing the s3:ListBucket permission on the bucket
B.The Glue job is writing to an S3 bucket that is not included in the Resource ARN
C.The S3 bucket is encrypted with AWS KMS and the policy does not include kms:Decrypt permissions
D.The Glue job does not have permission to call glue:StartJobRun
AnswerB

S3 access is granted per bucket ARN in the IAM policy. If the Glue job's target bucket falls outside the Resource ARN, the write is denied even though the role itself is valid. The error is authorisation scope, not credentials or networking.

Why this answer

The IAM policy shown in the exhibit explicitly restricts the `s3:PutObject` action to a specific ARN pattern (e.g., `arn:aws:s3:::example-bucket/*`). If the Glue job attempts to write to a different S3 bucket—one not matching that ARN—the request will be denied with an access denied error. AWS Glue ETL jobs use the attached IAM role's permissions, and a mismatch between the target bucket and the resource ARN in the policy is the most common cause of such failures.

Exam trap

The trap here is that candidates often overlook the explicit resource ARN in the policy and assume the error is due to missing permissions like `s3:ListBucket` or KMS, when the real issue is a bucket name mismatch in the resource specification.

How to eliminate wrong answers

Option A is wrong because the `s3:ListBucket` permission is required for listing objects (e.g., `s3:ListBucket` on the bucket resource), not for writing objects; the error occurs during a write operation, and the policy already includes `s3:PutObject` on the bucket objects. Option C is wrong because the error message is specifically 'access denied when writing to S3', not a KMS-related error; if KMS encryption were the issue, the error would typically mention 'KMS' or 'decrypt' and the policy would need `kms:GenerateDataKey` or `kms:Decrypt`, not just `kms:Decrypt`. Option D is wrong because `glue:StartJobRun` is a permission to start a Glue job run, not a permission for writing to S3; the error occurs during the job execution (writing to S3), not during job initiation.

245
MCQmedium

A data engineer must transform data in an AWS Glue ETL job. The transform requires calling an external REST API for each record to enrich the data. The Glue job runs on AWS Glue 4.0 with Python. The engineer wants to minimize the number of API calls and improve performance. Which approach should the engineer take?

A.Convert the DynamicFrame to a Spark DataFrame, use foreachPartition to batch records and call the API once per batch.
B.Use Glue DynamicFrame's map method to call the API for each record.
C.Use AWS Glue's built-in transform 'ApplyMapping' to call the API.
D.Configure the Glue job to use a larger number of DPUs to parallelize API calls.
AnswerA

Converting to a Spark DataFrame and using foreachPartition allows processing records in batches per partition. The engineer can accumulate records into batches and make a single API call per batch, reducing the number of calls. This leverages Spark's distributed processing and is efficient for external API enrichment. It minimizes API calls and improves performance by batching.

Why this answer

To minimize API calls in an AWS Glue job, the engineer should batch records and call the API once per batch. Converting the DynamicFrame to a Spark DataFrame and using foreachPartition allows processing each partition, accumulating records into batches, and making a single API call per batch. This reduces the number of calls and improves performance.

Other options either call the API per record or do not support API calls.

Exam trap

The trap here is assuming that increasing DPUs or using map will automatically optimize API calls, but neither reduces the number of calls per record.

246
Multi-Selectmedium

A data engineer is designing a data ingestion pipeline to load clickstream data from an Amazon S3 bucket into an Amazon Redshift cluster. The data arrives in 5-minute batches. Which TWO actions should the engineer take to ensure data consistency and avoid duplicates? (Select TWO.)

Select 2 answers
A.Define a SORTKEY on the target table to improve deduplication.
B.Disable workload management (WLM) to maximize resources.
C.Use the STL_LOAD_ERRORS system table to monitor and resolve load errors.
D.Load data into a single slice to maintain order.
E.Use a staging table and perform a MERGE operation to avoid duplicates.
AnswersC, E

STL_LOAD_ERRORS records every row Redshift rejected during COPY, including the raw error reason and the S3 source file. Reviewing it lets the engineer correct malformed clickstream records and reload them, preventing silent data loss that would otherwise break consistency across the 5-minute batches.

Why this answer

Option E is correct because loading each 5-minute batch into a staging table and then running a MERGE (or DELETE+INSERT) against the target table lets you match on a business key and update or insert rows idempotently, which prevents duplicate records when batches are retried or overlap. Option C is correct because monitoring STL_LOAD_ERRORS (and STL_LOAD_ERRORS is populated per load with err_code, err_reason, and raw_field_value) lets the engineer detect and resolve failed or partially loaded rows so the pipeline maintains consistent data. Option A is not correct because a SORTKEY only affects physical ordering and query performance; it does not deduplicate rows.

Option B is not correct because disabling WLM removes query queue management and does not address duplicates or consistency. Option D is not correct because Redshift distributes data across slices automatically, and forcing data into a single slice would hurt performance without providing deduplication.

Exam trap

DEA-C01 often tests the confusion between performance tuning (SORTKEY, distribution keys) and data consistency mechanisms (staging tables, MERGE). Candidates may think SORTKEY deduplicates, but it does not. Also, disabling WLM is never a correct answer for consistency.

247
MCQmedium

A company is ingesting log files from multiple EC2 instances into Amazon S3 using the CloudWatch agent. The logs are delivered to a CloudWatch Logs group, and a subscription filter sends them to a Lambda function for transformation, then to Firehose. The Firehose stream is configured with a buffer interval of 60 seconds and buffer size of 5 MB. The logs are critical and must be available in S3 within 5 minutes. What is the most cost-effective way to reduce the delivery latency?

A.Replace Firehose with Amazon Kinesis Data Streams
B.Increase the buffer size to 10 MB
C.Increase the buffer interval to 120 seconds
D.Decrease the buffer interval to 10 seconds
AnswerD

Lower buffer interval reduces delivery latency.

Why this answer

Decreasing the Firehose buffer interval to 10 seconds directly reduces the maximum time data waits in the buffer before being delivered to S3, ensuring logs reach S3 within the required 5-minute window. Since the current 60-second buffer interval is the primary contributor to latency, lowering it to 10 seconds minimizes delivery delay without incurring additional costs, as Firehose charges are based on data volume, not buffer frequency.

Exam trap

The trap here is that candidates may think increasing buffer size or interval improves throughput, but the question asks for reduced latency, so decreasing the buffer interval is the direct and cost-effective solution.

How to eliminate wrong answers

Option A is wrong because replacing Firehose with Kinesis Data Streams would require additional components (e.g., a consumer to write to S3) and increase cost and complexity, not reduce latency cost-effectively. Option B is wrong because increasing the buffer size to 10 MB would allow more data to accumulate before delivery, potentially increasing latency, not reducing it. Option C is wrong because increasing the buffer interval to 120 seconds would double the maximum buffering time, worsening delivery latency.

248
MCQhard

A company runs a nightly batch ETL job using AWS Glue to transform data from Amazon RDS for MySQL to Amazon S3. The job reads 100 tables and writes Parquet files partitioned by date. Recently, the job started failing with 'ThrottlingException' from the RDS database. The data volume has increased, and the Glue job is reading large tables without any filtering. The job uses a single Glue job with multiple Spark executors. The engineer needs to reduce the load on the RDS database while maintaining the same processing time. What should the engineer do?

A.Use AWS DMS to continuously replicate data to S3 and then run Glue on the S3 data.
B.Change the job to read all data from RDS into a staging table in S3 first, then transform.
C.Increase the number of DPUs to process data faster.
D.Modify the Glue job to use a JDBC connection with a WHERE clause to read only the latest partition.
AnswerD

Reading all rows from every table drives concurrent JDBC connections that overwhelm RDS, triggering ThrottlingException. Adding a WHERE clause restricts each read to the latest partition, sharply reducing rows scanned and database load while preserving the same overall processing time.

Why this answer

The ThrottlingException originates from RDS because the Glue job issues full-table JDBC reads across 100 tables with no predicate pushdown. Adding a WHERE clause to read only the latest partition (D) reduces the number of rows scanned and transferred per run, directly lowering the query load on RDS while keeping the same nightly processing window. This is the least disruptive change that preserves the existing architecture and processing time.

Exam trap

The trap is assuming 'more DPUs = faster and less load'; in JDBC-based Glue jobs, more parallelism increases concurrent connections and worsens source-side throttling, so the correct answer is to reduce rows read, not add compute.

How to eliminate wrong answers

Option A is wrong because introducing AWS DMS adds a continuous replication pipeline — a larger architectural change than needed, and it doesn't address the immediate throttling without new infrastructure and cutover effort. Option B is wrong because staging all data in S3 first still requires the same full-table reads from RDS, so it does not reduce load on the source database. Option C is wrong because adding DPUs increases parallelism and therefore increases concurrent JDBC connections and query pressure on RDS, making throttling worse rather than better.

249
MCQhard

A data engineer is using AWS Glue Studio to build a job that reads from an Amazon S3 bucket containing nested JSON, flattens arrays using Relationalize, and writes Parquet to another S3 bucket. The job must run only when new objects land in the source bucket. The engineer wants to avoid unnecessary job runs and minimize cost. Which approach should the engineer use?

A.Configure an Amazon EventBridge rule that matches S3 Object Created events on the source prefix and targets the Glue job as the rule's target.
B.Create an AWS Lambda function that polls the S3 bucket using the ListObjectsV2 API on a one-minute schedule and starts the Glue job when new keys appear.
C.Schedule the Glue job with a cron trigger every five minutes and rely on Glue job bookmarks to skip already-processed files.
D.Enable S3 Event Notifications to publish to an Amazon SNS topic and subscribe the data engineer's email address to the topic.
AnswerA

EventBridge can match S3 Object Created events filtered by bucket and key prefix, then invoke a Glue job directly as a target. This event-driven pattern starts the job exactly when new objects arrive, avoids idle scheduled runs, and reduces cost while meeting the requirement to run only on new data.

Why this answer

S3 can emit Object Created events that EventBridge matches by bucket and prefix, and EventBridge supports an AWS Glue job as a direct target. This is the canonical event-driven pattern: the job starts only when a new object lands, eliminating idle scheduled runs. Polling and SNS notifications either add latency or require manual steps, and a fixed cron schedule still burns DPUs on empty runs.

Exam trap

The trap here is choosing a scheduled cron trigger with bookmarks, which still launches the job on every interval even when no new objects exist.

250
Multi-Selectmedium

A data engineer is using AWS Glue DataBrew to prepare a dataset stored in Amazon S3. The dataset contains missing values, inconsistent date formats, and duplicate rows. The engineer needs to clean the data and produce a transformed output for downstream analytics. (Choose two.)

Select 2 answers
A.Schedule a DataBrew recipe job to run on a cron expression so the cleaning steps execute periodically.
B.Create a recipe step using the Fill missing values transformation to replace nulls with a specified value or strategy.
C.Use the DataBrew profile job to generate statistics about missing values and duplicates in the dataset.
D.Configure a DataBrew dataset to use the S3 bucket as a source and set the file type to CSV with header detection enabled.
E.Apply a recipe step that uses the Remove duplicates transformation to eliminate duplicate rows.
AnswersB, E

The Fill missing values transformation lets you replace null or missing entries with a constant, a calculated value, or a strategy such as most frequent. It directly resolves the missing values problem in the dataset. Because it is a recipe step, it remains part of the reusable transformation pipeline applied to the S3 data.

Why this answer

Cleaning the dataset requires transformation steps that actually modify data. Removing duplicates eliminates redundant rows, and filling missing values addresses nulls. Together these recipe steps resolve two of the stated data quality issues.

Date format normalization would need an additional step. Source configuration, profiling, and scheduling support the workflow but do not themselves clean the data.

Exam trap

The trap here is selecting operational or diagnostic actions like profiling or scheduling instead of the transformation steps that actually change the data.

251
MCQhard

A company runs a data ingestion pipeline that uses AWS Glue to read 500 GB of JSON files from an S3 bucket (s3://raw-data/) every hour. The Glue ETL job transforms the data and writes Parquet files to another S3 bucket (s3://processed-data/). The job is triggered by a time-based CloudWatch Events rule. Recently, the job has started taking over 2 hours to complete, causing delays in downstream processes. The data volume has been consistent, and no changes have been made to the job code or infrastructure. The S3 bucket 's3://raw-data/' receives new files continuously, but the Glue job reads all files in the bucket each run (no incremental processing). The engineer suspects that the job is reprocessing old data. Which action should the engineer take FIRST to reduce the job duration?

A.Enable Glue job bookmarking and configure the job to process only new data.
B.Increase the parallelism of the Spark job by repartitioning the data.
C.Add partition pruning by modifying the S3 path to include date-based partitions.
D.Increase the number of DPUs for the Glue job to 100.
AnswerA

Without job bookmarks, every run rereads the entire s3://raw-data/ prefix, so the 500 GB backlog is reprocessed hourly and runtime balloons. Enabling bookmarks tracks previously processed objects, letting Glue read only newly arrived files, which directly shortens the job.

Why this answer

The Glue job is reprocessing all 500 GB of data every hour because it lacks job bookmarking, which tracks previously processed data. Enabling job bookmarking allows Glue to persist state about which files have already been processed, so subsequent runs only read new files. This directly reduces the input volume and thus the job duration without changing the job logic or infrastructure.

Since the data volume is consistent and no code changes were made, the issue is purely due to reprocessing old data, making bookmarking the correct first step.

Exam trap

DEA-C01 often tests the misconception that scaling resources (more DPUs) or optimizing data layout (partitioning) is the first step to improve job performance, when the root cause is reprocessing old data due to missing job bookmarks.

How to eliminate wrong answers

Option B is wrong because repartitioning the data increases parallelism but does not reduce the total volume of data processed; the job would still read all 500 GB each run, so the duration would not improve significantly. Option C is wrong because partition pruning requires the data to be organized in a partitioned structure (e.g., by date) and the job to be written to leverage partitions; the scenario does not indicate that the data is partitioned, and adding partition pruning would require restructuring the data and modifying the job, which is not the first step. Option D is wrong because increasing DPUs scales resources but does not address the root cause of reprocessing old data; it would increase cost without solving the inefficiency, and the job would still read all data each run.

252
MCQmedium

A data engineer is designing a data ingestion pipeline to load data from an on-premises Oracle database to Amazon S3. The pipeline should capture changes in near real-time (within minutes) and minimize impact on the source database. The source table has a 'last_modified' timestamp column. Which service combination would meet these requirements?

A.AWS DMS with a replication task in CDC mode, writing to S3 in Parquet format.
B.Amazon Kinesis Data Firehose with a Lambda function that queries Oracle.
C.AWS Data Pipeline with a periodic SQL query activity to copy full table snapshots.
D.AWS Glue with a JDBC connection to Oracle, running a crawler every 5 minutes.
AnswerA

DMS change data capture reads the Oracle redo logs continuously, applying minimal query load on the source, and delivers near-real-time changes to S3. Writing Parquet reduces storage and query cost, satisfying the minutes-level latency and low source impact constraints.

Why this answer

AWS DMS with a replication task in CDC (Change Data Capture) mode is the correct choice because it continuously reads the Oracle redo logs to capture near real-time changes (within seconds to minutes) with minimal impact on the source database. It can directly write to S3 in Parquet format, meeting the requirement for low-latency ingestion without full table scans.

Exam trap

The trap here is that candidates assume a 'last_modified' timestamp column enables easy CDC via polling (options B, D), but the question tests that true near real-time CDC with minimal source impact requires reading database redo logs, not querying the table, and that services like Kinesis or Glue cannot perform log-based CDC without additional custom logic.

How to eliminate wrong answers

Option B is wrong because Kinesis Data Firehose with a Lambda function that queries Oracle would require periodic polling of the source table, which either misses changes between polls or causes high load on the database if polled too frequently, and it does not natively support CDC from Oracle redo logs. Option C is wrong because AWS Data Pipeline with a periodic SQL query activity to copy full table snapshots performs full table scans, which impacts the source database and cannot capture changes in near real-time (only at scheduled intervals). Option D is wrong because AWS Glue with a JDBC connection running a crawler every 5 minutes performs full table scans or incremental queries based on the 'last_modified' column, but this still causes repeated query load on Oracle and cannot achieve true near real-time CDC without reading redo logs.

253
MCQhard

A data engineer is designing a data ingestion pipeline for clickstream data from a mobile app. The data volume varies, with occasional spikes up to 10 MB/s. The pipeline must persist the raw data in Amazon S3 and make it available for near-real-time analytics via Amazon Athena. Which combination of services minimizes cost and operational overhead?

A.Amazon Kinesis Data Streams with Amazon Kinesis Data Analytics, then Amazon S3
B.Amazon SQS with an Auto Scaling group of EC2 instances writing to Amazon S3
C.Amazon Kinesis Data Streams with AWS Lambda for transformation, then Amazon S3
D.Amazon Kinesis Data Firehose with direct delivery to Amazon S3, then Amazon Athena
AnswerD

Firehose is fully managed, scales automatically, and delivers to S3.

Why this answer

Amazon Kinesis Data Firehose is the most cost-effective and low-overhead solution for ingesting variable-volume clickstream data (up to 10 MB/s) into Amazon S3 because it is a fully managed service that automatically scales, buffers, and compresses data before delivery. It integrates directly with S3 without requiring custom code or infrastructure management, and the data is immediately queryable by Amazon Athena with no additional transformation steps.

Exam trap

The trap here is that candidates often choose Amazon Kinesis Data Streams with Lambda (Option C) because they think it provides more control, but they overlook Lambda's concurrency limits and the operational burden of managing stream shards, making Firehose the simpler and cheaper choice for raw data ingestion to S3.

How to eliminate wrong answers

Option A is wrong because Amazon Kinesis Data Analytics adds unnecessary cost and complexity for a pipeline that only needs to persist raw data to S3, and it requires a separate stream consumer. Option B is wrong because Amazon SQS with Auto Scaling EC2 instances introduces significant operational overhead for managing instance scaling, SQS polling, and potential data loss or duplication, and it is not optimized for near-real-time streaming at 10 MB/s. Option C is wrong because AWS Lambda has a maximum invocation duration of 15 minutes and a concurrency limit that can cause throttling during spikes, making it unsuitable for sustained 10 MB/s throughput without complex sharding and retry logic.

254
MCQeasy

A data engineer needs to ingest streaming data from thousands of IoT devices and immediately process each record with minimal latency. Which AWS service should be used as the ingestion point?

A.AWS Lambda
B.Amazon S3
C.Amazon Kinesis Data Streams
D.AWS Glue
AnswerC

Amazon Kinesis Data Streams ingests high-volume device telemetry with millisecond-level latency and preserves record order per shard, letting consumers process each record immediately. It satisfies the minimal-latency, per-record processing constraint, unlike Amazon S3 or Kinesis Data Firehose, which batch and buffer before delivery.

Why this answer

Amazon Kinesis Data Streams is designed as a massively scalable, low-latency ingestion service for streaming data, capable of handling thousands of producers (IoT devices) and delivering records to consumers within milliseconds. It durably stores data in shards for up to 365 days and integrates natively with Lambda, Kinesis Data Analytics, and Firehose for immediate processing. This makes it the canonical ingestion point for real-time IoT pipelines on AWS.

Exam trap

DEA-C01 often tests the distinction between ingestion services (Kinesis, MSK, IoT Core) and processing services (Lambda, Glue, EMR), so candidates mistakenly pick Lambda as the 'streaming' answer.

How to eliminate wrong answers

Option A is wrong because AWS Lambda is a compute service for processing events, not a durable ingestion buffer for thousands of concurrent producers; it can be a consumer of Kinesis but not the ingestion endpoint itself. Option B is wrong because Amazon S3 is object storage optimized for batch/throughput, not sub-second streaming ingestion, and it lacks native record-level streaming semantics. Option D is wrong because AWS Glue is a serverless ETL/catalog service for batch and micro-batch jobs, not a real-time streaming ingestion layer.

255
MCQhard

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is transformed using an AWS Lambda function. Recently, the transformation errors have increased due to Lambda timeouts. The data engineer needs to diagnose and resolve the issue without losing data. What should the engineer do?

A.Increase the Lambda function timeout and ensure that failed records are sent to a backup S3 bucket
B.Enable Amazon CloudWatch Logs for the Lambda function to capture errors and store failed records in CloudWatch
C.Configure the Lambda function to write failed records to an Amazon SQS queue for later reprocessing
D.Modify the Lambda function to store failed records in Amazon S3 before processing
AnswerA

Increasing timeout reduces failures, and configuring a backup bucket prevents data loss.

Why this answer

Increasing the Lambda function timeout directly addresses the root cause of transformation errors (timeouts), and configuring a backup S3 bucket for failed records ensures no data loss. Kinesis Data Firehose can be configured to send failed records to a separate S3 bucket as a dead-letter queue, which preserves the data for later reprocessing while the primary transformation pipeline is fixed.

Exam trap

The trap here is that candidates may confuse logging (CloudWatch Logs) with actual data preservation, or assume that SQS is a native Firehose failure destination, when in fact Firehose only supports S3 or Redshift as backup destinations for failed records.

How to eliminate wrong answers

Option B is wrong because enabling CloudWatch Logs captures error logs but does not store the actual failed records; it only provides visibility into errors without preventing data loss. Option C is wrong because Kinesis Data Firehose does not natively support sending failed records to an SQS queue; the Lambda function would need custom code to write to SQS, and this does not address the timeout issue. Option D is wrong because storing failed records in S3 before processing would require modifying the Lambda function to write to S3 first, which adds complexity and does not resolve the timeout; the records are already in the Firehose stream and need to be processed or redirected after failure.

256
MCQmedium

A data engineer is configuring an AWS Glue ETL job that reads from an Amazon S3 bucket containing sensitive customer records. The job must remove all columns flagged as PII before writing to a target S3 location. The PII columns are not known in advance and vary by file. Which approach should the engineer use to ensure the PII is removed dynamically?

A.Use AWS Glue's `DetectPII` transform to automatically identify and redact PII columns based on built-in patterns.
B.Use the AWS Glue DynamicFrame's `drop_fields` method with a list of PII column names derived from the Glue Data Catalog metadata.
C.Use AWS Glue DataBrew to create a recipe that identifies PII columns using pattern matching and apply it as a transformation step in the Glue job.
D.Use a custom PySpark script within the Glue job that scans column names and data samples against a regex pattern for PII, then drops matching columns.
AnswerD

A custom PySpark script can dynamically inspect each DataFrame's schema and sample data to apply regex patterns that identify PII-like content (e.g., email, SSN). It can then drop those columns before writing. This approach is flexible, scales with Glue's distributed processing, and handles varying PII columns per file, satisfying the requirement without relying on non-existent built-in transforms.

Why this answer

The requirement is to dynamically remove PII columns that vary by file. AWS Glue does not have a native PII detection transform, and the Data Catalog does not store PII classifications unless manually added. A custom PySpark script within the Glue job can inspect schema and data, apply regex patterns, and drop matching columns.

This provides the necessary flexibility and scalability.

Exam trap

The trap here is assuming AWS Glue has a built-in PII detection transform or that the Data Catalog automatically identifies PII columns.

257
MCQeasy

A company needs to ingest data from a MySQL database into Amazon S3 in near real-time. The database is running on EC2. The data engineer wants to minimize the impact on the source database. Which service should be used?

A.AWS Database Migration Service (DMS) with ongoing replication
B.AWS Glue ETL job with a JDBC connection
C.Amazon RDS for MySQL with read replica
D.AWS Schema Conversion Tool (SCT)
AnswerA

DMS with ongoing replication performs change data capture from the MySQL binary log, streaming inserts, updates and deletes to Amazon S3 continuously. Reading the log rather than polling tables keeps load on the source EC2 database minimal while achieving near real-time ingestion.

Why this answer

AWS DMS with ongoing replication (change data capture) is the correct choice because it can continuously replicate changes from a MySQL source database to Amazon S3 with minimal performance impact. DMS uses a transactional log-based approach (MySQL binlog) to capture changes as they occur, avoiding heavy SELECT queries on the source. This enables near real-time ingestion without adding significant load to the production database.

Exam trap

The trap here is that candidates often confuse AWS Glue's batch JDBC capabilities with streaming ingestion, or assume that a read replica can directly feed data into S3 without an intermediary service like DMS or Kinesis.

How to eliminate wrong answers

Option B is wrong because AWS Glue ETL jobs with JDBC connections run batch queries that pull full table snapshots or large result sets, which can cause significant performance degradation on the source MySQL database and cannot achieve near real-time latency. Option C is wrong because Amazon RDS for MySQL with a read replica is a database migration or read scaling solution, not a data ingestion service to S3; it does not natively stream data to S3 without additional tooling. Option D is wrong because AWS Schema Conversion Tool (SCT) is designed for converting database schemas between different database engines (e.g., Oracle to Aurora), not for ingesting data into S3.

258
MCQhard

A data engineer is designing a streaming pipeline that ingests IoT sensor data from 10,000 devices. Each device sends a 1 KB message every second. The data must be processed in near real-time and stored in S3 for analytics. Which combination of services provides the most cost-effective solution?

A.AWS Data Pipeline with periodic S3 copy.
B.Amazon Kinesis Data Streams with Kinesis Data Firehose delivery to S3.
C.Amazon MSK (Managed Streaming for Kafka) with Kafka Connect S3 sink.
D.Amazon SQS FIFO queue with Lambda consumers writing to S3.
AnswerB

Handles high throughput, Firehose batches to S3.

Why this answer

B is correct because Kinesis Data Streams ingests high-throughput IoT data (10,000 messages/sec at 1 KB each) with low latency, and Kinesis Data Firehose automatically batches and compresses data before delivering it to S3, eliminating the need for custom code or manual scaling. This combination provides the most cost-effective near-real-time solution by leveraging Firehose's built-in buffering and compression to minimize S3 storage costs and reduce the number of PUT requests.

Exam trap

The trap here is that candidates often choose MSK (Option C) thinking it is more scalable or flexible, but they overlook the higher operational cost and complexity for a simple S3 sink use case, where Kinesis Data Firehose's fully managed batching, compression, and direct S3 integration is more cost-effective and simpler to maintain.

How to eliminate wrong answers

Option A is wrong because AWS Data Pipeline is a batch-oriented orchestration service, not designed for near-real-time streaming ingestion; it would introduce significant latency and require periodic polling, failing the near-real-time requirement. Option C is wrong because Amazon MSK (Managed Streaming for Apache Kafka) introduces unnecessary operational overhead and cost for this use case, as it requires managing Kafka clusters, brokers, and the Kafka Connect S3 sink, while Kinesis Data Firehose provides a simpler, fully managed, and more cost-effective direct S3 delivery. Option D is wrong because SQS FIFO queues are designed for exactly-once processing and low throughput (300 transactions per second by default), making them unsuitable for 10,000 messages per second; additionally, Lambda consumers would incur high costs due to the large number of invocations and lack built-in batching and compression for S3 writes.

259
MCQmedium

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The job runs daily and processes only new data added since the last run. The engineer notices that the job is reprocessing old data, increasing runtime and cost. Which AWS Glue feature should be enabled to ensure only new data is processed?

A.AWS Glue DataBrew
B.Job bookmarks
C.Amazon S3 event notifications
D.AWS Glue Crawler incremental crawl
AnswerB

AWS Glue job bookmarks track the data that has already been processed in previous runs. When enabled, the job reads only new data based on the last modified time of objects in S3. This prevents reprocessing of old data, reducing runtime and cost. It is the standard feature for incremental data processing in AWS Glue.

Why this answer

Job bookmarks are the AWS Glue feature designed to track processed data and enable incremental processing. When enabled, the job reads only new data based on S3 object metadata. This reduces runtime and cost by avoiding reprocessing.

The other options do not provide this capability: DataBrew is for data preparation, crawler incremental crawl updates the catalog, and S3 event notifications are for triggering actions.

Exam trap

The trap here is confusing incremental crawls or event-driven triggers with job bookmarks, which are specifically needed for incremental ETL processing within AWS Glue.

260
MCQmedium

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The job must run incrementally and process only new data since the last run. The source data is partitioned by date in S3, and new partitions are added daily. Which AWS Glue feature should the engineer enable to track previously processed data?

A.Data Catalog partition index
B.Glue DataBrew profile
C.Glue FindMatches
D.Job bookmarks
AnswerD

Job bookmarks in AWS Glue track the state of data that has already been processed. When enabled, Glue stores bookmark state for each job run, allowing the job to process only new data since the last successful run. For S3 sources, bookmarks track objects and partitions based on their last modified time and other metadata. This directly addresses the requirement to process only new data incrementally without reprocessing old partitions.

Why this answer

Job bookmarks are the AWS Glue feature designed to track previously processed data. When enabled, Glue maintains state for each job, so subsequent runs process only new data, such as new S3 partitions or files. This is the standard way to implement incremental ETL in Glue without reprocessing historical data.

Exam trap

The trap here is confusing Data Catalog partition indexes, which optimize metadata queries, with job bookmarks, which track processed data for incremental runs.

261
MCQhard

A data engineer is using AWS Glue DataBrew to clean a dataset stored in Amazon S3. The dataset contains a column with inconsistent date formats (for example, '2023-01-15', '01/15/2023', and '15-Jan-2023'). The engineer needs to standardize all values to ISO 8601 format and then write the cleaned data back to S3. Which approach should the engineer use?

A.Use the 'Standardize date' transform in DataBrew, select the column, and choose the target format as ISO 8601.
B.Use the 'Format' transform in DataBrew to change the column data type to date.
C.Use the 'Replace value' transform to manually replace each non-ISO date string with its ISO equivalent.
D.Write a custom Python function in a DataBrew recipe step using the 'Custom transform' option.
AnswerA

AWS Glue DataBrew provides a built-in 'Standardize date' transform that can parse multiple date formats and convert them to a consistent target format such as ISO 8601. The engineer selects the column, configures the transform, and DataBrew applies it across the dataset. The cleaned data can then be written back to S3 as part of a DataBrew recipe or job.

Why this answer

AWS Glue DataBrew includes a 'Standardize date' transform that parses multiple date formats and converts them to a specified target format such as ISO 8601. This transform is designed for exactly this scenario, where a column contains mixed date representations. Using it avoids custom code and ensures consistent, repeatable cleaning across the dataset.

Exam trap

The trap here is assuming that a custom transform is needed for mixed date formats, when DataBrew already provides a built-in date standardization transform.

262
MCQhard

A data engineer is troubleshooting a Lambda function that reads from a Kinesis Data Stream, processes records, and writes to a Kinesis Data Firehose delivery stream. The Firehose delivery stream is configured to deliver data to an S3 bucket. The Lambda function is failing with an access denied error. The IAM policy attached to the Lambda execution role is shown in the exhibit. Which permission is missing?

A.firehose:PutRecord on the Firehose delivery stream
B.firehose:DescribeDeliveryStream on the Firehose delivery stream
C.logs:CreateLogGroup and logs:CreateLogStream on the CloudWatch log group
D.s3:PutObjectAcl on the S3 bucket
AnswerA

Firehose ingestion uses the PutRecord (and PutRecordBatch) API, distinct from kinesis:PutRecord used for Kinesis Data Streams. Without firehose:PutRecord on the delivery stream, the Lambda execution role's write attempt is denied, producing the access denied error.

Why this answer

The Lambda function is failing due to a missing permission to write to the Kinesis Data Firehose delivery stream. The required permission is firehose:PutRecord (or firehose:PutRecordBatch). The IAM policy likely lacks this permission, causing an access denied error when the Lambda attempts to write records.

Option A correctly identifies this missing permission.

Exam trap

Candidates often confuse permissions for Kinesis Data Streams and Kinesis Data Firehose. Writing to a Firehose delivery stream requires firehose:PutRecord, not kinesis:PutRecord, which is for Kinesis Data Streams.

How to eliminate wrong answers

Option A is wrong because the Lambda function reads from the Kinesis Data Stream (requiring `kinesis:GetRecords`, `kinesis:DescribeStream`, etc.), not writes to it, so `kinesis:PutRecord` is irrelevant. Option C is wrong because CloudWatch Logs permissions (`logs:CreateLogGroup`, `logs:CreateLogStream`) are needed for logging but would cause a different error (e.g., 'Unable to create log stream'), not an access denied on Firehose. Option D is wrong because `s3:PutObjectAcl` is not required for Firehose to deliver to S3; Firehose uses `s3:PutObject` with bucket owner full control by default, and ACLs are not involved in this scenario.

263
MCQeasy

A company wants to ingest streaming data from thousands of IoT devices into AWS for real-time processing. Each device sends JSON payloads of about 2 KB at a rate of 1 message per second. The data must be processed with a durable, ordered stream per device. Which service should the company use as the ingestion layer?

A.Amazon Simple Queue Service (Amazon SQS) with a FIFO queue.
B.Amazon Kinesis Data Streams.
C.Amazon Simple Notification Service (Amazon SNS) with a Lambda subscriber.
D.Amazon Kinesis Data Firehose with Direct Put.
AnswerB

Amazon Kinesis Data Streams partitions records by partition key, giving each device a durable, strictly ordered shard sequence, satisfying the per-device ordering constraint. It also ingests thousands of concurrent 2 KB/second producers without provisioning brokers, unlike Amazon MSK, which requires cluster management, or SQS, which offers no ordering guarantee.

Why this answer

Amazon Kinesis Data Streams is the correct choice because it provides durable, ordered stream processing per shard, which can be partitioned by device ID to maintain message order for each device. It supports real-time ingestion from thousands of IoT devices at 2 KB per message per second, with a retention period of up to 365 days and the ability to reprocess data via multiple consumers.

Exam trap

The trap here is that candidates often confuse Kinesis Data Streams with Kinesis Data Firehose, assuming Firehose can handle real-time ordered streams, but Firehose does not provide per-record ordering or real-time consumer access; it is a delivery stream, not a stream processing layer.

How to eliminate wrong answers

Option A is wrong because Amazon SQS FIFO queues guarantee exactly-once processing and strict ordering within a message group, but they are designed for decoupling microservices, not for high-throughput streaming ingestion from thousands of devices; FIFO throughput is limited to 300 transactions per second (with batching) and does not support multiple consumers reading the same stream in real-time. Option C is wrong because Amazon SNS is a pub/sub messaging service that does not provide ordered delivery or durable stream storage; it pushes messages to subscribers like Lambda, but ordering is not guaranteed and messages are not persisted for replay. Option D is wrong because Amazon Kinesis Data Firehose is a near-real-time delivery service that buffers data and writes it to destinations like S3 or Redshift, but it does not support ordered stream processing per device and cannot be consumed by multiple real-time applications directly; it is designed for batch loading, not for real-time ordered stream consumption.

264
MCQmedium

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data is then processed by an AWS Lambda function that transforms the records and writes them to an Amazon S3 bucket. Recently, the Lambda function has been timing out and the S3 bucket is not receiving all expected data. The Kinesis stream is not throttling and has sufficient shards. Which step should the company take to resolve this issue?

A.Increase the Lambda function's reserved concurrency.
B.Increase the Lambda function's timeout and memory allocation.
C.Increase the number of shards in the Kinesis stream.
D.Enable enhanced fan-out on the Kinesis stream to reduce latency.
AnswerB

Lambda timeouts arise when per-invocation processing exceeds the configured limit, and memory allocation proportionally increases CPU, accelerating transformation. Raising both values lets each batch complete before the timeout, so all Kinesis records reach S3 without the stream itself being throttled.

Why this answer

The Lambda function is timing out, indicating that it cannot process the records within the allotted time. Increasing the Lambda function's timeout and memory allocation (Option B) provides more execution time and CPU resources, which can help process larger or more frequent records before timing out. Option A is incorrect because reserved concurrency limits the maximum number of concurrent instances, not the execution duration of a single instance.

Option C is incorrect because increasing shards is unnecessary; the stream is not throttling, and more shards do not resolve Lambda timeouts. Option D is incorrect because enhanced fan-out reduces latency for multiple consumers but does not prevent a single Lambda function from timing out due to processing time.

265
MCQhard

A CloudFormation template defines an AWS Glue job. The job fails during execution with the error 'Unable to locate script: s3://scripts-bucket/etl-script.py'. The S3 bucket 'scripts-bucket' exists and the script file is present. What is the most likely cause?

A.The script location path is incorrect; it should include the bucket's region.
B.The IAM role for the Glue job does not have s3:GetObject permission on the scripts bucket.
C.The Glue job requires Python version 2, but the script uses Python 3 syntax.
D.The S3 bucket is in a different AWS region than the Glue job.
AnswerB

Glue retrieves the script from S3 using the job's IAM role, so a missing s3:GetObject permission on scripts-bucket produces exactly this 'Unable to locate script' error even though the object exists. The bucket policy and object presence are irrelevant if the role is denied.

Why this answer

The Glue job fails to locate the script because the IAM role assigned to the job lacks the s3:GetObject permission on the scripts-bucket. Even though the bucket and object exist, AWS Glue requires the execution role to have explicit read access to the S3 object to download and execute the script. Without this permission, the job cannot retrieve the file, resulting in the 'Unable to locate script' error.

Exam trap

The trap here is that candidates assume the error is about the script path or region mismatch, but the real cause is almost always missing IAM permissions for the Glue execution role to read the script from S3.

How to eliminate wrong answers

Option A is wrong because S3 object paths do not include the bucket's region; the path format is s3://bucket-name/key, and region is irrelevant to the path. Option C is wrong because Python version compatibility would cause a syntax error during execution, not a 'Unable to locate script' error, which is a file access issue. Option D is wrong because S3 buckets and Glue jobs can operate across regions as long as the Glue job's IAM role has appropriate cross-region permissions; the error message specifically indicates a missing object, not a region mismatch.

266
MCQmedium

A company uses AWS Data Pipeline to copy data from DynamoDB to S3 daily. Recently, the pipeline started failing with 'ThrottlingException' errors. The DynamoDB table has on-demand capacity. Which action should be taken to resolve the issue?

A.Increase the write capacity units of the DynamoDB table.
B.Replace Data Pipeline with AWS Glue using a DynamoDB connector.
C.Configure the pipeline to use a retry strategy with exponential backoff.
D.Disable the pipeline's retry logic and increase the timeout.
AnswerC

On-demand DynamoDB tables still throttle when request rates spike above what the table can absorb, and Data Pipeline's default retries are too aggressive. Adding a retry strategy with exponential backoff spaces out retries, letting the table recover and the copy complete without manual intervention.

Why this answer

ThrottlingException errors in AWS Data Pipeline when reading from DynamoDB indicate that the pipeline's read requests are exceeding the table's available throughput. Since the table uses on-demand capacity, which can handle spikes but has a per-second throughput limit, implementing exponential backoff in the pipeline's retry strategy allows it to reduce request rate upon throttling, aligning with AWS SDK best practices for handling DynamoDB throttling.

Exam trap

The trap here is that candidates assume on-demand capacity eliminates all throttling, but it only handles traffic spikes within a per-second limit, so throttling can still occur with sustained high read rates, and the correct fix is to implement exponential backoff in the pipeline's retry strategy rather than modifying capacity or switching tools.

How to eliminate wrong answers

Option A is wrong because DynamoDB on-demand capacity does not use provisioned write capacity units; increasing write capacity units is irrelevant and would require switching to provisioned mode, which is unnecessary. Option B is wrong because replacing Data Pipeline with AWS Glue using a DynamoDB connector does not inherently resolve throttling; Glue also uses the same DynamoDB read APIs and would face the same throttling issue without proper retry handling. Option D is wrong because disabling retry logic and increasing the timeout would cause the pipeline to fail permanently on the first throttling error, as it would not retry the request, and a longer timeout does not prevent throttling.

267
MCQhard

A data engineer is designing a data ingestion pipeline for real-time financial transactions. The pipeline must ensure exactly-once processing semantics and must handle duplicate records that may occur due to retries. Which combination of AWS services can achieve exactly-once processing?

A.Amazon Kinesis Data Streams with Amazon Kinesis Data Analytics for Apache Flink
B.Amazon SQS with AWS Lambda
C.Amazon MSK with AWS Lambda
D.Amazon Kinesis Data Firehose with AWS Lambda
AnswerA

Flink supports exactly-once processing with KDS.

Why this answer

Amazon Kinesis Data Streams with Kinesis Data Analytics for Apache Flink enables exactly-once processing by leveraging Flink's built-in checkpointing and two-phase commit protocols. Kinesis Data Streams provides a durable, ordered record store, while Flink uses its internal state and idempotent sinks to deduplicate records that arise from retries, ensuring each record is processed exactly once.

Exam trap

The trap here is that candidates often assume SQS FIFO or Lambda's idempotency can achieve exactly-once, but they overlook that without a stream processing framework with checkpointing and transactional sinks, retries can still introduce duplicates in distributed systems.

How to eliminate wrong answers

Option B is wrong because Amazon SQS with AWS Lambda provides at-least-once delivery by default; SQS does not guarantee deduplication without a FIFO queue and idempotent Lambda logic, and even then, exactly-once is not natively supported across retries. Option C is wrong because Amazon MSK (Kafka) with AWS Lambda requires custom checkpointing and idempotency logic in the Lambda function; MSK does not natively provide exactly-once semantics without a stream processing framework like Flink or Kafka Streams. Option D is wrong because Amazon Kinesis Data Firehose with AWS Lambda delivers records at least once and cannot guarantee exactly-once processing; Firehose buffers and batches data but does not support transactional commits or deduplication across retries.

268
MCQeasy

A company needs to ingest data from multiple SaaS sources (e.g., Salesforce, Marketo) into Amazon S3 for analytics. Which AWS service is designed for this purpose?

A.AWS Transfer Family
B.AWS Glue
C.Amazon AppFlow
D.AWS DataSync
AnswerC

Amazon AppFlow provides managed connectors for SaaS sources such as Salesforce and Marketo, transferring data directly into Amazon S3 without custom code. This satisfies the stem's requirement to ingest from multiple SaaS applications into S3 for analytics.

Why this answer

Amazon AppFlow is a fully managed integration service specifically designed to securely transfer data between SaaS applications (like Salesforce, Marketo, Slack, and Zendesk) and AWS services such as Amazon S3 and Amazon Redshift. It supports scheduled, event-driven, or on-demand data ingestion with built-in transformations, filtering, and validation, making it the ideal choice for this use case.

Exam trap

The trap here is that candidates often confuse AWS Glue's ETL capabilities with native SaaS connectivity, but Glue requires custom connectors or AWS Glue Studio's visual ETL jobs to connect to SaaS sources, whereas AppFlow is purpose-built for this task.

How to eliminate wrong answers

Option A is wrong because AWS Transfer Family is used for transferring files into and out of Amazon S3 or Amazon EFS using SFTP, FTPS, or FTP protocols, not for integrating with SaaS APIs. Option B is wrong because AWS Glue is a serverless data integration service for ETL (extract, transform, load) jobs, but it does not natively connect to SaaS sources like Salesforce or Marketo without custom connectors or third-party libraries. Option D is wrong because AWS DataSync is designed for moving large volumes of data between on-premises storage and AWS (e.g., NFS, SMB, S3) or between AWS storage services, not for ingesting data from SaaS applications.

269
Multi-Selecteasy

A company is building a data lake on Amazon S3 and needs to ingest data from various on-premises sources. Which TWO AWS services can be used to transfer data securely over the internet?

Select 2 answers
A.AWS Snowcone
B.AWS DataSync
C.Amazon Kinesis Data Firehose
D.AWS Direct Connect
E.AWS CLI
AnswersB, E

AWS DataSync transfers files over the internet using TLS encryption, satisfying the secure on-premises ingestion requirement. It supports NFS, SMB, and object storage sources, with bandwidth throttling and integrity verification. Agents deploy on-premises, replicating data directly to Amazon S3 without VPN or AWS Direct Connect, which the stem does not mandate.

Why this answer

AWS DataSync (B) is correct because it is a purpose-built data transfer service that moves data between on-premises storage and Amazon S3 over the internet using TLS encryption, and it supports scheduled, incremental, and bandwidth-limited transfers. AWS CLI (E) is correct because commands such as aws s3 cp or aws s3 sync transfer data from on-premises systems to S3 over the public internet using HTTPS, making it a valid secure internet-based ingestion method. AWS Snowcone (A) is not correct here because it is an edge computing and offline/physical data transfer device, not an internet-based transfer service.

Amazon Kinesis Data Firehose (C) is not correct because it delivers streaming data to destinations like S3 but is not designed to pull files from on-premises sources over the internet. AWS Direct Connect (D) is not correct because it uses a dedicated private network connection rather than the public internet.

Exam trap

The trap here is that candidates often confuse AWS DataSync with AWS Direct Connect, thinking both are required for secure transfers, but Direct Connect is a network service that bypasses the internet entirely, while DataSync is a data transfer service that works over the internet or Direct Connect.

270
Multi-Selectmedium

A company is building a data lake on Amazon S3. They need to ingest data from multiple sources, including relational databases, streaming data, and log files. Which THREE AWS services can be used to ingest data into the data lake?

Select 3 answers
A.Amazon Kinesis Data Firehose
B.AWS Database Migration Service (DMS)
C.Amazon Athena
D.Amazon Redshift Spectrum
E.AWS Glue
AnswersA, B, E

Ingests streaming data into S3.

Why this answer

Amazon Kinesis Data Firehose is a fully managed service for streaming data ingestion that can capture, transform, and load streaming data into Amazon S3 in near real-time. It supports sources like Amazon CloudWatch Logs, AWS IoT, and custom producers via the Kinesis Agent, making it ideal for log files and streaming data.

Exam trap

The trap here is confusing query engines (Athena, Redshift Spectrum) with ingestion services, as candidates often assume any service that touches S3 can be used for data loading.

271
MCQeasy

A data engineer must load a 50 GB uncompressed CSV file from Amazon S3 into an Amazon Redshift cluster using the COPY command. The load is taking a long time and the engineer wants to improve performance. Which action should the engineer take?

A.Convert the CSV file to JSON and load it with the COPY command using the JSON 'auto' option.
B.Compress the file with gzip and split it into multiple smaller files, then run the COPY command with the GZIP option.
C.Use the COPY command with the PARALLEL OFF option to force Redshift to distribute the load across all slices.
D.Increase the cluster's node count and rerun the COPY command on the same single uncompressed file.
AnswerB

Compressing the data reduces the bytes transferred from Amazon S3 and splitting the file into multiple parts lets Redshift load slices in parallel across the cluster. The COPY command supports the GZIP parameter to decompress on ingest. Together these changes dramatically reduce load time compared with a single large uncompressed file.

Why this answer

Redshift COPY performance depends on parallelizing reads across slices and minimizing bytes transferred. Compressing the CSV with gzip reduces transfer volume, and splitting it into multiple files lets each slice load a portion concurrently. The GZIP parameter handles decompression during ingest.

A single large uncompressed file limits parallelism regardless of cluster size.

Exam trap

The trap here is believing that adding cluster nodes automatically speeds up loading a single large file.

272
MCQhard

A company uses AWS Glue to run ETL jobs that process data from Amazon RDS to Amazon S3. The jobs run nightly and take 3 hours to complete. The data volume is growing by 20% each month. The engineer needs to reduce job runtime and cost. The source RDS is a db.r5.large instance. Which approach would be MOST effective?

A.Reduce the number of DPUs to lower cost and accept longer runtime.
B.Increase the number of Glue workers and choose a G.1X or G.2X worker type.
C.Create a read replica of the RDS instance and point the Glue job to the replica.
D.Enable S3 Transfer Acceleration on the destination bucket.
AnswerB

Glue scales horizontally, so adding workers and moving to G.1X or G.2X types increases per-worker compute and memory, cutting the three-hour runtime. This addresses the growing data volume without the cost of resizing the source RDS instance, which is not the bottleneck.

Why this answer

Increasing the number of Glue workers and choosing a G.1X or G.2X worker type directly increases parallelism and provides more memory per worker, reducing job runtime at a manageable cost increase. Option A is wrong; reducing DPUs would increase runtime, not reduce it. Option C is wrong because a read replica does not improve Glue processing speed; the bottleneck is Glue's processing capacity, not the source database's read capacity.

Option D is wrong because S3 Transfer Acceleration improves upload speed to S3, not Glue job processing.

273
Multi-Selectmedium

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The data must be transformed and stored in Amazon S3 for batch analytics. The engineer wants to use AWS Lambda for transformation. Which TWO configurations are required? (Choose two.)

Select 2 answers
A.Configure the Lambda function to write to S3 via Kinesis Data Firehose.
B.Configure the Kinesis stream to send records directly to S3.
C.Set up an SQS queue as a destination for Lambda errors.
D.Create an event source mapping from the Kinesis stream to the Lambda function.
E.Assign an IAM role to Lambda with permissions to read from Kinesis and write to S3.
AnswersD, E

An event source mapping registers the Kinesis stream as a Lambda trigger, letting the service poll shards, batch records, and invoke the function with checkpointing. Without it, Lambda cannot consume the stream for transformation before writing to S3.

Why this answer

Option D is correct because AWS Lambda consumes records from Kinesis Data Streams through an event source mapping, which polls the stream using the enhanced fan-out or standard iterator and invokes the function with batches of records. Option E is correct because the Lambda execution role must include IAM permissions for the required actions, such as kinesis:GetRecords, kinesis:GetShardIterator, kinesis:DescribeStream, and kinesis:ListShards to read the stream, plus s3:PutObject to store the transformed data in Amazon S3. Option A is not required because Kinesis Data Firehose is a separate delivery service; the Lambda function can write directly to S3 using the AWS SDK, and Firehose is not a mandatory intermediary.

Option B is incorrect because a Kinesis data stream cannot send records directly to Amazon S3; it requires a consumer such as Lambda or Firehose. Option C is incorrect because an SQS dead-letter queue for Lambda errors is an optional reliability configuration, not a required configuration for transforming and storing the data.

Exam trap

DEA-C01 often tests the required components for Lambda-Kinesis integration, and candidates may confuse optional error handling (SQS) or alternative services (Firehose) with mandatory configurations.

274
MCQmedium

A data engineer is using AWS Glue DataBrew to clean a dataset stored in Amazon S3. The recipe must replace all null values in a specific column with the string 'UNKNOWN' and then convert the column to uppercase. The engineer wants to apply these steps in a repeatable recipe. Which combination of DataBrew transforms should be used?

A.Use the 'Fill missing values' transform with a custom value, then the 'Upper case' transform on the column.
B.Use the 'Remove nulls' transform, then the 'Upper case' transform.
C.Use the 'Replace value' transform with a regex for null, then the 'Upper case' transform.
D.Use the 'Custom formula' transform with an IFNULL expression, then the 'Upper case' transform.
AnswerA

The 'Fill missing values' transform replaces nulls with a specified value such as 'UNKNOWN', and the 'Upper case' transform converts the column's text to uppercase. Applied in sequence within a recipe, they achieve the required cleaning and normalization. DataBrew recipes are repeatable and can be scheduled or run as jobs.

Why this answer

DataBrew provides a dedicated 'Fill missing values' transform for substituting nulls with a constant, and an 'Upper case' transform for case normalization. Together they form a repeatable recipe that preserves rows and produces the required output. Removing nulls or using regex replacement does not achieve the same result.

Exam trap

The trap here is assuming that nulls can be handled with a regex-based replace transform, when DataBrew requires the dedicated fill-missing-values transform for null substitution.

275
MCQmedium

A data engineer is designing a data ingestion pipeline to load data from an on-premises Oracle database into Amazon Redshift. The pipeline must capture changes (inserts, updates, deletes) with low latency and minimal impact on the source database. Which combination of AWS services should the engineer use?

A.AWS Database Migration Service (DMS) to Amazon S3, then COPY into Redshift
B.Amazon Kinesis Data Streams with a custom producer on Oracle
C.AWS Glue with JDBC connection to Oracle, writing to Redshift
D.AWS Lambda reading from Oracle logs and writing to Redshift
AnswerA

AWS DMS reads Oracle redo logs via change data capture, so inserts, updates and deletes replicate continuously without querying source tables, satisfying the low-latency and minimal-impact constraints. Landing changes in Amazon S3 lets Redshift COPY them in, though deletes require merge handling.

Why this answer

AWS DMS with ongoing replication (change data capture) can capture changes from Oracle with minimal impact and replicate to S3, then COPY into Redshift. Option B is wrong because Kinesis Data Streams requires a custom producer to capture Oracle changes and does not provide native CDC from Oracle databases. Option C is wrong because AWS Glue with JDBC is batch-oriented and does not support real-time change data capture natively.

Option D is wrong because AWS Lambda can process events but is not designed for continuous, low-latency CDC from a database and would require polling or triggers.

276
Multi-Selecthard

A company uses a Kinesis Data Firehose delivery stream to load data into an S3 bucket. The data is in JSON format and must be converted to Parquet before landing in S3. Which steps are required to achieve this? (Choose THREE.)

Select 3 answers
A.Configure the Firehose delivery stream to enable data format conversion to Parquet.
B.Create a table in the AWS Glue Data Catalog with the schema.
C.Store the schema in Amazon DynamoDB.
D.Set the Firehose's schema mapping to reference the Glue table.
E.Use Kinesis Data Analytics to convert the data.
AnswersA, B, D

Firehose has built-in conversion capability.

Why this answer

Kinesis Data Firehose natively supports converting incoming data from JSON to Parquet format. This conversion is enabled directly in the delivery stream configuration, eliminating the need for separate processing steps.

Exam trap

The trap here is that candidates may think DynamoDB is needed for schema storage or that Kinesis Data Analytics is required for the conversion, but Firehose's built-in Parquet conversion with Glue schema support is the correct and simpler approach.

277
MCQmedium

A company uses Amazon S3 to store raw data and needs to transform it into Parquet format for analytics. The transformation job runs daily on a schedule. Which AWS service is BEST suited for this task?

A.Amazon Redshift
B.Amazon EMR
C.AWS Lambda
D.AWS Glue
AnswerD

AWS Glue provides serverless Spark-based ETL with a scheduler and built-in Parquet conversion, so the daily transformation runs without managing infrastructure. This satisfies the requirement to convert S3 raw data to Parquet on a recurring schedule.

Why this answer

AWS Glue is a fully managed, serverless ETL service that can automatically convert data formats (e.g., from CSV to Parquet) and run on a schedule (e.g., daily). It is ideal for this use case because it is purpose-built for ETL transformations and handles schema discovery, data cataloging, and job scheduling without managing infrastructure. Option A (Amazon Redshift) is wrong because Redshift is a data warehouse for querying, not a transformation service; it could load Parquet but not convert raw data to Parquet directly.

Option B (Amazon EMR) is wrong because EMR requires provisioning and managing clusters, adding operational overhead. Option C (AWS Lambda) is wrong because Lambda has a maximum execution timeout of 15 minutes, which is too short for daily large-scale data transformation jobs.

278
MCQmedium

A data engineer runs an AWS Glue job that reads from a JDBC connection to a PostgreSQL database. The job fails with a 'Connection timed out' error. The Glue job runs in a VPC with the appropriate security group. What is the most likely cause?

A.The network ACL associated with the Glue job's subnet is blocking outbound traffic.
B.The Glue job does not have permission to access the database.
C.The security group does not allow inbound traffic from the Glue job.
D.The database credentials are incorrect.
AnswerA

Security groups are stateful and already correct, so the remaining subnet-level control is the network ACL. A stateless NACL blocking outbound traffic to the PostgreSQL port or return ephemeral range would cause the JDBC connection to time out, matching the reported error.

Why this answer

A network ACL (NACL) is a stateless subnet-level firewall that can block outbound traffic. If the NACL associated with the Glue job's subnet does not allow outbound traffic to the PostgreSQL database's port (e.g., 5432), the connection times out. Since the security group is stated to be appropriate, the NACL is the most likely remaining network-layer cause.

Exam trap

The trap is confusing security groups with NACLs—candidates often blame the security group, but when the SG is stated as correct, the stateless NACL is the next network-layer suspect for timeouts.

How to eliminate wrong answers

Option B is wrong because a permissions issue would typically produce an authentication or authorization error (e.g., 'access denied'), not a connection timeout—timeouts indicate network reachability problems. Option C is wrong because security groups are stateful and the question says the security group is appropriate; also, the Glue job initiates outbound connections, so inbound rules on the Glue side are not the issue. Option D is wrong because incorrect credentials would yield an authentication failure, not a timeout.

279
Multi-Selecthard

Which THREE factors should be considered when choosing between AWS Glue and Amazon EMR for data transformation? (Choose three.)

Select 3 answers
A.Glue automatically stores data in S3 after transformation.
B.EMR allows fine-grained control over cluster configuration and software.
C.EMR supports real-time stream processing with Spark Streaming.
D.Glue is serverless, reducing operational overhead.
E.Glue integrates natively with the Glue Data Catalog for schema management.
AnswersB, D, E

EMR provides flexibility to install custom software and tune clusters.

Why this answer

Amazon EMR provides full control over cluster configuration, including the ability to customize software, install libraries, and tune Spark, Hadoop, or Hive parameters. This fine-grained control is essential for complex or specialized data transformation pipelines that require specific versions or custom configurations.

Exam trap

The trap here is that candidates may confuse Glue's automatic schema discovery with automatic data storage, or assume EMR is the only option for streaming, when in fact both services support streaming but with different levels of control and operational overhead.

280
MCQmedium

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data is consumed by an AWS Lambda function that writes to Amazon DynamoDB. The Lambda function is seeing high error rates due to DynamoDB write throttling. Which action should be taken to reduce throttling?

A.Use Amazon Kinesis Data Firehose instead of Kinesis Data Streams
B.Add an Amazon SQS queue between Lambda and DynamoDB
C.Increase the Lambda function memory
D.Enable auto scaling on the DynamoDB table
AnswerD

Enabling DynamoDB auto scaling adjusts provisioned write capacity automatically as consumed capacity rises, directly relieving the write throttling that Lambda is hitting. Since the stem identifies DynamoDB write throttling as the error source, scaling the table's write capacity units satisfies that constraint without changing the Kinesis or Lambda components.

Why this answer

Enabling DynamoDB auto scaling increases write capacity automatically when needed. Using Kinesis Data Firehose would change the architecture but does not address throttling directly. Increasing Lambda memory does not help with DynamoDB throttling.

Using SQS would add a queue but does not increase DynamoDB capacity.

281
MCQeasy

A company is using AWS Glue to run ETL jobs that transform data from Amazon DynamoDB to Amazon S3. The DynamoDB table has a large number of items (over 10 million) and is heavily used by production applications. The Glue job reads the entire DynamoDB table each time it runs, causing increased read capacity consumption and affecting production performance. The team wants to reduce the impact on the source DynamoDB table while still keeping the S3 data up-to-date. What should the team do?

A.Use DynamoDB Streams and AWS Lambda to capture changes and write them to S3, then run incremental Glue jobs.
B.Increase the DynamoDB read capacity units to handle the Glue job's read load.
C.Use the DynamoDB console to export the table to S3 in Parquet format.
D.Reduce the parallelism of the Glue job to lower the read throughput.
AnswerA

DynamoDB Streams capture item-level changes without consuming provisioned read capacity, so the production table is no longer scanned in full. Lambda writes those changes to S3, and Glue then processes only the incremental delta, satisfying the requirement to keep S3 current while removing the read-capacity impact on the heavily used source table.

Why this answer

Using DynamoDB Streams and AWS Lambda to capture changes and write them to S3, then running incremental Glue jobs, reduces the read load on the DynamoDB table because only changed data is processed. This approach keeps S3 data up-to-date without scanning the entire table, minimizing impact on production performance.

Exam trap

DEA-C01 often tests the misconception that increasing read capacity or reducing parallelism solves the impact on DynamoDB, but the real solution is to avoid full table scans by using change data capture with Streams and Lambda.

How to eliminate wrong answers

Option B is wrong because increasing read capacity units would allow the Glue job to read more but does not reduce the impact on the table; it just provides more capacity, which may be costly and still affects performance if the table is heavily used. Option C is wrong because exporting the table to S3 in Parquet format is a full export and does not provide incremental updates, and it still consumes read capacity. Option D is wrong because reducing parallelism lowers read throughput but also slows down the job, and it does not eliminate the full table scan; it just makes it slower, which may not keep S3 up-to-date efficiently.

282
MCQeasy

A data engineer is troubleshooting a Kinesis Data Firehose delivery stream that ingests JSON log data from web servers. The stream is configured to transform records with an AWS Lambda function and deliver to an Amazon S3 bucket. Recently, the stream has been failing with 'InvalidData' errors. Which action should the engineer take to resolve the issue?

A.Verify the S3 bucket policy allows Firehose to write.
B.Increase the buffer size and interval in the Firehose delivery stream.
C.Change the data format to CSV in the Firehose configuration.
D.Check the CloudWatch Logs for the Lambda function to identify transformation errors.
AnswerD

Firehose returns InvalidData when the Lambda transform function throws an error or returns a malformed record. Inspecting the function's CloudWatch Logs reveals the exact transformation exception, letting the engineer correct the code rather than guessing at stream configuration.

Why this answer

The 'InvalidData' error in Kinesis Data Firehose typically indicates that the Lambda function used for data transformation is failing or returning malformed records. By checking the CloudWatch Logs for the Lambda function, the engineer can identify specific transformation errors, such as incorrect JSON parsing, missing fields, or exceptions, which cause Firehose to reject the records. This is the most direct troubleshooting step because Firehose relies on the Lambda function to return valid transformed data in the expected format.

Exam trap

The trap here is that candidates often confuse 'InvalidData' errors with S3 permission issues or buffer configuration problems, but the error specifically points to a failure in the data transformation step, not the delivery destination or batching settings.

How to eliminate wrong answers

Option A is wrong because if the S3 bucket policy were the issue, the error would be a permission or access denied error, not 'InvalidData'. Option B is wrong because increasing buffer size or interval would not resolve data transformation errors; it only affects how data is batched before delivery. Option C is wrong because changing the data format to CSV would not fix transformation errors; Firehose expects the Lambda function to return data in the same format as the input (JSON) unless explicitly configured otherwise, and the 'InvalidData' error is unrelated to the output format.

283
MCQhard

A data engineer is using AWS Glue to run an ETL job that reads from Amazon S3, performs a join between two large datasets, and writes the result to Amazon Redshift. The job is taking longer than expected, and the engineer suspects data skew. Which technique can help mitigate data skew in the join?

A.Enable the 'spark.sql.adaptive.enabled' configuration and set 'spark.sql.adaptive.skewJoin.enabled' to true.
B.Use AWS Glue's 'groupFiles' option to combine small files before the join.
C.Increase the number of DPUs to provide more memory for the join operation.
D.Use a broadcast join to replicate the smaller dataset across all nodes.
AnswerA

Adaptive Query Execution (AQE) in Spark dynamically optimizes query plans at runtime. Enabling skew join handling allows AQE to detect skewed partitions and split them into smaller sub-partitions, balancing the workload. This significantly reduces the impact of data skew during joins, improving job performance.

Why this answer

Enabling Adaptive Query Execution with skew join handling allows Spark to dynamically detect and split skewed partitions during the join. This redistributes the workload more evenly across executors, mitigating the performance bottleneck caused by data skew.

Exam trap

The trap here is assuming that adding more resources or using broadcast join will solve skew, when in fact skew requires specific runtime optimization techniques like AQE.

284
Multi-Selecthard

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The job must handle upserts (inserts and updates) into an existing Redshift table based on a primary key. The engineer needs to ensure that the job efficiently processes only changed records and minimizes data movement. Which two AWS Glue features or techniques should be used to achieve this? (Choose two.)

Select 2 answers
A.Enable job bookmarks to track previously processed data and only process new or changed files in Amazon S3.
B.Use the 'write_dynamic_frame.from_jdbc_conf' method with the 'preactions' and 'postactions' parameters to run SQL commands before and after writing.
C.Stage the transformed data in Amazon S3 and use a Redshift COPY command to load into a staging table, then perform a MERGE operation.
D.Use the 'glueContext.write_dynamic_frame.from_options' with 'upsert' mode and specify the primary key.
E.Configure the Glue job to use the 'redshift-upsert' connection type and set the 'mergeKey' parameter.
AnswersA, C

This option is correct because AWS Glue job bookmarks maintain state information about data that has already been processed. When reading from Amazon S3, bookmarks allow the job to process only new files or files that have changed since the last run. This reduces the amount of data read and transformed, improving efficiency and minimizing data movement, which is essential for incremental upserts.

Why this answer

To efficiently upsert data into Redshift, job bookmarks reduce processing to only new or changed S3 files, and staging the data in S3 followed by a COPY into a staging table and a MERGE operation applies changes without full reloads. These techniques together minimize data movement and ensure only changed records are processed.

Exam trap

The trap here is assuming AWS Glue has native upsert capabilities for Redshift, when in fact upserts require manual staging and merge logic.

285
Multi-Selectmedium

A company is designing a data ingestion pipeline for real-time sensor data from thousands of devices. The data must be processed with low latency and stored in Amazon S3. Which TWO services would be appropriate for this use case? (Choose TWO.)

Select 2 answers
A.AWS Glue
B.AWS DataSync
C.Amazon Athena
D.Amazon Kinesis Data Firehose
E.Amazon Kinesis Data Streams
AnswersD, E

Firehose can deliver streaming data to S3 with buffering.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to ingest real-time streaming data, transform it on the fly (e.g., convert to Parquet/ORC), and deliver it directly to Amazon S3 with low latency. It handles buffering, compression, and partitioning automatically, making it ideal for the described sensor data pipeline.

Exam trap

The trap here is that candidates may choose only one streaming service, but the question requires two services, and the correct pairing is Kinesis Data Streams for real-time ingestion and Kinesis Data Firehose for delivery to S3, as Firehose alone cannot provide sub-second latency.

286
MCQmedium

A data engineer needs to design a data ingestion pipeline that ingests CSV files from an Amazon S3 bucket, transforms the data by adding a timestamp column, and loads it into an Amazon Redshift table. The pipeline should run automatically whenever a new file is uploaded to the S3 bucket. Which AWS service should be used to trigger the transformation?

A.AWS Lambda
B.AWS Step Functions
C.Amazon EventBridge
D.Amazon Simple Queue Service (SQS)
AnswerA

Lambda integrates natively with S3 event notifications, so an object-created event invokes the function immediately. The function runs the transformation code, adding the timestamp column, then loads results into Redshift, meeting the automatic per-upload trigger requirement.

Why this answer

Amazon S3 can be configured to send events directly to AWS Lambda when a new CSV file is uploaded. Lambda then executes the transformation (adding a timestamp column) and loads the data into Redshift. Option B (Step Functions) is not triggered directly by S3 events without an intermediate service like Lambda.

Option C (EventBridge) can route S3 events to Lambda, but a direct S3 event notification to Lambda is simpler and more common; however, the key point is that Lambda is the direct trigger. Option D (SQS) requires a separate process to poll the queue and invoke Lambda; it is not a direct trigger.

287
MCQhard

A data engineer is using AWS Glue to process a large dataset stored in Amazon S3. The dataset is partitioned by year/month/day and consists of Parquet files. The engineer notices that the Glue job is running slowly and consuming excessive DPU hours. The job performs a join between two large tables and writes the output back to S3. Which optimization technique should the engineer implement to improve performance and reduce cost?

A.Increase the number of DPUs allocated to the job to speed up execution.
B.Use predicate pushdown to filter data before the join and partition the output by a common column.
C.Convert the Parquet files to CSV to reduce storage size and improve read performance.
D.Enable AWS Glue job bookmarks to avoid reprocessing old data.
AnswerB

Predicate pushdown filters data at the source, reducing the amount of data read and shuffled during the join. Partitioning the output by a common column improves downstream query performance and reduces write overhead. These techniques directly address the slow join and high DPU usage by minimizing data movement and processing. This is a standard optimization for Glue ETL jobs dealing with large partitioned datasets.

Why this answer

Predicate pushdown reduces the amount of data read and shuffled by applying filters early in the execution plan. Partitioning the output by a common column improves data layout for subsequent queries and reduces write overhead. Together, these techniques optimize join performance and lower DPU consumption, directly addressing the slow job and high cost.

Other options either do not target the join inefficiency or would worsen performance.

Exam trap

The trap here is thinking that simply adding more DPUs will solve performance issues without addressing the underlying data movement and join inefficiencies.

288
MCQhard

The exhibit shows an IAM policy attached to a role used by an AWS Glue ETL job. The job reads from an S3 bucket and writes to another S3 bucket. However, the job fails with an access denied error when trying to write to the output bucket. What is the most likely cause?

A.The policy is missing permissions for AWS KMS to decrypt/encrypt objects
B.The policy does not allow glue:StartJobRun on the specific job
C.The policy only allows PutObject on the my-data-lake bucket, but the job writes to a different bucket
D.The policy does not include s3:ListBucket permission
AnswerC

IAM evaluates the role's policy against the actual resource ARN in the request. The statement permits s3:PutObject only on the my-data-lake bucket, so writing to the separate output bucket matches no Allow statement and is implicitly denied, producing the access denied error.

Why this answer

The IAM policy attached to the Glue job's role grants s3:PutObject only on the my-data-lake bucket ARN, but the ETL job's output path points to a different bucket. S3 authorization is evaluated per-resource ARN, so a PutObject call to any bucket not listed in the Resource element is implicitly denied. The fix is to add the output bucket's ARN (or a wildcard) to the policy's Resource list.

Exam trap

DEA-C01 often tests the misconception that a role with any S3 write permission can write to any bucket, when in fact IAM Resource ARNs are bucket-specific and a missing ARN produces an AccessDenied even when the action itself is granted.

How to eliminate wrong answers

Option A is wrong because KMS permissions are only required when objects are encrypted with SSE-KMS; the symptom here is a write denial, not a decrypt/encrypt failure, and the question gives no evidence of KMS encryption. Option B is wrong because glue:StartJobRun controls whether the job can be launched at all — if it were missing, the job would never start, not fail mid-execution on an S3 write. Option D is wrong because s3:ListBucket is needed for listing/head operations, not for PutObject; a missing ListBucket would cause listing errors, not an access-denied on write.

289
MCQmedium

A data engineer needs to ingest data from an Amazon S3 bucket into an Amazon Redshift cluster. The data is in Parquet format and is partitioned by date. The engineer wants to load only the latest partition into Redshift and ensure the load is efficient. Which method should the engineer use?

A.Use the Redshift COPY command specifying the S3 path for the latest partition.
B.Use Amazon Redshift Spectrum to query the latest partition and insert into a Redshift table.
C.Use AWS Glue to read the entire S3 bucket and write to Redshift, filtering by date.
D.Use the Redshift COPY command with the PARTITION option.
AnswerA

The COPY command can load data directly from a specific S3 prefix. By specifying the path to the latest partition (e.g., s3://bucket/data/date=2023-10-01/), you load only that partition efficiently. This is the most direct and performant method for Parquet data.

Why this answer

The Redshift COPY command is the most efficient way to load Parquet data from S3. By specifying the S3 path to the latest partition, you load only the required data. Other methods like using Glue or Spectrum involve extra processing or are less efficient for this specific task.

Exam trap

The trap here is thinking there is a PARTITION parameter in the COPY command, or that you must process the entire bucket to filter later.

290
MCQhard

A company runs an AWS Glue ETL job that reads data from Amazon S3, transforms it, and writes back to S3 in a different partition structure. The job uses the 'spark.sql.shuffle.partitions' option set to 200. After the job completes, the output has many small files. The data engineer wants to minimize the number of output files while maintaining job performance. Which action should the engineer take?

A.Use 'coalesce(n)' with n based on target file size (e.g., 128 MB) before writing.
B.Enable S3 multipart upload for the Glue job.
C.Increase the 'spark.sql.shuffle.partitions' to 500.
D.Reduce the 'spark.sql.shuffle.partitions' to 50.
AnswerA

Using `coalesce(n)` merges partitions without a full shuffle, so the 200 shuffle partitions collapse into roughly 128 MB output files, satisfying the requirement to minimise small files while preserving job performance. Unlike `repartition`, it avoids the network overhead of redistributing data across executors, keeping the write efficient.

Why this answer

'coalesce(n)' reduces the number of partitions without triggering a full shuffle, allowing you to control the number of output files based on a target file size (e.g., 128 MB). This minimizes small files while preserving job performance, as coalesce is a narrow transformation that avoids the overhead of a shuffle. In contrast, 'repartition(n)' would cause a full shuffle, degrading performance.

Exam trap

The trap here is that candidates often confuse 'coalesce' with 'repartition' or assume that adjusting 'spark.sql.shuffle.partitions' directly controls output file count, when in fact it only controls the number of partitions during shuffle operations, not the final write partition count.

How to eliminate wrong answers

Option B is wrong because enabling S3 multipart upload does not reduce the number of output files; it only improves upload reliability and throughput for large objects, but the job still writes the same number of small files. Option C is wrong because increasing 'spark.sql.shuffle.partitions' to 500 would increase the number of shuffle partitions, leading to even more small output files and potentially worse performance due to higher task overhead. Option D is wrong because reducing 'spark.sql.shuffle.partitions' to 50 would reduce the number of shuffle partitions, but it does not directly control the number of output files written; it may cause data skew and memory pressure, and the output file count still depends on the final partition count, which may remain high if the job uses repartition or other transformations.

291
Multi-Selecthard

A data engineer is troubleshooting an AWS Glue job that reads from Amazon RDS MySQL and writes to Amazon S3. The job runs successfully but takes longer than expected. The engineer wants to optimize performance. Which THREE actions would improve job performance?

Select 3 answers
A.Increase the number of DPUs allocated to the Glue job.
B.Use a single JDBC connection per partition.
C.Increase the JDBC fetch size parameter.
D.Convert the output format from Parquet to CSV.
E.Use a pushdown predicate to filter data at the source.
AnswersA, C, E

More DPUs provide more parallelism.

Why this answer

Increasing the number of DPUs (Data Processing Units) allocated to the AWS Glue job provides more parallel processing capacity, allowing the job to process data faster. This is a direct way to improve performance when the job is CPU or memory-bound, as Glue distributes the workload across the allocated DPUs.

Exam trap

The trap here is that candidates might think converting to CSV improves performance due to simplicity, but in reality, Parquet's columnar storage and compression provide significant performance benefits for analytics workloads on S3.

292
Multi-Selectmedium

Which TWO AWS services can be used to ingest streaming data from a mobile application into Amazon S3 for near-real-time analytics? (Choose 2.)

Select 2 answers
A.Amazon Kinesis Data Firehose
B.Amazon Kinesis Data Streams
C.Amazon DynamoDB Streams
D.AWS Glue
E.Amazon SQS
AnswersA, B

Firehose can ingest streaming data and deliver to S3 near real-time.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to load streaming data directly into Amazon S3, Redshift, Elasticsearch, or Splunk without requiring custom code. It can capture and transform streaming data from mobile applications in near-real-time and automatically deliver it to S3, making it ideal for near-real-time analytics pipelines.

Exam trap

The DEA-C01 exam often tests the distinction between managed ingestion (Firehose) and raw stream processing (Data Streams), and the trap here is that candidates may incorrectly choose DynamoDB Streams or SQS because they associate 'streaming' with any service containing 'stream' or 'queue', without understanding that DynamoDB Streams only captures internal table changes and SQS requires custom code to write to S3.

293
Multi-Selecthard

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams. The stream receives records from thousands of IoT devices, and the engineer must ensure that records from the same device are processed in order. The engineer also needs to scale the stream to handle peak loads without manual intervention. Which two actions should the engineer take? (Choose two.)

Select 2 answers
A.Enable enhanced fan-out for consumers to increase read throughput.
B.Use the device ID as the partition key when putting records into the stream.
C.Use a random partition key to evenly distribute records across shards.
D.Configure the stream to use on-demand capacity mode.
E.Increase the retention period of the stream to 365 days.
AnswersB, D

Using the device ID as the partition key ensures that all records from the same device are routed to the same shard. Within a shard, records are processed in the order they arrive. This guarantees per-device ordering, which is a key requirement. It also distributes the load across shards based on the number of devices, enabling horizontal scaling.

Why this answer

To ensure per-device ordering, records from the same device must go to the same shard, which is achieved by using the device ID as the partition key. To handle peak loads without manual intervention, the stream should use on-demand capacity mode, which automatically scales shards. Together, these actions meet both ordering and scaling requirements for the IoT streaming pipeline.

Exam trap

The trap here is assuming that enhanced fan-out or random partitioning improves ordering or scaling; enhanced fan-out only affects read throughput, and random partitioning breaks ordering.

294
MCQmedium

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon S3 in Parquet format. The source data is in JSON format and contains nested structures. The engineer needs to flatten the nested data and write it to Parquet. Which AWS Glue transform should the engineer use to flatten the nested structure?

A.Map
B.Relationalize
C.ApplyMapping
D.Filter
AnswerB

The Relationalize transform in AWS Glue flattens nested JSON structures into a relational schema, producing multiple tables that can be joined. It is specifically designed to handle nested data and is the correct choice for flattening. It preserves the relationships between the flattened tables through generated keys.

Why this answer

The Relationalize transform in AWS Glue is designed to flatten nested JSON structures into a relational format. It automatically creates multiple DynamicFrames for nested arrays and objects, generating keys to join them. This makes it the correct choice for the scenario, as it simplifies the flattening process without custom code.

Exam trap

The trap here is assuming that a generic transform like Map can flatten nested data with custom code, but Relationalize is the specialized transform for this purpose.

295
MCQmedium

A data engineer runs an AWS Glue ETL job that joins a 4 TB Parquet dataset in Amazon S3 with a small 40 MB reference lookup table stored as a single CSV file in S3. The join is taking hours and the job frequently fails with executor out-of-memory errors. The engineer wants to reduce shuffle and memory pressure with the least development effort. Which approach should the engineer take?

A.Increase the number of Glue DPU workers so more executors can hold the shuffled join partitions.
B.Convert the small lookup CSV into a broadcast join by loading it with a broadcast hint, avoiding a shuffle of the large dataset.
C.Repartition the large Parquet dataset by the join key using a Glue transform before the join.
D.Enable the AWS Glue job bookmark and rerun the job so only new files are processed.
AnswerB

Broadcasting the 40 MB lookup table ships it to every executor so the large 4 TB dataset never has to be shuffled for the join. This eliminates the expensive shuffle and the memory pressure caused by co-locating both sides, directly addressing the OOM failures. It requires only a small change to the join logic, satisfying the least-effort requirement.

Why this answer

Because one side of the join is only 40 MB, broadcasting that lookup table to every executor removes the need to shuffle the multi-terabyte dataset, cutting network I/O and eliminating the memory spikes that cause executor OOM failures. This is a minimal code change, satisfying the least-effort constraint, whereas scaling workers or repartitioning the large side still incurs a full shuffle.

Exam trap

The trap here is assuming that adding more DPU workers will fix a shuffle-heavy join, when the real fix is to change the join strategy so the large dataset is never shuffled.

296
MCQeasy

Refer to the exhibit. An S3 event notification is configured to trigger an AWS Lambda function when objects are created in 'my-bucket'. The Lambda function processes the JSON file and writes results to Amazon DynamoDB. The function fails with a timeout error. Which action should the engineer take to resolve the issue?

A.Modify the S3 event notification to use a different event type
B.Grant the Lambda function permission to access DynamoDB
C.Change the trigger to Amazon SQS instead of S3
D.Increase the Lambda function timeout
AnswerD

The Lambda function exceeds its configured execution limit while processing the JSON and writing to DynamoDB, so raising the timeout lets the invocation complete. This addresses the timeout error directly, unlike memory, concurrency or retry adjustments that leave the duration ceiling unchanged.

Why this answer

The Lambda function is failing with a timeout error, which indicates that the function is taking longer to execute than the default timeout of 3 seconds. Increasing the Lambda function timeout allows the function to run longer and complete its processing of the JSON file and DynamoDB write operation without being prematurely terminated.

Exam trap

The DEA-C01 exam often tests the distinction between timeout errors and permission errors, leading candidates to incorrectly choose a permissions fix (Option B) when the error message explicitly states 'timeout'.

How to eliminate wrong answers

Option A is wrong because changing the S3 event notification to a different event type (e.g., from s3:ObjectCreated:* to s3:ObjectCreated:Put) does not address the timeout issue; the function still fails due to execution duration, not the trigger event. Option B is wrong because a timeout error is not a permissions issue; if the Lambda function lacked DynamoDB permissions, it would fail with an access denied error (e.g., 403), not a timeout. Option C is wrong because changing the trigger to Amazon SQS instead of S3 does not resolve the timeout; the Lambda function would still have the same execution duration limit and would timeout regardless of the trigger source.

297
MCQmedium

A data engineer is designing a pipeline to ingest data from an Amazon Kinesis Data Stream into an Amazon S3 bucket. The data must be delivered in near real-time with minimal latency, and the engineer wants to use a fully managed service that automatically scales. The data is in JSON format and needs to be converted to Parquet before storage. Which AWS service should be used to achieve this?

A.Amazon Kinesis Data Firehose with record format conversion enabled to Parquet.
B.AWS Lambda function that reads from the Kinesis stream, converts JSON to Parquet, and writes to S3.
C.AWS Glue streaming ETL job that reads from Kinesis, converts to Parquet, and writes to S3.
D.Amazon Kinesis Data Analytics application that reads from Kinesis and writes Parquet to S3.
AnswerA

This option is correct because Amazon Kinesis Data Firehose is a fully managed service that can ingest data from a Kinesis Data Stream and deliver it to Amazon S3. It supports record format conversion from JSON to Parquet using an AWS Glue Data Catalog table schema. Firehose automatically scales to handle the throughput and provides near real-time delivery (with buffer intervals as low as 60 seconds). This meets all requirements without custom code.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service that can ingest from a Kinesis Data Stream, automatically scale, and deliver to Amazon S3 with near real-time latency. It supports record format conversion from JSON to Parquet using a Glue Data Catalog table, eliminating the need for custom code.

Exam trap

The trap here is overlooking Firehose's built-in format conversion capability and assuming that a custom Lambda or Glue job is required to convert JSON to Parquet.

298
MCQeasy

A data engineer needs to ingest JSON files from an S3 bucket into a DynamoDB table. The files are updated hourly and contain new records. Which AWS service should be used to trigger a Lambda function for each new object?

A.Kinesis Data Firehose
B.Amazon EventBridge
C.S3 Event Notifications
D.Amazon SQS
AnswerC

S3 Event Notifications publish an event to Lambda whenever a new object lands in the bucket, invoking the function per object. This satisfies the stem's requirement to trigger processing for each hourly-uploaded file without polling, unlike CloudWatch Events schedules or manual invocation.

Why this answer

S3 Event Notifications are the correct choice because they are designed to trigger AWS Lambda functions directly in response to object creation events (e.g., `s3:ObjectCreated:*`) in an S3 bucket. This allows the data engineer to automatically invoke a Lambda function for each new JSON file as it is uploaded, enabling ingestion into DynamoDB without polling or additional infrastructure.

Exam trap

The trap here is that candidates may confuse Amazon EventBridge with S3 Event Notifications, but EventBridge is not the native S3 event trigger—S3 Event Notifications are the direct, service-integrated mechanism for invoking Lambda on object creation.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Firehose is a streaming ingestion service that loads data into destinations like S3, Redshift, or Elasticsearch, but it cannot directly trigger a Lambda function for each new object; it is not an event source for Lambda. Option B is wrong because Amazon EventBridge can schedule or react to events from various AWS services, but it is not the native, direct integration for S3 object creation events—S3 Event Notifications are the simpler and more appropriate mechanism. Option D is wrong because Amazon SQS is a message queue service that decouples components, but it does not natively trigger Lambda from S3 events without additional configuration (e.g., S3 Event Notification to SQS, then Lambda polling SQS), which adds unnecessary complexity for this use case.

299
MCQeasy

A data engineer is troubleshooting an AWS Glue ETL job that fails with the error: 'An error occurred while calling o137.pyWriteDynamicFrame. No such file or directory: s3://bucket/output/part-00000.parquet'. The job reads from a JDBC source and writes to S3. What is the most likely cause?

A.The schema of the source data has changed, causing a mismatch during write
B.The output S3 path does not exist and the Glue job does not have permission to create it
C.The Glue job ran out of memory during the transformation phase
D.The IAM role attached to the Glue job lacks permissions to read from the JDBC source
AnswerB

The error message indicates missing directory; Glue may not auto-create if permissions are insufficient.

Why this answer

The error 'No such file or directory: s3://bucket/output/part-00000.parquet' indicates that the Glue job is trying to write to an S3 path that does not exist. By default, AWS Glue does not automatically create the target S3 bucket or prefix; it requires the path to already exist or the IAM role to have s3:PutObject permissions that allow the S3 service to create the object. Since the error occurs at the write stage (pyWriteDynamicFrame) and not during read, the most likely cause is that the output S3 path does not exist and the Glue job lacks the necessary permissions to create it.

Exam trap

The trap here is that candidates often confuse a missing S3 path with a permissions issue, but the error message explicitly states 'No such file or directory', which points to the path not existing rather than a generic access denied, and the exam expects you to recognize that Glue does not auto-create the output S3 prefix.

How to eliminate wrong answers

Option A is wrong because a schema mismatch during write would typically cause a different error, such as a schema compatibility or type conversion error, not a 'No such file or directory' file system error. Option C is wrong because an out-of-memory error would manifest as a Java heap space or memory limit exception, not a missing file error on S3. Option D is wrong because the error occurs during the write phase (pyWriteDynamicFrame) and not during the JDBC read; if the IAM role lacked JDBC read permissions, the job would fail earlier with a connection or authentication error.

300
MCQhard

A data engineer is using AWS Glue Studio to create an ETL job that reads from an Amazon S3 bucket and writes to another S3 bucket. The source data is in JSON format and contains nested structures. The engineer needs to flatten the nested structures and write the output in Parquet format. The job must be efficient and scalable. Which transformation should the engineer use in Glue Studio to flatten the nested data?

A.Use the 'Relationalize' transform.
B.Use the 'ApplyMapping' transform.
C.Use the 'Filter' transform.
D.Use the 'Join' transform.
AnswerA

The Relationalize transform in AWS Glue flattens nested JSON structures into a relational schema. It unnestes arrays and structs, producing multiple tables that can be joined. This is specifically designed for flattening nested data. It is efficient and scalable because it leverages Spark. The transform outputs a DynamicFrame collection, which can then be written to Parquet. This is the correct choice for flattening nested JSON.

Why this answer

The Relationalize transform in AWS Glue is specifically designed to flatten nested JSON data. It unnests arrays and structs into a relational schema, producing multiple tables. This is the correct transform for the scenario.

ApplyMapping, Filter, and Join do not flatten nested structures. Relationalize is efficient and scalable for this purpose.

Exam trap

The trap here is confusing ApplyMapping with flattening; ApplyMapping only maps fields and does not unnest arrays or structs.

← PreviousPage 4 of 6 · 447 questions totalNext →

Ready to test yourself?

Try a timed practice session using only Data Ingestion Transformation questions.