Courseiva

CCNA Data Ingestion Transformation Questions

75 of 447 questions · Page 3/6 · Data Ingestion Transformation topic · Answers revealed

151
MCQmedium

A data engineer is using AWS Glue to run a nightly ETL job that reads from an Amazon DynamoDB table and writes to Amazon S3 in Parquet format. The DynamoDB table is large and has a high read capacity. The engineer wants to minimize the impact on the DynamoDB table's performance and reduce the ETL job's runtime. Which approach should the engineer take?

A.Use AWS Glue with DynamoDB connector and enable DynamoDB Accelerator (DAX) for the table.
B.Use AWS Glue with DynamoDB connector and increase the job's number of workers to read the table faster.
C.Use AWS Glue with DynamoDB connector and configure the job to use parallel scans with a segment ratio.
D.Export the DynamoDB table to Amazon S3 using DynamoDB's native export feature, then use AWS Glue to read from S3 and write to Parquet.
AnswerD

DynamoDB's native export to S3 does not consume read capacity units and does not affect table performance. The export creates data in DynamoDB JSON format in S3. AWS Glue can then read from S3, transform as needed, and write to Parquet. This approach minimizes impact on the table and can be faster for large tables since it uses a separate export process. It also reduces the ETL job's runtime because Glue reads from S3 instead of DynamoDB.

Why this answer

To minimize impact on DynamoDB performance and reduce ETL runtime, exporting the table to S3 is ideal because it does not consume read capacity. AWS Glue can then process the exported data from S3, which is faster and does not affect the source table. Other methods like parallel scans or increasing workers still consume capacity and can impact performance.

Exam trap

The trap here is assuming that using DAX or parallel scans will avoid impacting DynamoDB when they still consume read capacity or add complexity.

152
MCQmedium

A data engineer is building an AWS Glue ETL job that reads a large JDBC table from Amazon RDS for PostgreSQL. The job must read the table in parallel to reduce runtime, but the table has no numeric primary key or monotonically increasing column. Which AWS Glue connection property should the engineer configure to enable parallel reads?

A.partitionColumn with lowerBound and upperBound
B.hashfield
C.hashpartitions and hashfield
D.enableParallelRead with numPartitions
AnswerC

AWS Glue supports hashpartitions and hashfield connection options specifically for JDBC sources that lack a suitable numeric partitioning column. hashpartitions defines the number of parallel read partitions, and hashfield specifies a column (often a string or UUID) used to hash rows into those partitions. This enables parallel reads without requiring a numeric key, directly addressing the scenario's constraint.

Why this answer

For JDBC sources without a numeric or monotonic column, AWS Glue provides hashpartitions and hashfield to enable parallel reads. hashpartitions sets the number of partitions, and hashfield names a column used for hashing rows across those partitions. This avoids the need for a numeric partitionColumn and is the documented approach for this exact situation.

Exam trap

The trap here is assuming that partitionColumn with lowerBound and upperBound always works for parallel JDBC reads, when it actually requires a numeric, evenly distributed column.

153
MCQhard

A company uses AWS Glue to process JSON logs from S3. The logs have a nested structure and the schema evolves over time. The data engineer needs to ensure the Glue job can handle schema changes without failing. Which configuration should be used?

A.Manually update the table schema in the Glue Data Catalog before each run
B.Use Spark SQL with a static schema definition in the script
C.Set the job parameter '--enable-glue-datacatalog' and '--mergeDynamicColumns' to true
D.Enable AWS Glue Schema Registry and define a schema version
AnswerC

This allows Glue DynamicFrame to merge schema variations automatically.

Why this answer

Setting '--enable-glue-datacatalog' allows the Glue job to use the Data Catalog as the metastore, and '--mergeDynamicColumns' (or the equivalent '--enable-schema-evolution' in newer Glue versions) instructs the job to dynamically merge new columns from the evolving JSON schema into the existing table schema during runtime, preventing job failures due to schema mismatches. This is specifically designed for nested, schema-evolving data like JSON logs, as it automatically reconciles differences between the source data and the catalog definition.

Exam trap

The trap here is that candidates often confuse the AWS Glue Schema Registry (which enforces schema compatibility and versioning) with the schema evolution capabilities of the Glue DynamicFrame, leading them to choose Option D even though it would reject schema changes rather than adapt to them.

How to eliminate wrong answers

Option A is wrong because manually updating the table schema before each run is not scalable, error-prone, and defeats the purpose of automated schema evolution; it also introduces operational overhead and potential downtime. Option B is wrong because using Spark SQL with a static schema definition in the script will cause the job to fail when new fields appear in the JSON logs, as Spark's static schema cannot adapt to dynamic changes without manual code modifications. Option D is wrong because the AWS Glue Schema Registry is designed for schema validation and serialization/deserialization (e.g., Avro, Protobuf) to enforce compatibility rules, not for dynamically merging evolving schemas during ETL processing; it would reject records that don't conform to the registered schema version, causing job failures instead of handling changes gracefully.

154
MCQmedium

A company runs a SQL Server transactional database on Amazon RDS. They need to capture change data (inserts, updates, deletes) in near real-time and replicate them to an Amazon S3 data lake. Which AWS service is most suitable?

A.AWS Database Migration Service (DMS) with change data capture
B.AWS Glue DataBrew
C.Amazon Kinesis Data Streams with Kinesis Client Library
D.Amazon Redshift Spectrum
AnswerA

DMS change data capture reads the database transaction log continuously, streaming inserts, updates and deletes to S3 with minimal source impact. This satisfies the near real-time replication requirement, unlike batch exports or snapshot-only tools that cannot capture ongoing changes.

Why this answer

AWS DMS with change data capture (CDC) is the most suitable service because it can continuously capture and replicate incremental changes (inserts, updates, deletes) from a SQL Server transactional database on Amazon RDS to an S3 data lake in near real-time. DMS uses native SQL Server transaction logs (e.g., MS-CDC or log-based replication) to read changes without impacting source performance, and it supports target S3 in formats like Parquet or CSV. This directly meets the requirement for near-real-time CDC replication to a data lake.

Exam trap

The trap here is that candidates may confuse Kinesis Data Streams as a general-purpose streaming solution for any real-time data, but it lacks native CDC capabilities for relational databases without additional custom code or connectors, making DMS the correct choice for database-to-S3 replication.

How to eliminate wrong answers

Option B is wrong because AWS Glue DataBrew is a visual data preparation tool for cleaning and normalizing data, not a service for capturing and replicating change data from a live database. Option C is wrong because Amazon Kinesis Data Streams is a real-time streaming service that requires custom producers and consumers (e.g., KCL) to ingest and process data, but it cannot natively capture CDC from a SQL Server database without additional middleware like Debezium or a custom application. Option D is wrong because Amazon Redshift Spectrum is a query engine that allows running SQL queries directly against data in S3, not a service for ingesting or replicating change data from a source database.

155
Multi-Selectmedium

Which TWO practices improve the performance of AWS Glue ETL jobs? (Choose two.)

Select 2 answers
A.Use pushdown predicates to filter data at the source
B.Increase the number of DPUs to the maximum allowed
C.Use the smallest possible file size for input data
D.Enable AWS Glue job metrics and debug logging
E.Use column pruning to select only required columns
AnswersA, E

Filters data early, reducing data scanned.

Why this answer

Pushdown predicates (Option A) improve AWS Glue ETL performance by filtering data at the source before it is read into the job. This reduces the volume of data transferred and processed, which is especially effective when using formats like Parquet or ORC that support predicate pushdown natively. By applying filters early, Glue avoids scanning unnecessary partitions or rows, leading to faster execution and lower costs.

Exam trap

The trap here is that candidates often confuse monitoring features (like enabling metrics and logging) with performance optimizations, or mistakenly believe that maximizing resources (DPUs) always improves speed, ignoring the overhead of small files and the benefits of early filtering and column selection.

156
MCQhard

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The application reads from a Kinesis data stream, performs a 1-minute tumbling window aggregation, and writes results to an S3 bucket. Recently, the application started experiencing checkpoint failures and increasing processing delay. Which action should the engineer take FIRST to diagnose the issue?

A.Increase the parallelism of the Flink application.
B.Monitor CPU and memory utilization of the Flink application using Amazon CloudWatch metrics.
C.Switch to the Kinesis Client Library (KCL) for checkpointing.
D.Increase the checkpoint interval to reduce checkpoint frequency.
AnswerB

Checkpoint failures and rising processing delay typically stem from resource exhaustion, such as CPU or memory pressure, or backpressure within the Flink application. Reviewing CloudWatch CPU and memory metrics first identifies whether the task managers are saturated, directly addressing the stem's diagnostic constraint before deeper tuning.

Why this answer

When a Flink application shows checkpoint failures and growing processing delay, the first diagnostic step is to check resource utilization via CloudWatch metrics (CPU, memory, heap, checkpoint duration). Checkpoint failures often stem from backpressure or resource exhaustion, so understanding whether the application is CPU- or memory-bound guides the next action. This is the least invasive, most informative first step.

Exam trap

DEA-C01 often tests the urge to jump to a fix (increase parallelism, change checkpoint interval) instead of first gathering diagnostic data, so candidates pick a remediation rather than a monitoring step.

How to eliminate wrong answers

Option A is wrong because increasing parallelism before diagnosing the root cause can worsen the problem (e.g., if the bottleneck is the source shard count or downstream sink). Option C is wrong because KCL is for Kinesis consumer applications, not Flink checkpointing — Flink manages its own checkpointing via the Flink runtime. Option D is wrong because increasing the checkpoint interval masks symptoms rather than diagnosing them, and can increase recovery time.

157
Multi-Selecthard

A company is ingesting real-time financial transactions into Amazon Kinesis Data Streams. The data is then consumed by a Kinesis Data Analytics for Apache Flink application that calculates running totals. The application is experiencing high latency and checkpoint failures. Which TWO steps should the engineer take to improve performance and reliability? (Select TWO.)

Select 2 answers
A.Enable enhanced fan-out for the Flink application.
B.Reduce the batch size of records processed per checkpoint.
C.Increase the number of shards in the Kinesis data stream.
D.Increase the number of KPUs (Kinesis Processing Units) for the Flink application.
E.Decrease the checkpoint interval to reduce state size.
AnswersC, D

Adding shards raises the stream's ingest and read throughput ceiling, relieving the backpressure that stalls the Flink consumer and triggers checkpoint timeouts. This directly addresses the latency and checkpoint failures caused by insufficient parallelism in the source.

Why this answer

Option C is correct because increasing the number of shards in the Kinesis data stream raises the stream's read throughput capacity (each shard supports up to 2 MB/sec read and 1 MB/sec write, or 1,000 records/sec), which relieves the ingestion bottleneck that causes high latency in the Flink application. Option D is correct because increasing KPUs adds parallel processing resources (each KPU provides 1 vCPU and 4 GB memory) to the Kinesis Data Analytics for Apache Flink application, allowing it to consume and process records faster and complete checkpoints more reliably. Option A is not correct because enhanced fan-out is a Kinesis Data Streams consumer feature that gives dedicated 2 MB/sec read throughput per consumer via HTTP/2, but it does not address the Flink application's internal processing or checkpointing bottlenecks.

Option B is not correct because reducing the batch size per checkpoint does not improve throughput and can actually increase checkpoint overhead and latency. Option E is not correct because decreasing the checkpoint interval makes checkpoints more frequent, increasing overhead and state I/O rather than reducing state size or improving reliability.

Exam trap

DEA-C01 often tests the misconception that reducing checkpoint interval or batch size improves reliability, when in reality checkpoint failures under load are solved by increasing parallelism (KPUs) and stream capacity (shards), not by making checkpoints more frequent or smaller.

158
MCQeasy

A data engineer needs to schedule a daily AWS Glue ETL job that transforms data in Amazon S3. The job must run at 2:00 AM UTC every day. What is the simplest way to achieve this?

A.Use AWS Step Functions with a Wait state and a Glue task to run the job daily.
B.Create an AWS Glue trigger of type SCHEDULED with a cron expression.
C.Use Amazon EventBridge to schedule a Lambda function that calls the Glue StartJobRun API.
D.Configure an AWS Glue workflow with a schedule and add the job as a starting trigger.
AnswerB

AWS Glue triggers can be scheduled with a cron expression. You can create a trigger that fires at 2:00 AM UTC daily and starts the Glue job. This is the native and simplest method within AWS Glue, requiring no external services. It integrates directly with the job and provides status monitoring.

Why this answer

The simplest way to schedule a Glue job is to use an AWS Glue scheduled trigger with a cron expression. It is a built-in feature that directly starts the job at the specified time. Other options involve additional services like Lambda, EventBridge, or Step Functions, which add complexity without benefit for a single scheduled job.

Exam trap

The trap here is overcomplicating the solution by using external orchestration services when AWS Glue provides a native scheduling mechanism.

159
MCQhard

A data engineer is building an AWS Glue ETL job that reads JSON files from Amazon S3, flattens nested arrays, and writes Parquet to another S3 bucket. The job runs daily and processes about 2 TB. The engineer notices that job runs are failing intermittently with OutOfMemory errors during the shuffle phase. The job uses 10 G.1X workers. Which change should the engineer make to resolve the memory failures while minimizing cost?

A.Reduce the number of workers from 10 to 5 to lower concurrency.
B.Convert the source JSON files to CSV before running the Glue job.
C.Enable AWS Glue job bookmarks to skip previously processed files.
D.Switch the worker type to G.2X to provide more memory and disk per worker.
AnswerD

G.2X workers provide twice the memory and disk of G.1X workers, which directly addresses OutOfMemory failures during shuffle-intensive operations like flattening nested arrays. Because the job is memory-bound rather than CPU-bound, increasing per-worker memory is the most targeted fix. Keeping the worker count the same with larger workers often costs less than adding many small workers for shuffle-heavy workloads.

Why this answer

OutOfMemory errors during the shuffle phase of a Glue job indicate insufficient per-worker memory. G.2X workers provide double the memory and disk of G.1X, which is the most direct and cost-effective fix for shuffle-heavy transformations such as flattening nested arrays. Reducing workers or changing the source format does not address the memory bottleneck and can worsen the problem.

Exam trap

The trap here is assuming that adding more small workers or changing the input format will fix shuffle memory errors, when the issue is per-executor memory that only a larger worker type resolves.

160
MCQhard

A data engineer is using AWS Step Functions to orchestrate a daily ingestion workflow. The workflow calls an AWS Glue job, then runs an AWS Lambda function to validate output, and finally starts an Amazon Redshift stored procedure. The engineer needs to ensure that if the Glue job fails, the workflow retries the Glue job up to three times with exponential backoff before failing the entire execution. Which Step Functions feature should the engineer configure?

A.Enable idempotency on the Glue job so repeated invocations succeed.
B.Configure a Retry policy on the Glue job task with MaxAttempts and BackoffRate.
C.Increase the Step Functions execution timeout to allow the Glue job to rerun.
D.Configure a Catch block on the Glue job task to transition to a failure state.
AnswerB

Step Functions Retry policies are applied per state and support MaxAttempts, IntervalSeconds, MaxDelaySeconds, and BackoffRate. Setting MaxAttempts to three with a BackoffRate greater than one implements exponential backoff for the Glue job task specifically, which is exactly the required behavior for retrying only that step before the execution fails.

Why this answer

Step Functions Retry policies are defined on individual states and accept MaxAttempts, IntervalSeconds, MaxDelaySeconds, and BackoffRate. Applying a Retry to the Glue job task with MaxAttempts of three and a BackoffRate above one produces exponential backoff retries scoped to that task. If all attempts fail, the state then fails and can be handled by a Catch block.

Exam trap

The trap here is conflating Catch, which redirects execution on error, with Retry, which re-invokes the failed state and is the only feature that produces the required retry-with-backoff behavior.

161
MCQmedium

A data engineer is building a streaming ingestion pipeline using Amazon Kinesis Data Streams. The producer application writes records with an explicit partition key derived from the device ID, and there are approximately 2,000 active devices. The engineer needs to ensure that records for the same device are processed in order by a downstream consumer. Which configuration should the engineer verify to guarantee per-device ordering?

A.Confirm that the partition key is stable per device so records map to the same shard.
B.Increase the stream's retention period to 365 days to preserve ordering.
C.Ensure all records use the same partition key so they land in a single shard.
D.Enable enhanced fan-out on the stream so each consumer gets a dedicated read throughput.
AnswerA

Kinesis Data Streams preserves order within a shard, and records with the same partition key are consistently routed to the same shard via the MD5 hash of the key. By keeping the partition key stable per device, all records from a given device land in the same shard and are delivered in order, satisfying the requirement while retaining parallelism across shards.

Why this answer

Kinesis Data Streams guarantees ordering only within an individual shard. Because the partition key is hashed to select a shard, any record sharing the same partition key is consistently routed to the same shard. Keeping the device identifier as the partition key ensures all records for a device are serialized in one shard, delivering the required per-device ordering without sacrificing horizontal scale across the fleet.

Exam trap

The trap here is assuming that any stream-level feature such as enhanced fan-out or extended retention can enforce ordering, when ordering in Kinesis Data Streams is strictly a per-shard property driven by the partition key.

162
MCQhard

A data engineer is using AWS Glue to process a large dataset in Amazon S3. The dataset consists of many small JSON files (average 100 KB each) stored in a single prefix. The Glue job reads these files, performs transformations, and writes the output to Parquet in another S3 location. The job is running slowly and consuming many DPUs. Which action should the data engineer take to improve performance?

A.Use AWS Glue's `groupFiles` option to group multiple small files into larger chunks for processing.
B.Convert the JSON files to Parquet using an AWS Glue crawler before processing.
C.Enable AWS Glue job bookmarks to skip already processed files.
D.Increase the number of DPUs allocated to the job to enable more parallel tasks.
AnswerA

AWS Glue provides the `groupFiles` option (e.g., 'inPartition') to group small files into larger groups, reducing the number of tasks and improving read throughput. This is specifically designed to handle many small files by coalescing them, which reduces overhead and improves job performance without excessive DPUs.

Why this answer

The `groupFiles` option in AWS Glue allows the job to group multiple small files into a single partition, reducing the number of tasks and the overhead of reading many small files. This directly addresses the performance bottleneck caused by many small JSON files, improving job speed and reducing DPU consumption. Other options do not solve the small file issue.

Exam trap

The trap here is assuming that adding more DPUs or using bookmarks will solve performance issues with many small files, when the real solution is to group files to reduce overhead.

163
MCQmedium

A company wants to ingest streaming data from thousands of IoT devices into Amazon S3 with minimal latency and then transform the data using Spark SQL. Which AWS service should be used for data ingestion?

A.Amazon EMR
B.AWS Glue
C.Amazon Athena
D.Amazon Kinesis Data Firehose
AnswerD

Kinesis Data Firehose ingests streaming data and delivers it directly into Amazon S3 with minimal latency, requiring no custom consumer code. It satisfies the ingestion requirement, after which Spark SQL can transform the landed data.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed for ingesting streaming data into Amazon S3 with near-real-time latency (typically 60 seconds or less). It can directly write data to S3 without requiring custom code or additional infrastructure, and it supports optional transformations via AWS Lambda, making it ideal for the described use case of streaming IoT data ingestion.

Exam trap

The trap here is confusing data ingestion services (Kinesis Data Firehose) with data processing or query services (EMR, Glue, Athena), leading candidates to pick EMR for its Spark SQL capability instead of recognizing that Firehose handles the ingestion step before transformation.

How to eliminate wrong answers

Option A is wrong because Amazon EMR is a big data processing service for running frameworks like Spark and Hadoop, not a streaming ingestion service; it would require additional setup (e.g., Kinesis or Kafka) to ingest data into S3. Option B is wrong because AWS Glue is a serverless ETL service primarily for batch data transformation and cataloging, not designed for real-time streaming ingestion into S3. Option C is wrong because Amazon Athena is an interactive query service for analyzing data in S3 using SQL, not an ingestion tool; it cannot ingest streaming data.

164
MCQhard

A data engineer is using AWS Glue DataBrew to clean a dataset stored in Amazon S3. The dataset contains a column with inconsistent date formats and another with trailing whitespace. The engineer wants to apply these transformations reproducibly and schedule the recipe to run daily. Which combination of steps should the engineer take?

A.Use the DataBrew console to apply transformations, then export the results manually each day without creating a recipe or job.
B.Create a recipe, then attach it to a Glue ETL job as a transform step and schedule the Glue job using a trigger.
C.Create a Glue crawler to catalog the dataset, then use a Glue ETL job with custom PySpark code to apply the transformations on a schedule.
D.Create a project, add the dataset, build a recipe with the appropriate transformation steps, publish the recipe, and create a job that runs the recipe on a schedule.
AnswerD

DataBrew projects are used to explore and build recipes interactively. Publishing the recipe captures the transformation steps, and a DataBrew job applies the recipe to the dataset on a defined schedule. This workflow is the intended way to perform reproducible, scheduled cleaning tasks in DataBrew.

Why this answer

DataBrew workflows use projects to build recipes, which are published and then executed by jobs on a schedule. This captures the date-format and whitespace transformations reproducibly and runs them daily, matching the requirement exactly.

Exam trap

The trap here is assuming a DataBrew recipe can be attached to a Glue ETL job, when recipes are executed only through DataBrew jobs.

165
MCQhard

A data streaming application uses Kinesis Data Streams with 10 shards. The data producer is throttled frequently. Which action should be taken to resolve this issue?

A.Decrease the data retention period
B.Use enhanced fan-out for consumers
C.Enable server-side encryption
D.Increase the number of shards
AnswerD

Each Kinesis shard supports a fixed ingest ceiling of 1 MB/s or 1,000 records/s, so ten shards cap throughput and cause throttling once exceeded. Adding shards raises that aggregate write capacity, directly resolving the producer throttling described in the stem.

Why this answer

Throttling in Kinesis Data Streams occurs when the write throughput exceeds the shard limits. Each shard supports up to 1 MB/s or 1,000 records/s for writes. With 10 shards, the total write capacity is 10 MB/s or 10,000 records/s.

Increasing the number of shards (Option D) directly increases the write capacity, resolving the throttling issue by distributing the load across more shards.

Exam trap

The trap here is that candidates confuse consumer-side features (like enhanced fan-out or retention period) with producer-side capacity issues, leading them to pick options that do not address the root cause of write throttling.

How to eliminate wrong answers

Option A is wrong because decreasing the data retention period (default 24 hours, up to 365 days) does not affect write throughput or throttling; it only controls how long records are stored. Option B is wrong because enhanced fan-out is a consumer-side feature that provides dedicated read throughput (2 MB/s per consumer per shard) and does not address producer-side write throttling. Option C is wrong because enabling server-side encryption (SSE-S3 or SSE-KMS) secures data at rest but has no impact on write throughput or throttling.

166
MCQmedium

A data engineer needs to load data from an Amazon DynamoDB table into an Amazon S3 bucket for analytics. The table is approximately 500 GB and has a high volume of write traffic. The engineer must minimize the impact on the table's read capacity and avoid consuming provisioned throughput. What is the MOST appropriate method to export the data?

A.Use the DynamoDB export to Amazon S3 feature to export the table to an S3 bucket.
B.Use AWS Database Migration Service (AWS DMS) with a DynamoDB source and an S3 target.
C.Use AWS Glue to read the DynamoDB table with the dynamodb connector and write to Amazon S3.
D.Enable DynamoDB Streams and use an AWS Lambda function to write changed records to Amazon S3.
AnswerA

DynamoDB's native export to Amazon S3 uses a consistent point-in-time snapshot and does not consume any read capacity units on the table. It is designed for large tables and high-traffic workloads, making it the ideal choice to avoid impacting provisioned throughput while exporting 500 GB.

Why this answer

The DynamoDB export to Amazon S3 feature creates a point-in-time export without consuming any read capacity, which is critical for a high-traffic 500 GB table. Other methods such as Glue or DMS perform scans that consume RCUs, and Streams only provides incremental changes. Therefore, the native export is the only option that fully satisfies the requirement.

Exam trap

The trap here is assuming that any export method must scan the table and consume read capacity, overlooking the zero-RCU native export feature.

167
MCQhard

A company uses Amazon Kinesis Data Firehose to deliver data to an Amazon S3 bucket. The data is in JSON format and contains a 'timestamp' field with a Unix epoch value. The company wants to partition the S3 objects by year, month, day, and hour based on the timestamp. What is the MOST efficient method to achieve this?

A.Use the dynamic partitioning feature of Kinesis Data Firehose with inline parsing to extract the timestamp and create the S3 prefix.
B.Configure a custom S3 prefix in Firehose using the 'YYYY/MM/dd/HH' format based on the current time.
C.Use an AWS Glue ETL job to read from Firehose, partition, and write to S3.
D.Use Amazon Athena to run a CTAS query that partitions the data by timestamp.
AnswerA

Dynamic partitioning with inline parsing extracts the timestamp field directly within Firehose and derives year/month/day/hour prefixes automatically, avoiding custom Lambda transformation or downstream reprocessing. This satisfies the requirement for the most efficient, serverless partitioning method without managing extra compute.

Why this answer

Kinesis Data Firehose dynamic partitioning with inline parsing extracts the 'timestamp' field from each JSON record and uses it to build the S3 prefix (year/month/day/hour) automatically as records are delivered. This is the native, serverless, most efficient approach because Firehose handles the partitioning logic per-record without any additional compute services. It also supports JQ expressions for extracting and formatting the timestamp into the desired prefix pattern.

Exam trap

The trap here is confusing Firehose's static custom prefix (which uses delivery time) with dynamic partitioning (which uses record content), causing candidates to pick option B as a simpler solution.

How to eliminate wrong answers

Option B is wrong because a custom S3 prefix in Firehose uses the delivery time (current time), not the timestamp inside each record, so records would be grouped by arrival time rather than event time. Option C is wrong because an AWS Glue ETL job adds unnecessary cost, latency, and operational overhead — Firehose can partition natively without a separate ETL pipeline. Option D is wrong because Athena CTAS is a query-time operation that reads existing data and writes new partitioned data; it does not partition data as it is streamed into S3 and would require re-running on new data.

168
MCQeasy

A company wants to ingest real-time data from a social media API into Amazon S3 for analysis. The API provides data as JSON records. Which AWS service is best suited for this ingestion?

A.AWS Glue
B.Amazon Kinesis Data Firehose
C.Amazon Simple Queue Service (SQS)
D.Amazon DataZone
AnswerB

Amazon Kinesis Data Firehose is a fully managed delivery service that ingests streaming JSON records and loads them directly into Amazon S3, handling buffering, batching and scaling. It satisfies the real-time ingestion-to-S3 requirement without custom consumers, unlike Kinesis Data Streams, which requires separate delivery logic.

Why this answer

Amazon Kinesis Data Firehose is the best choice because it is a fully managed service designed to ingest real-time streaming data, such as JSON records from a social media API, and automatically load it into Amazon S3 with optional data transformation and compression. It handles scaling, buffering, and delivery without requiring custom code or infrastructure management, making it ideal for this use case.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Streams (which requires custom consumers) with Kinesis Data Firehose (which is serverless and directly writes to S3), or they incorrectly assume SQS can directly deliver to S3 without additional processing.

How to eliminate wrong answers

Option A is wrong because AWS Glue is a serverless data integration service for batch ETL (extract, transform, load) jobs and cataloging, not designed for real-time streaming ingestion from an API. Option C is wrong because Amazon Simple Queue Service (SQS) is a message queue for decoupling application components, but it does not natively write data to S3; you would need additional compute to poll and deliver messages, adding complexity. Option D is wrong because Amazon DataZone is a data governance and catalog service for managing data assets across an organization, not a data ingestion service for real-time streaming.

169
MCQeasy

A data engineer needs to ingest data from an Amazon S3 bucket into an Amazon Redshift cluster. The data is in CSV format and is updated daily. The engineer wants to use a fully managed service that can handle the load without managing infrastructure. Which AWS service should the engineer use?

A.AWS Glue
B.AWS Database Migration Service (AWS DMS)
C.Amazon Redshift Spectrum
D.Amazon Kinesis Data Firehose
AnswerA

AWS Glue is a fully managed extract, transform, and load (ETL) service that can read from S3 and write to Redshift. It handles provisioning and scaling of the underlying infrastructure. The engineer can create a Glue job to load the CSV data into Redshift on a schedule. This meets the requirement for a managed service without infrastructure management.

Why this answer

AWS Glue is a fully managed ETL service that can read CSV files from Amazon S3, apply transformations if needed, and load the data into Amazon Redshift. It handles the underlying infrastructure, scales automatically, and can be scheduled to run daily. This makes it the ideal choice for the scenario.

Exam trap

The trap here is confusing Redshift Spectrum with an ingestion service; Spectrum queries data in S3 but does not load it into Redshift, so it does not meet the ingestion requirement.

170
MCQmedium

A data engineer is using AWS Glue DataBrew to clean a dataset stored in Amazon S3. The dataset contains inconsistent date formats in a column named 'transaction_date'. The engineer needs to standardize all dates to ISO 8601 format (YYYY-MM-DD) and then write the cleaned data to a new S3 location. Which transformation should the engineer apply?

A.Use the 'Change data type' transformation to convert the column to a date type.
B.Use the 'Replace value' transformation to replace all date separators with hyphens.
C.Use the 'Format date' transformation with a custom date format pattern.
D.Use the 'Split column' transformation to separate day, month, and year into individual columns.
AnswerC

The 'Format date' transformation in AWS Glue DataBrew allows specifying input and output date formats. By setting the output format to 'yyyy-MM-dd', the engineer can standardize all dates to ISO 8601. This transformation handles multiple input formats if configured correctly, making it the appropriate choice for this scenario.

Why this answer

The 'Format date' transformation in AWS Glue DataBrew is designed to parse and reformat date values. By specifying the input format(s) and setting the output format to 'yyyy-MM-dd', the engineer can standardize inconsistent date strings into ISO 8601. This ensures consistency for downstream processing and analytics, and it handles multiple input patterns without manual string manipulation.

Exam trap

The trap here is confusing data type conversion with date formatting; changing the type does not reformat the string, and simple string replacements cannot handle varying date orders.

171
MCQeasy

A data engineer is using AWS Glue to run a job that reads from an Amazon S3 bucket and writes to another S3 bucket. The job must be scheduled to run every day at 2:00 AM UTC. Which AWS Glue feature should the engineer use to set up this schedule?

A.AWS Glue Triggers with a schedule type of 'Scheduled' and a cron expression.
B.AWS Glue job bookmarks with a time-based configuration.
C.AWS Glue DataBrew schedules with a cron expression.
D.AWS Glue Workflows with an on-demand trigger started by an Amazon EventBridge rule.
AnswerA

Glue Triggers support scheduled triggers defined with cron expressions in UTC. The engineer can create a trigger with a cron expression like 'cron(0 2 * * ? *)' to run the job daily at 2:00 AM UTC. This is the native Glue scheduling mechanism and integrates directly with the job, requiring no external orchestration.

Why this answer

AWS Glue Triggers of type 'Scheduled' accept cron expressions and run jobs on a defined schedule in UTC. Creating a scheduled trigger with the appropriate cron expression is the direct, native way to run a Glue job daily at 2:00 AM UTC without external orchestration.

Exam trap

The trap here is confusing Glue job bookmarks, which track processed data, with Glue triggers, which actually schedule job runs.

172
MCQeasy

A company wants to import data from an external FTP server into Amazon S3 on a daily basis. The data volumes are moderate. Which AWS service is MOST suitable for this task?

A.Amazon S3 Transfer Acceleration
B.AWS Transfer Family
C.AWS DataSync
D.AWS Glue with a JDBC connection
AnswerB

AWS Transfer Family provides a managed FTP endpoint that ingests files directly into Amazon S3, satisfying the daily external FTP import requirement without custom scripts or servers. It supports scheduled or continuous transfers and handles the moderate volume comfortably.

Why this answer

AWS Transfer Family is the most suitable service because it provides fully managed support for SFTP, FTPS, and FTP protocols, enabling direct, secure file transfers from an external FTP server to Amazon S3 without needing to manage any infrastructure. It integrates natively with S3, so files are automatically stored in the specified bucket upon transfer completion, making it ideal for daily imports of moderate data volumes.

Exam trap

The trap here is that candidates often confuse AWS DataSync with a general-purpose file transfer tool, but DataSync requires an agent on the source and does not natively support FTP protocols, whereas AWS Transfer Family is purpose-built for FTP-based transfers to S3.

How to eliminate wrong answers

Option A is wrong because Amazon S3 Transfer Acceleration is a feature that speeds up uploads to S3 over the internet by using AWS edge locations, but it does not support FTP protocols or act as a server-side endpoint to receive files from an external FTP server. Option C is wrong because AWS DataSync is designed for moving large volumes of data between on-premises storage and AWS services, but it requires installing an agent on the source environment and does not natively support FTP as a source protocol. Option D is wrong because AWS Glue with a JDBC connection is intended for extracting data from databases using JDBC drivers, not for handling file transfers over FTP/SFTP/FTPS protocols.

173
MCQmedium

A data engineer is building a real-time data pipeline to ingest sensor data from IoT devices. The data is sent to AWS IoT Core, which publishes messages to a Kinesis Data Stream. Each message is about 1 KB in size. The data must be transformed (add a device location field) and then stored in Amazon S3 for long-term analytics. The engineer has set up a Lambda function to transform the records and write to S3. However, the engineer notices that the Lambda function is invoked thousands of times per second, causing high costs and occasional throttling. The Lambda function processes only one record at a time. The engineer wants to reduce the number of Lambda invocations and improve throughput. What should the engineer do?

A.Reduce the number of shards in the Kinesis stream to limit concurrency.
B.Increase the Lambda function's memory allocation to improve performance.
C.Replace the Lambda function with Amazon Kinesis Data Firehose and use its built-in transformation.
D.Configure the event source mapping to use a larger batch size and set a batch window.
AnswerD

Increasing the batch size lets each Lambda invocation process many records instead of one, and the batch window accumulates records before invoking. Both settings cut invocation counts and throttling while raising throughput, satisfying the cost and performance goals.

Why this answer

Configuring the event source mapping with a larger batch size and a batch window allows Lambda to process multiple records per invocation, reducing the number of invocations and costs. This improves throughput and reduces throttling. Option A is incorrect because reducing shards reduces the stream capacity, causing backpressure and potential data loss.

Option B is incorrect because increasing memory does not reduce the number of invocations; it only speeds up processing per invocation, but still processes one record at a time. Option C is incorrect because Kinesis Data Firehose can batch records, but it still uses per-record Lambda transformation if you use a Lambda function, or it can use built-in transformations but not the flexible logic described. The most direct solution is to batch records in the existing Lambda function via event source mapping parameters.

Exam trap

A candidate might think that reducing the number of shards will reduce invocations, but that actually reduces the stream's ability to handle the data volume and can cause throttling or data loss.

174
Multi-Selecthard

A data engineering team is building a data lake on Amazon S3. They need to ingest data from multiple sources: (1) streaming IoT data, (2) daily CSV exports from an on-premises system via SFTP, and (3) change data capture (CDC) from an Amazon Aurora database. Which THREE services should the team use to ingest these data sources?

Select 3 answers
A.Amazon Kinesis Data Streams for IoT data ingestion.
B.AWS Database Migration Service (DMS) for CDC from Aurora.
C.AWS Transfer Family for SFTP-based file ingestion.
D.AWS Glue ETL for CDC from Aurora.
E.Amazon EMR for daily CSV ingestion.
AnswersA, B, C

Kinesis Data Streams ingests continuous, high-volume IoT telemetry with durable ordering and replay, matching the streaming source. It feeds downstream consumers that land the data in Amazon S3, satisfying the real-time ingestion requirement of the data lake.

Why this answer

Amazon Kinesis Data Streams (A) is the right choice for streaming IoT data because it is a fully managed, scalable service designed for real-time ingestion of high-volume streaming records into S3 via Kinesis Data Firehose or consumers. AWS Database Migration Service (B) supports ongoing change data capture (CDC) from Amazon Aurora (and other relational databases), replicating inserts, updates, and deletes continuously, which matches the CDC requirement. AWS Transfer Family (C) provides a fully managed SFTP endpoint backed by Amazon S3, so the on-premises system can push daily CSV exports over SFTP directly into the data lake.

AWS Glue ETL (D) is a batch/transform service, not a CDC ingestion mechanism, so it does not satisfy the Aurora CDC requirement. Amazon EMR (E) is a big data processing cluster, not a managed ingestion service for daily SFTP CSV files, so it is not the appropriate ingestion tool here.

Exam trap

The trap here is confusing AWS Glue ETL (a batch ETL tool) with AWS DMS (a database migration and CDC service), and assuming Amazon EMR is an ingestion service rather than a processing framework for large-scale data transformations.

175
MCQmedium

Refer to the exhibit. A data engineer has attached this IAM policy to an AWS Glue job role. The Glue job fails when trying to write transformed data to an S3 bucket located in a different AWS account. What is the most likely reason?

A.The policy does not allow lambda:InvokeAsync
B.The Glue job role does not have permissions to write to S3
C.The policy does not grant s3:ListBucket, and the bucket policy may not allow cross-account access
D.The policy does not include kinesis:DescribeStream
AnswerC

Cross-account S3 access requires both bucket policy and IAM permissions, including ListBucket.

Why this answer

The IAM policy shown does not include the s3:ListBucket permission, which is required for the Glue job to list objects in the S3 bucket before writing. Additionally, cross-account access requires both the source account's IAM policy (this one) to grant write permissions and the target account's S3 bucket policy to explicitly allow the source account's role, which may not be configured. Without s3:ListBucket, the Glue job cannot verify the bucket's existence or structure, causing the write operation to fail.

Exam trap

The trap here is that candidates assume s3:PutObject alone is sufficient for writing to S3, but AWS requires s3:ListBucket for bucket-level operations like listing and validation, especially in cross-account scenarios where the bucket's existence must be confirmed.

How to eliminate wrong answers

Option A is wrong because lambda:InvokeAsync is a permission for invoking AWS Lambda functions asynchronously, which is irrelevant to writing data to S3 from a Glue job. Option B is wrong because the policy does include s3:PutObject and s3:PutObjectAcl, which are write permissions; the failure is due to missing s3:ListBucket, not a lack of write permissions entirely. Option D is wrong because kinesis:DescribeStream is a permission for Amazon Kinesis streams, which is unrelated to S3 write operations in this cross-account scenario.

176
MCQeasy

A data engineer needs to run a PySpark transformation on a 2 TB dataset stored in Amazon S3 and write the output back to S3 in Parquet. The team wants to use AWS Glue but does not want to manage clusters or tune Spark configuration manually. They also want to pay only for the time the job runs. Which AWS Glue component should the engineer use?

A.AWS Glue DataBrew recipe job
B.Amazon EMR on EC2 cluster with a bootstrap action
C.AWS Glue Python shell job
D.AWS Glue for Apache Spark job
AnswerD

AWS Glue for Apache Spark is a serverless Spark environment. You supply a PySpark script, and Glue provisions and manages the Spark cluster, applies tuning defaults, and bills per DPU-hour for the job's runtime. This matches the requirement for custom PySpark on a 2 TB S3 dataset without cluster management and with pay-per-run pricing.

Why this answer

AWS Glue for Apache Spark provides a serverless, distributed Spark runtime. The engineer submits a PySpark script, Glue handles cluster provisioning and Spark tuning, and billing is based on DPU-hours consumed during the run. This fits the need for custom PySpark on a 2 TB S3 dataset with no cluster management and pay-per-run cost, unlike single-node shell jobs or self-managed EMR clusters.

Exam trap

The trap here is equating 'AWS Glue' generically with any Glue job type, when Python shell jobs cannot run distributed PySpark and DataBrew targets visual preparation rather than custom code.

177
Multi-Selectmedium

A data engineer is designing a data ingestion pipeline using Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format and must be converted to Apache Parquet before storage. The engineer wants to minimize costs and operational effort. Which two actions should the engineer take to meet these requirements? (Choose two.)

Select 2 answers
A.Enable compression on the Firehose delivery stream to reduce storage costs.
B.Enable record format conversion in the Firehose delivery stream and select Apache Parquet as the output format.
C.Use AWS Glue to create a table in the AWS Glue Data Catalog that defines the schema of the JSON data for the Firehose stream to reference during conversion.
D.Configure the Firehose delivery stream to invoke an AWS Lambda function to convert each record from JSON to Parquet.
E.Set the Firehose buffer size to the maximum value to reduce the number of S3 PUT operations.
AnswersB, C

Firehose supports record format conversion from JSON to Parquet using an AWS Glue table that defines the schema. This built-in feature eliminates the need for custom processing, reducing operational effort and cost. It is the recommended approach for format conversion in Firehose.

Why this answer

To convert JSON to Parquet in Firehose with minimal effort, enable record format conversion and create an AWS Glue table defining the schema. These two actions allow Firehose to perform the conversion automatically. Other options either add overhead, do not address the conversion, or are supplementary but not required.

Exam trap

The trap here is thinking that a Lambda function is needed for format conversion, but Firehose has native support using AWS Glue Data Catalog.

178
Multi-Selectmedium

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The engineer needs to ensure that the job handles data quality issues such as duplicate records and missing values before loading. The job must also minimize the amount of data shuffled across the network. Which two actions should the engineer take? (Choose two.)

Select 2 answers
A.Use the FillMissingValues transform to replace missing values with a default.
B.Repartition the DynamicFrame to a single partition before writing to Redshift.
C.Use the DropDuplicates transform to remove duplicate records.
D.Use the ResolveChoice transform to handle data type conflicts.
E.Enable job bookmarks to track processed data and avoid reprocessing.
AnswersA, C

FillMissingValues is an AWS Glue transform that fills null or missing values in specified columns with a default value. This handles missing values as required. It is a built-in transform that can be easily added to the job, ensuring data quality before loading to Redshift, and it does not require custom code.

Why this answer

DropDuplicates removes duplicate records, and FillMissingValues replaces missing values with defaults, directly addressing the data quality requirements. Both are built-in AWS Glue transforms that can be applied without custom code. The other options do not handle duplicates or missing values, and repartitioning to a single partition would increase network shuffle, contrary to the requirement.

Exam trap

The trap here is confusing data quality transforms with performance tuning; repartitioning and job bookmarks are often mistakenly associated with data cleaning but do not resolve duplicates or missing values.

179
MCQhard

A data engineer runs an AWS Glue job that reads from an Amazon Kinesis Data Stream and writes to Amazon S3. The job must process records in order per shard and must checkpoint progress so it can resume after a failure without reprocessing all records. Which Glue configuration BEST supports this?

A.Use a Glue streaming ETL job with a custom checkpoint table in Amazon DynamoDB and set `--starting_position` to `TRIM_HORIZON`.
B.Use a Glue streaming ETL job with `--starting_position` set to `LATEST` and disable job bookmarks to always read new records.
C.Use the Glue streaming ETL job type with `--starting_position` set to `TRIM_HORIZON` and rely on Glue's built-in checkpointing in the job bookmarks.
D.Use a standard Glue batch job with the Kinesis connector and set `--starting_position` to `LATEST` for each run.
AnswerC

Glue streaming ETL jobs support Kinesis Data Streams as a source and use job bookmarks to track processed records. Setting `--starting_position` to `TRIM_HORIZON` defines where to start on the first run, and bookmarks checkpoint progress so subsequent runs resume from the last processed record. This provides ordered, per-shard processing and fault-tolerant resumption without custom checkpoint code.

Why this answer

Glue streaming ETL jobs natively support Kinesis Data Streams and use job bookmarks to track progress per shard. Setting `--starting_position` to `TRIM_HORIZON` establishes the initial read point, and bookmarks checkpoint processed records so a failed job resumes without reprocessing everything. Batch jobs, disabled bookmarks, and custom checkpoint tables either lose data, reprocess, or add unnecessary overhead.

Exam trap

The trap here is confusing `--starting_position` on each run with true checkpointing, when resumption without reprocessing depends on job bookmarks being enabled.

180
MCQmedium

A data engineer is configuring an AWS Glue ETL job that reads semi-structured JSON from Amazon S3 and must flatten nested arrays before loading into Amazon Redshift. During testing, the job fails with an AnalysisException stating that a column named 'events' cannot be resolved, even though the AWS Glue Data Catalog table shows the column. The engineer confirms the catalog table was created by a crawler and the S3 data is present. Which action will most directly resolve the schema resolution failure?

A.Call resolveChoice with the 'MatchCatalogSchema' option on the DynamicFrame before referencing the nested column.
B.Increase the number of DPUs allocated to the Glue job so the executor can read the full nested JSON payload.
C.Enable the 'Group files' option in the crawler and re-crawl the S3 prefix to consolidate the JSON into a single object.
D.Convert the source JSON to Parquet with an AWS Glue crawler and then point the job at the Parquet table.
AnswerA

When a crawler infers a schema that differs from what Spark infers at read time, nested fields can be dropped or renamed, producing unresolved-column errors. resolveChoice with MatchCatalogSchema forces the DynamicFrame to conform to the Data Catalog table definition, restoring the 'events' column so downstream transforms and the Redshift write can proceed.

Why this answer

The failure occurs because Spark's inferred schema at read time diverges from the Data Catalog definition created by the crawler, so nested fields such as 'events' are not resolvable in the DynamicFrame. Applying resolveChoice with MatchCatalogSchema reconciles the runtime schema with the catalog, restoring the column. Capacity changes and file grouping do not alter schema resolution, and crawlers do not perform format conversion.

Exam trap

The trap here is assuming any schema error means the catalog is wrong, when the real cause is a mismatch between Spark-inferred and catalog-inferred schemas that resolveChoice must reconcile.

181
Multi-Selectmedium

A company is designing a data ingestion pipeline for clickstream data from a website. The data must be ingested in near real-time. Which TWO services can be used together to build this pipeline?

Select 2 answers
A.Amazon Kinesis Data Streams
B.Amazon Simple Queue Service (SQS)
C.Amazon S3
D.Amazon Kinesis Data Firehose
E.Amazon DynamoDB
AnswersA, D

Amazon Kinesis Data Streams ingests clickstream events in near real time with low latency and durable retention. Paired with a consumer such as AWS Lambda or Kinesis Data Analytics, it forms the ingestion layer of a real-time pipeline, meeting the near-real-time requirement.

Why this answer

Amazon Kinesis Data Streams (A) is correct because it is designed for real-time streaming ingestion of high-volume data such as clickstream events, allowing producers to continuously write records that can be processed within milliseconds by consumers. Amazon Kinesis Data Firehose (D) is correct because it can consume streaming data and reliably load it into destinations like S3, Redshift, or Elasticsearch in near real-time, making it a natural complement to Kinesis Data Streams for building an end-to-end ingestion pipeline. Together, Kinesis Data Streams captures and buffers the clickstream events while Firehose delivers them to downstream storage or analytics services with minimal latency.

Amazon SQS (B) is a message queue for decoupling applications but is not purpose-built for real-time streaming analytics pipelines. Amazon S3 (C) is a storage service, not an ingestion mechanism, and Amazon DynamoDB (E) is a NoSQL database, so neither serves as the streaming ingestion component required here.

182
MCQmedium

Refer to the exhibit. An AWS Glue ETL job is failing with an OutOfMemoryError. The job reads from Amazon S3 and performs a GROUP BY on a large dataset. Which change should the data engineer make to resolve this error?

A.Use coalesce to reduce the number of partitions.
B.Increase the number of DPUs allocated to the Glue job.
C.Increase the number of partitions in the DataFrame.
D.Use repartition to increase the number of partitions.
AnswerB

OutOfMemoryError during a GROUP BY on a large dataset occurs because each executor has insufficient memory for the shuffle and aggregation. Increasing DPUs adds more executors and memory, distributing the GROUP BY workload and preventing the driver or executor from exhausting heap during the shuffle phase.

Why this answer

The OutOfMemoryError in an AWS Glue ETL job performing a GROUP BY on a large dataset indicates that the executors do not have enough memory to handle the shuffle operations required for aggregation. Increasing the number of DPUs (Data Processing Units) allocated to the Glue job increases the total memory and compute resources available, allowing the job to process larger partitions without running out of memory.

Exam trap

The trap here is that candidates often confuse partition tuning (coalesce/repartition) with resource allocation, mistakenly thinking that adjusting partitions alone can fix memory errors without increasing the underlying compute and memory capacity.

How to eliminate wrong answers

Option A is wrong because using coalesce to reduce the number of partitions would decrease parallelism and concentrate data into fewer partitions, potentially worsening memory pressure and making the OutOfMemoryError more likely. Option C is wrong because increasing the number of partitions in the DataFrame without adding more resources (DPUs) would spread data across more tasks but still rely on the same total memory, which does not resolve the underlying memory shortage. Option D is wrong because repartitioning to increase the number of partitions similarly does not add memory; it only redistributes data, which can even increase shuffle overhead and exacerbate memory issues.

183
MCQmedium

A data engineer needs to load data from an Amazon DynamoDB table into an Amazon S3 data lake nightly. The table is large and the engineer wants to avoid consuming provisioned read capacity on the live table. Which approach should the engineer use?

A.Create an Amazon ElastiCache for Redis cluster and replicate the DynamoDB table into it, then export from Redis to Amazon S3.
B.Use the DynamoDB export to Amazon S3 feature to export a point-in-time snapshot to S3 without consuming read capacity.
C.Enable DynamoDB Streams on the table and use an AWS Lambda function to write each change to Amazon S3 in near real time.
D.Use AWS Glue with a DynamoDB connection and read the table directly with the DynamoDB reader, accepting the consumed read capacity.
AnswerB

DynamoDB export to S3 creates a full point-in-time snapshot in S3 without consuming provisioned read capacity on the table, because it reads from the underlying storage. It is designed for analytics and backup use cases, and the resulting data can be queried or transformed directly in S3.

Why this answer

DynamoDB export to S3 produces a point-in-time snapshot directly in Amazon S3 without consuming read capacity on the source table, making it ideal for nightly analytics loads that must not impact production traffic. The exported data is written in a format suitable for downstream processing by services such as Athena or Glue.

Exam trap

The trap here is assuming that any read of DynamoDB consumes read capacity, when the export-to-S3 feature reads from storage and bypasses provisioned throughput.

184
MCQhard

A data engineer is using AWS Glue to read from an Amazon S3 bucket that contains data in Apache Parquet format, partitioned by year/month/day. The Glue job needs to read only the data for the last 7 days. The engineer wants to minimize the amount of data scanned and improve job performance. Which approach should be used to filter the partitions efficiently?

A.Create a separate Glue Data Catalog table for each day's data and read from the last 7 tables.
B.Read the entire dataset into a DynamicFrame and then apply a filter transformation on the partition columns.
C.Use a pushdown predicate in the Glue job's 'create_dynamic_frame.from_catalog' call to filter on partition columns.
D.Use the 'glueContext.read_from_options' with a 'filter' parameter to specify the partition range.
AnswerC

This option is correct because AWS Glue supports pushdown predicates when reading from the AWS Glue Data Catalog. By specifying a filter expression on partition columns (e.g., 'year >= 2023 AND month = 10 AND day BETWEEN 1 AND 7'), Glue pushes the filter down to the data source, so only the relevant partitions are read from S3. This significantly reduces the amount of data scanned and improves performance.

Why this answer

Using a pushdown predicate in the 'create_dynamic_frame.from_catalog' call allows AWS Glue to filter partitions at the source, so only the relevant S3 partitions for the last 7 days are read. This minimizes data scanned and improves job performance by avoiding loading unnecessary data.

Exam trap

The trap here is assuming that filtering after reading the data is equivalent to filtering at the source, when in fact pushdown predicates are required to avoid scanning all partitions.

185
MCQhard

A company runs an e-commerce platform that generates clickstream data from user interactions on their website. The data is sent as JSON objects via HTTP POST to an API Gateway endpoint, which triggers a Lambda function that writes each record to a Kinesis Data Stream (100 shards). A second Lambda function consumes the stream, transforms the data (enriches with geolocation from a DynamoDB table), and writes to a Kinesis Data Firehose delivery stream that delivers Parquet files to an S3 data lake every 5 minutes. The system has been working for months, but recently the Firehose delivery stream started showing 'DeliveryFailed' errors for a subset of records. The errors point to 'InvalidData' from the Lambda transformation. The engineer reviews the Lambda transformation code and notices that the geolocation lookup occasionally fails because the DynamoDB table has a throttling issue. The engineer needs to handle these failures gracefully so that records that fail enrichment are still delivered to S3 with a null geolocation field, without blocking other records. Which course of action should the engineer take?

A.Configure the Kinesis Data Firehose delivery stream to send failed records to a dead-letter queue (DLQ) for later reprocessing.
B.Modify the Lambda function to send failed records to a separate Kinesis Data Stream for manual processing.
C.Modify the Lambda function to catch exceptions during the geolocation lookup, set the geolocation field to null, and continue processing the record.
D.Increase the read capacity units (RCUs) on the DynamoDB table to eliminate throttling.
AnswerC

Wrapping the DynamoDB geolocation lookup in try/catch lets the function substitute a null geolocation and emit the record rather than throwing, which is what currently produces the InvalidData delivery failures. This satisfies the requirement that unenriched records still reach S3 without blocking other records in the batch.

Why this answer

The requirement is to keep records flowing to S3 with a null geolocation when enrichment fails, without blocking other records. Wrapping the DynamoDB lookup in a try/catch inside the Lambda transformation, setting geolocation to null on failure, and returning the record as 'Ok' ensures Firehose treats it as successfully transformed and delivers it. This is the standard graceful-degradation pattern for Firehose Lambda transformations.

Exam trap

DEA-C01 often tests whether candidates choose 'route failures elsewhere' (DLQ, separate stream) versus 'handle failures inline and continue' — the trap is picking DLQ because it sounds robust, when the requirement is to still deliver the record with a null field.

How to eliminate wrong answers

Option A is wrong because Firehose's DLQ/S3 error bucket captures records that fail transformation or delivery, but the requirement is to still deliver those records to S3 with a null geolocation — sending them to a DLQ means they are not delivered to the data lake as intended and require separate reprocessing. Option B is wrong because routing failed records to a separate Kinesis Data Stream adds operational complexity and still does not deliver them to S3 with a null field; it also does not prevent the Firehose DeliveryFailed errors. Option D is wrong because increasing DynamoDB RCUs addresses the root cause of throttling but does not handle the failure gracefully — throttling can still occur (e.g., during bursts or hot partitions), and the requirement is explicitly about handling failures so records are not lost or blocked.

186
MCQmedium

Refer to the exhibit. A data engineer is using a Kinesis Data Stream with one shard. The application writes 2000 records per second, each 1 KB. The put record calls are frequently throttled. What is the most likely cause?

A.The stream has only one shard, which limits writes to 1000 records per second
B.The retention period of 24 hours is too short
C.The stream uses KMS encryption, causing additional latency
D.Enhanced monitoring is not enabled, causing performance issues
AnswerA

A single shard caps writes at 1,000 records per second and 1 MB per second. The application attempts 2,000 records per second (about 2 MB per second), exceeding both limits, so Kinesis throttles the put calls. Adding shards raises the ingest capacity to match the required throughput.

Why this answer

A Kinesis Data Stream shard has a write throughput limit of 1,000 records per second (or 1 MB per second). Since the application is writing 2,000 records per second (each 1 KB) to a single shard, it exceeds the shard's record-per-second quota, causing the PutRecord calls to be throttled. The solution is to increase the number of shards to at least two to distribute the load.

Exam trap

The DEA-C01 exam often tests the misconception that throttling is caused by encryption latency or monitoring settings, but the real trap is forgetting that each shard has a hard limit of 1,000 records per second, regardless of other configurations.

How to eliminate wrong answers

Option B is wrong because the retention period (default 24 hours, max 365 days) controls how long records are stored, not the write throughput; throttling is unrelated to retention. Option C is wrong because KMS encryption adds latency to encrypt/decrypt operations but does not reduce the shard-level write limit of 1,000 records per second; throttling is a capacity issue, not a latency issue. Option D is wrong because enhanced monitoring provides detailed metrics (e.g., user, request, stream-level) but does not affect the shard's write throughput limits; throttling occurs regardless of monitoring settings.

187
MCQmedium

A company uses AWS Lambda to process messages from an Amazon SQS queue. The messages contain JSON payloads that need to be transformed and written to an Amazon DynamoDB table. Recently, the Lambda function has been timing out and messages are being sent to the dead-letter queue (DLQ). What is the BEST way to troubleshoot and resolve this issue?

A.Use a standard SQS queue instead of a DLQ to reprocess failed messages automatically.
B.Increase the Lambda function timeout and monitor DynamoDB write capacity to ensure it is not throttling.
C.Switch the SQS queue to a FIFO queue to ensure exactly-once processing.
D.Increase the visibility timeout of the SQS queue to 30 minutes.
AnswerB

Timeouts and DLQ routing usually stem from the function exceeding its configured duration or downstream throttling. Raising the timeout addresses slow execution, while monitoring DynamoDB write capacity reveals throttling that silently extends processing time, letting you right-size capacity or batch writes.

Why this answer

The Lambda function is timing out, which suggests the function's execution duration is exceeding its configured timeout. Increasing the timeout gives the function more time to process messages. Additionally, if DynamoDB write capacity is insufficient, throttling can cause retries that further delay processing, so monitoring and possibly increasing write capacity units addresses the root cause of timeouts.

Exam trap

The trap here is that candidates often focus on SQS queue configuration (visibility timeout, queue type) rather than addressing the actual performance bottleneck in the Lambda function or downstream DynamoDB service.

How to eliminate wrong answers

Option A is wrong because using a standard SQS queue instead of a DLQ does not automatically reprocess failed messages; it only changes the queue type and does not address the underlying timeout or throttling issue. Option C is wrong because switching to a FIFO queue enforces exactly-once processing and message ordering, but it does not resolve timeouts or DynamoDB throttling; it could even introduce additional latency. Option D is wrong because increasing the visibility timeout to 30 minutes only delays when a message becomes visible again after a failure, but it does not fix the root cause of timeouts or throttling; it may mask the problem.

188
Multi-Selecteasy

A company uses AWS Glue to run ETL jobs daily. The data engineer wants to reduce costs by optimizing the job configuration. Which two actions will help reduce costs? (Choose TWO.)

Select 2 answers
A.Use G.1X worker type instead of G.2X
B.Increase the job timeout to 48 hours
C.Enable Spark UI logging for debugging
D.Reduce the number of DPUs allocated to the job if the data volume is small
E.Increase the number of job retries to handle transient failures
AnswersA, D

G.1X is half the cost of G.2X.

Why this answer

G.1X workers provide 16 GB of memory and 4 vCPUs, while G.2X workers provide 32 GB and 8 vCPUs. For many ETL jobs, especially those with smaller data volumes or less complex transformations, G.1X workers are sufficient, and using them instead of G.2X directly reduces the cost per DPU-hour since AWS Glue pricing is based on DPU capacity. This optimization lowers costs without sacrificing performance if the job is not memory- or CPU-bound.

Exam trap

The trap here is that candidates often confuse cost optimization with reliability improvements, such as retries or timeouts, and may overlook that reducing worker size or DPU count directly lowers resource consumption and cost.

189
MCQmedium

A data engineer is using AWS Database Migration Service (AWS DMS) to replicate ongoing changes from an on-premises Oracle database to an Amazon S3 bucket in near real time. The source table has a primary key and the database is configured for ARCHIVELOG mode. The engineer needs change data capture (CDC) to capture INSERT, UPDATE, and DELETE operations. Which AWS DMS task setting should be used to capture ongoing changes?

A.Set the migration type to 'full-load' and configure ongoing replication using Oracle GoldenGate.
B.Set the migration type to 'cdc' only and rely on the Oracle redo logs without supplemental logging.
C.Set the migration type to 'full-load-and-cdc' and enable supplemental logging on the source table.
D.Set the migration type to 'full-load-and-cdc' and increase the commit rate to reduce latency.
AnswerC

For Oracle CDC, AWS DMS requires supplemental logging to be enabled at the database or table level to capture column data for updates and deletes. The migration type 'full-load-and-cdc' performs an initial full load and then continues capturing ongoing changes, satisfying the requirement for near real-time replication of INSERT, UPDATE, and DELETE operations.

Why this answer

To capture ongoing changes from Oracle to S3 with AWS DMS, the task must be configured with migration type 'full-load-and-cdc' and supplemental logging must be enabled on the source. Supplemental logging provides the additional column data needed for UPDATE and DELETE operations. Without it, CDC cannot function correctly.

The other options either use incorrect migration types or omit the critical supplemental logging requirement.

Exam trap

The trap here is assuming that setting the migration type to 'cdc' alone is sufficient for Oracle, while overlooking the mandatory supplemental logging requirement on the source database.

190
MCQhard

A data engineer is designing a data ingestion pipeline for IoT sensor data. The sensors send JSON messages every second. The data must be available in Amazon S3 within 5 minutes and must be transformed (JSON to Parquet) before storage. Which combination of services meets these requirements?

A.Amazon Kinesis Data Streams with AWS Glue streaming ETL
B.Amazon Kinesis Data Firehose with data transformation and Parquet conversion
C.Amazon Kinesis Data Analytics with output to S3
D.Amazon S3 with S3 Event Notifications to AWS Lambda for transformation
AnswerB

Firehose buffers records and invokes a Lambda function for JSON-to-Parquet transformation before delivering to S3, meeting the five-minute latency requirement without managing clusters. Its native record format conversion writes Parquet directly, unlike Kinesis Data Streams, which only stores raw bytes.

Why this answer

Amazon Kinesis Data Firehose natively supports AWS Lambda-based record transformation and can convert the transformed JSON records to Parquet using its built-in format conversion before delivering to S3. Firehose buffers data and delivers within its configurable buffer interval (as low as 60 seconds), comfortably meeting the 5-minute SLA. This is the only option that combines ingestion, transformation, and Parquet conversion in a single managed service.

Exam trap

DEA-C01 often tests whether candidates know that Firehose has native Parquet conversion and Lambda transformation, versus assuming you must bolt on Glue or Lambda separately — leading them to pick the more complex option A.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Streams alone does not transform or convert to Parquet — you would need to add Glue streaming ETL, which adds complexity and typically has higher latency than Firehose's built-in transformation, and Data Streams requires custom consumers. Option C is wrong because Kinesis Data Analytics (now Managed Service for Apache Flink) performs stream processing but does not natively write Parquet to S3 without additional custom code, and it is overkill for simple JSON-to-Parquet conversion. Option D is wrong because S3 Event Notifications only fire after objects land in S3, meaning raw JSON would be stored first and transformation would be post-hoc, violating the 'transformed before storage' requirement and adding latency.

191
MCQhard

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams and AWS Lambda. The Lambda function processes records and writes to Amazon DynamoDB. The engineer notices that the Lambda function is throttled during high traffic. Which action should the engineer take to reduce throttling?

A.Increase the Lambda function timeout
B.Disable retries on the Lambda function
C.Increase the number of shards in the Kinesis data stream
D.Use an Amazon SQS queue as an intermediate buffer
AnswerC

More shards allow more Lambda concurrent executions, reducing throttling.

Why this answer

Increasing the number of shards in the Kinesis data stream increases the overall throughput of the stream, which allows the Lambda event source mapping to poll more shards concurrently. Each shard is processed by one Lambda invocation at a time, so more shards mean more concurrent Lambda executions, reducing the per-invocation load and the likelihood of throttling.

Exam trap

The DEA-C01 exam often tests the misconception that throttling is caused by Lambda function performance (timeout or retries) rather than the stream's shard count, leading candidates to choose options that affect execution duration or error handling instead of scaling the source.

How to eliminate wrong answers

Option A is wrong because increasing the Lambda function timeout does not reduce throttling; it only allows the function to run longer, which does not affect the rate at which Lambda invokes the function. Option B is wrong because disabling retries on the Lambda function would cause records to be dropped or sent to a dead-letter queue, but it does not prevent throttling; throttling occurs when the concurrent execution limit is reached, not from retries. Option D is wrong because using an Amazon SQS queue as an intermediate buffer would decouple the stream from Lambda but does not directly address the root cause of throttling, which is insufficient shard count to handle the incoming data volume; SQS would add latency and complexity without increasing concurrency.

192
MCQeasy

A company needs to ingest streaming data from multiple sources and store it in Amazon S3. The data volume is up to 5 GB per hour. What is the MOST cost-effective ingestion service?

A.AWS Glue
B.Amazon Kinesis Data Streams
C.Amazon Kinesis Data Firehose
D.AWS Lambda
AnswerC

Kinesis Data Firehose buffers and delivers streaming records directly into Amazon S3 without managing consumers, and its pricing suits 5 GB per hour. This satisfies the stem's cost-effectiveness constraint while meeting the ingestion and S3 delivery requirement.

Why this answer

Amazon Kinesis Data Firehose is the most cost-effective service for ingesting streaming data into Amazon S3 at 5 GB/hour. It is fully managed, automatically scales, and charges only for data ingested (per GB), with no upfront provisioning. While Amazon Kinesis Data Streams (KDS) may have lower throughput cost for steady loads, it requires manual shard management and typically needs additional components (e.g., Lambda functions) to deliver data to S3, increasing operational overhead and total cost.

AWS Glue is a batch ETL service, not designed for streaming. AWS Lambda is a compute service and would require custom code and scaling logic, making it more expensive and complex for this use case. Therefore, Kinesis Data Firehose provides the simplest and most cost-effective solution for streaming data ingestion directly to S3.

193
MCQeasy

A data engineer must load a 250 GB uncompressed CSV dataset from Amazon S3 into Amazon Redshift. The data must be loaded daily, and the engineer wants to minimize load time and avoid saturating the cluster's leader node. Which approach meets these requirements?

A.Use the COPY command with a single large file and the COMPUPDATE OFF option.
B.Use Amazon Redshift Spectrum to query the CSV files directly from Amazon S3 without loading them.
C.Use AWS Database Migration Service (AWS DMS) with a full load task to copy the CSV data into Amazon Redshift.
D.Use the COPY command with a manifest file that references multiple files in Amazon S3.
AnswerD

The COPY command can load multiple files in parallel when a manifest is provided. Each slice on the compute nodes reads a portion of the data, distributing the workload and avoiding leader node saturation. This is the recommended way to load large datasets from Amazon S3 into Amazon Redshift efficiently.

Why this answer

The COPY command is the most efficient way to load large datasets from Amazon S3 into Amazon Redshift. By using a manifest file that lists multiple S3 objects, the load is parallelized across all slices, reducing load time and preventing the leader node from becoming a bottleneck. This is the standard best practice for bulk data ingestion into Redshift.

Exam trap

The trap here is assuming that a single large file or an ETL service like AWS DMS is required for bulk loading, when the COPY command with a manifest is designed specifically for parallel, high-performance loads from Amazon S3.

194
MCQeasy

A data pipeline uses AWS Glue to process data from an S3 data lake. The pipeline fails intermittently with a 'ThrottlingException' when writing to a DynamoDB table. What is the MOST likely cause?

A.The DynamoDB table's write capacity is insufficient for the workload.
B.The network connection between Glue and DynamoDB is unstable.
C.The Glue job's timeout setting is too low.
D.The Glue job does not have sufficient IAM permissions to write to DynamoDB.
AnswerA

DynamoDB returns ThrottlingException when write requests exceed the table's provisioned write capacity units. Intermittent failures during Glue writes indicate the table's write capacity is insufficient for the pipeline's throughput, not a code or schema fault.

Why this answer

A ThrottlingException from DynamoDB indicates that the request rate to the table has exceeded the provisioned write capacity. AWS Glue jobs can generate high-throughput writes, and if the DynamoDB table's write capacity units (WCUs) are not sufficient to handle the burst, DynamoDB will throttle the requests. This is the most direct cause of the intermittent failure described.

Exam trap

The trap here is that candidates may confuse ThrottlingException with permission errors (Option D) or network issues (Option B), but AWS specifically tests the understanding that DynamoDB throttling is a capacity management mechanism, not a connectivity or authorization problem.

How to eliminate wrong answers

Option B is wrong because network instability between Glue and DynamoDB would typically result in connection timeouts or retryable network errors, not a specific ThrottlingException which is an application-level error from DynamoDB's API. Option C is wrong because a Glue job's timeout setting controls how long the job can run before being terminated, not how it handles individual API throttling errors; a timeout would cause a different error (e.g., 'Timeout exceeded'). Option D is wrong because insufficient IAM permissions would result in an AccessDeniedException, not a ThrottlingException; the error message directly indicates capacity limits, not authorization failures.

195
MCQeasy

A company stores IoT sensor data in S3 as JSON files. They need to convert the data to Parquet format for efficient querying with Amazon Athena. Which AWS service can perform this transformation with minimal effort?

A.Kinesis Data Firehose
B.Amazon Athena
C.AWS Glue ETL job
D.AWS Lambda
AnswerC

AWS Glue ETL jobs read JSON from S3, apply schema and transformation logic, and write Parquet back to S3 with minimal coding via the visual editor or generated scripts. This satisfies the conversion requirement while remaining serverless and low-effort for Athena querying.

Why this answer

AWS Glue ETL jobs can easily convert JSON to Parquet with built-in transforms. Option A is wrong because Kinesis Data Firehose is for streaming data ingestion, not batch transformations. Option B is wrong because Amazon Athena is a query engine, not a transformation service.

Option D is wrong because AWS Lambda is for small, event-driven transformations and is not ideal for large-scale batch conversion.

196
MCQmedium

A company is using AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job fails intermittently with a 'MemoryError' when the stream has a sudden spike in data volume. Which configuration change would best prevent this error?

A.Increase the number of DPUs (Data Processing Units) for the Glue job.
B.Store intermediate results in Amazon RDS.
C.Use a batch transformation instead of streaming.
D.Increase the number of shards in the Kinesis data stream.
AnswerA

MemoryError arises when executors exhaust heap during volume spikes. Adding DPUs increases the number of workers and total memory available, spreading the load across more executors. This directly addresses the memory constraint rather than tuning batch size or windowing.

Why this answer

Increasing the number of DPUs for the AWS Glue job provides more compute and memory resources to the job, which directly addresses the MemoryError caused by sudden data volume spikes. Glue DPUs are the unit of processing capacity, and scaling them up allows the job to handle larger volumes without running out of memory. This is the most direct configuration change to prevent memory-related failures.

Exam trap

The trap is confusing stream-side scaling (adding Kinesis shards) with job-side scaling (adding Glue DPUs), leading candidates to pick D when the error is in the Glue job's memory.

How to eliminate wrong answers

Option B is wrong because storing intermediate results in Amazon RDS does not increase the Glue job's memory and adds latency and complexity; it does not address the root cause of the MemoryError. Option C is wrong because switching to batch transformation changes the processing model and may not be feasible for a streaming use case; it also does not directly solve memory pressure during spikes. Option D is wrong because increasing Kinesis shards increases stream throughput but does not give the Glue job more memory; it could actually worsen the spike by delivering more data faster to an under-provisioned job.

197
MCQeasy

A company uses AWS Glue ETL jobs to transform data in Amazon S3. The data arrives in JSON format but needs to be converted to Parquet for efficient querying. Which AWS Glue feature should be used to infer the schema and generate transformation code?

A.Amazon S3 Select
B.Amazon Athena
C.Amazon Kinesis Data Analytics
D.AWS Glue crawlers
AnswerD

Crawlers populate the Data Catalog with schema information used by Glue ETL jobs.

Why this answer

AWS Glue crawlers are the correct feature because they automatically connect to data stores (like S3), infer the schema of JSON data by sampling it, and populate the AWS Glue Data Catalog with table definitions. This catalog schema can then be used by AWS Glue ETL jobs to generate transformation code (e.g., converting JSON to Parquet) without manual schema definition.

Exam trap

The trap here is that candidates confuse AWS Glue crawlers with Amazon Athena or S3 Select, assuming any query or analysis tool can infer schemas for ETL, but only crawlers are designed to automatically discover and catalog schemas for Glue ETL jobs.

How to eliminate wrong answers

Option A is wrong because Amazon S3 Select is a query-in-place service that retrieves subsets of data from S3 objects using SQL, but it does not infer schemas or generate ETL transformation code. Option B is wrong because Amazon Athena is an interactive query service that uses SQL to analyze data directly in S3, but it does not generate ETL transformation code or automatically infer schemas for Glue ETL jobs (though it can query Glue Data Catalog tables). Option C is wrong because Amazon Kinesis Data Analytics processes streaming data in real time using SQL or Apache Flink, not batch transformation of JSON to Parquet in S3, and it does not infer schemas for Glue ETL jobs.

198
MCQeasy

A company wants to transform data in Amazon S3 using SQL queries without provisioning servers. The transformations are ad-hoc and run occasionally. Which service should be used?

A.AWS Glue
B.Amazon Redshift Spectrum
C.Amazon EMR
D.Amazon Athena
AnswerD

Athena is serverless and supports SQL queries directly on S3 data.

Why this answer

Amazon Athena is the correct choice because it enables serverless, ad-hoc SQL querying directly on data stored in Amazon S3 without requiring any infrastructure provisioning. Since the transformations are occasional and ad-hoc, Athena's pay-per-query model and zero setup overhead align perfectly with the requirement.

Exam trap

The trap here is that candidates often confuse AWS Glue's ability to run SQL via Spark SQL or Athena as a transformation engine, but Glue requires provisioning resources and is not designed for ad-hoc serverless SQL queries.

How to eliminate wrong answers

Option A is wrong because AWS Glue is primarily an ETL service that requires provisioning and managing crawlers, jobs, and triggers, and is designed for scheduled or event-driven batch transformations, not lightweight ad-hoc SQL queries. Option B is wrong because Amazon Redshift Spectrum extends Redshift's query capabilities to S3 but still requires a provisioned Redshift cluster to run, which violates the 'without provisioning servers' constraint. Option C is wrong because Amazon EMR is a managed big data platform that requires provisioning EC2 instances and configuring clusters, making it unsuitable for occasional, serverless SQL queries.

199
MCQeasy

An e-commerce company wants to capture clickstream data from its website and store it in Amazon S3 for analytics. The data arrives continuously and the company needs near-real-time processing. Which solution is most appropriate?

A.AWS Data Pipeline
B.AWS Snowball Edge
C.Amazon Kinesis Data Firehose
D.Amazon S3 Transfer Acceleration
AnswerC

Amazon Kinesis Data Firehose ingests streaming clickstream data continuously and delivers it into Amazon S3 with near-real-time buffering, satisfying the stem's continuous-arrival and near-real-time requirements without custom consumer code. It handles scaling and delivery automatically, unlike batch uploads or query-based approaches.

Why this answer

Amazon Kinesis Data Firehose is the most appropriate solution because it is a fully managed service designed to ingest streaming data and deliver it to destinations like Amazon S3 with near-real-time latency. The company needs continuous clickstream capture and near-real-time processing, which Firehose provides. Option A (AWS Data Pipeline) is for batch processing, not streaming.

Option B (AWS Snowball Edge) is for offline data transfer, not real-time. Option D (S3 Transfer Acceleration) improves upload speed but is not a streaming ingestion service.

200
MCQhard

Refer to the exhibit. A data engineer is configuring an IAM policy for a Lambda function that writes transformed data to S3. The function writes to both 'example-bucket/data/' and 'example-bucket/public/'. The policy is intended to enforce server-side encryption with SSE-S3 for all objects written to the 'public/' prefix, while allowing all operations on other prefixes. However, the Lambda function is failing with an AccessDenied error when writing to 'example-bucket/public/'. What is the most likely cause?

A.The policy denies DeleteObject on 'public/'.
B.The policy denies PutObject on 'public/' unconditionally.
C.The policy does not allow GetObject for 'public/'.
D.The Lambda function is not setting the 'x-amz-server-side-encryption' header to 'AES256' when writing to 'public/'.
AnswerD

The policy's `s3:PutObject` condition requires the `x-amz-server-side-encryption` header to equal `AES256` for the `public/` prefix. Because the Lambda function omits this header, the request fails the condition and S3 returns AccessDenied, satisfying the stem's encryption-enforcement constraint.

Why this answer

The policy enforces SSE-S3 encryption for objects written to the 'public/' prefix. When a Lambda function writes to S3 without setting the 'x-amz-server-side-encryption' header to 'AES256', the request fails with an AccessDenied error if the bucket policy requires SSE-S3. The policy explicitly denies PutObject unless the encryption header is present, so the function must include this header to succeed.

Exam trap

The DEA-C01 exam often tests the nuance that bucket policies can conditionally deny operations based on request headers, and candidates mistakenly think the error is due to missing IAM permissions rather than a missing encryption header.

How to eliminate wrong answers

Option A is wrong because the error is about writing (PutObject), not deleting (DeleteObject), and the policy focuses on encryption enforcement, not delete permissions. Option B is wrong because the policy does not unconditionally deny PutObject; it denies PutObject only when the required SSE-S3 encryption header is missing, which is a conditional denial. Option C is wrong because GetObject is not relevant to the write operation failing; the error occurs during PutObject, and the policy does not restrict read access for 'public/'.

201
Multi-Selecteasy

A data engineer is designing a data ingestion pipeline for real-time clickstream data. Which TWO services can be used to ingest the data into Amazon Kinesis Data Streams?

Select 2 answers
A.Amazon S3
B.Kinesis Producer Library (KPL)
C.Kinesis Data Firehose
D.AWS SDK
E.AWS Glue
AnswersB, D

The Kinesis Producer Library aggregates and batches records, then sends them to Kinesis Data Streams using the PutRecords API. It handles retries and throughput optimisation, satisfying the real-time clickstream ingestion requirement without custom serialisation or buffering code.

Why this answer

Options B and D are correct. The Kinesis Producer Library (KPL) is a library for producers to send data to Kinesis Data Streams. AWS SDK can also be used directly.

Option A is wrong because Amazon S3 is a storage service, not a producer. Option C is wrong because Kinesis Data Firehose is a downstream consumer or delivery service, not a producer. Option E is wrong because AWS Glue is an ETL service, not a producer.

202
MCQmedium

Refer to the exhibit. A data engineer is troubleshooting a Kinesis Data Streams consumer that is falling behind. The stream has 2 shards and is receiving data at a rate of 2 MB/s. The consumer is an AWS Lambda function with a batch size of 100 records. What should the engineer do to improve consumer throughput?

A.Decrease the Lambda batch size to 10 records
B.Increase the retention period of the stream to 168 hours
C.Increase the number of shards in the stream to 4
D.Increase the memory allocation of the Lambda function
AnswerC

More shards increase parallelism and throughput for both producers and consumers.

Why this answer

Increasing the number of shards to 4 doubles the stream's total ingestion capacity to 4 MB/s, which directly increases the number of concurrent Lambda invocations and thus consumer throughput. With 2 shards, each shard can support up to 1 MB/s input and 2 MB/s output, so the current 2 MB/s load is at the shard-level output limit, causing the consumer to fall behind.

Exam trap

The DEA-C01 exam often tests the misconception that Lambda memory or batch size adjustments are the primary levers for throughput, when in fact the shard count directly controls the parallelism and read capacity of a Kinesis stream consumer.

How to eliminate wrong answers

Option A is wrong because decreasing the batch size reduces the number of records processed per invocation, which increases the number of Lambda invocations and overhead, potentially worsening throughput rather than improving it. Option B is wrong because increasing the retention period (up to 365 days) only affects how long records are stored in the stream, not the rate at which the consumer can read or process data. Option D is wrong because while increasing Lambda memory can improve CPU performance, the bottleneck here is the shard-level read throughput limit (2 MB/s per shard for the consumer), not Lambda compute capacity.

203
MCQeasy

A data engineer needs to transform JSON data from an S3 bucket into Parquet format and load it into Amazon Redshift. The transformation must be performed incrementally as new data arrives. Which AWS service is BEST suited for this task?

A.Use AWS Lambda to transform the data on the fly and write to Redshift.
B.Use Amazon EMR with Apache Spark to transform the data and load it into Redshift.
C.Use AWS Glue to create an ETL job that runs on a schedule or trigger.
D.Use Amazon Kinesis Data Firehose to transform and load data into Redshift in real time.
AnswerC

AWS Glue provides managed, serverless Spark ETL with job bookmarks, enabling incremental processing of only new S3 objects. Its native Redshift connector and Parquet conversion satisfy the transformation and load requirements, while schedule or event triggers handle the incremental arrival constraint.

Why this answer

AWS Glue provides a serverless ETL service that can run jobs triggered by S3 events to transform data incrementally and load into Redshift. Option A (AWS Lambda) can be used for simple transformations but may hit time limits for complex transformations. Option B (Amazon EMR) is more suited for large-scale big data processing but requires cluster management.

Option D (Amazon Kinesis Data Firehose) is for streaming data, not for batch transformation of existing S3 objects.

204
MCQhard

A company is ingesting streaming data from multiple sources using Amazon Kinesis Data Streams. The data is then processed by an AWS Lambda function that transforms the records and writes them to an Amazon S3 bucket. The Lambda function is failing intermittently with timeout errors. The average record size is 5 KB, and the shard count is 2. What is the MOST likely cause of the timeout errors?

A.The Lambda function timeout is set too low for the processing time required.
B.The Kinesis data retention period is too short, causing data to be lost before processing.
C.The Lambda function's reserved concurrency is set too low, causing throttling.
D.The Lambda function is receiving too many records per invocation, exceeding the 6 MB payload limit.
AnswerA

Lambda timeouts occur when execution exceeds the configured limit; intermittent failures with small 5 KB records and two shards point to processing duration, not throughput. Raising the function timeout addresses the stem's constraint directly, unlike shard or batch-size changes.

Why this answer

The Lambda function is timing out, indicating that the configured timeout is insufficient for the actual processing time. Lambda has a default timeout of 3 seconds, but it can be set from 1 second to 15 minutes. If the transformation logic or S3 write operation takes longer than the configured timeout, the function will fail with a timeout error.

Option B is incorrect because the Kinesis data retention period (default 24 hours) affects data availability, not Lambda execution time. Option C is incorrect because reserved concurrency controls the number of concurrent invocations and can cause throttling, not timeouts. Option D is incorrect because with an average record size of 5 KB, even the default batch size of 100 records results in only 500 KB per invocation, well below the 6 MB payload limit.

205
MCQeasy

Refer to the exhibit. A data engineer runs this AWS Glue Data Catalog DDL statement to create a table. The CSV files in 's3://my-bucket/sales/' use a pipe delimiter (|) instead of a comma. What change is needed to correctly read the data?

A.Change the 'field.delim' property to '|'.
B.Change the LOCATION to read from a subfolder.
C.Add a partition projection configuration.
D.Run a crawler to detect the schema automatically.
AnswerA

Setting `field.delim` to `|` overrides the default comma separator, so the Glue SerDe splits each CSV row on the pipe character rather than commas. This satisfies the stem's constraint that the source files in `s3://my-bucket/sales/` are pipe-delimited, allowing columns to be parsed correctly.

Why this answer

The AWS Glue Data Catalog DDL statement uses the default 'field.delim' property, which expects comma-separated values. Since the CSV files use a pipe delimiter (|), the table will not parse rows correctly. Setting 'field.delim' to '|' in the SerDe properties tells the Hive-compatible SerDe to split on pipes instead of commas, enabling correct data ingestion.

Exam trap

The DEA-C01 exam often tests the misconception that changing the LOCATION or adding partition projection will fix parsing issues, when in fact the core problem is the SerDe delimiter property not matching the actual file format.

How to eliminate wrong answers

Option B is wrong because changing the LOCATION to a subfolder does not alter the delimiter interpretation; it only changes the source path, leaving the parsing issue unresolved. Option C is wrong because partition projection configuration optimizes partition pruning for partitioned tables, but it does not affect how individual records are parsed within files. Option D is wrong because running a crawler would detect the schema and delimiter automatically, but the question explicitly asks what change is needed to the given DDL statement, and a crawler is an alternative approach, not a modification to the existing DDL.

206
MCQeasy

A data engineer needs to transform JSON data into CSV format using AWS Glue. The transformation is simple and must be executed on a schedule. Which Glue component is MOST suitable?

A.Glue Crawler
B.Glue Data Catalog
C.Glue Development Endpoint
D.Glue ETL job
AnswerD

A Glue ETL job runs Apache Spark transformations, converting JSON to CSV, and supports scheduling via triggers. Glue crawlers only catalogue data and DataBrew is interactive, so the ETL job matches the simple scheduled transformation requirement.

Why this answer

A Glue ETL job is the component designed to run transformation scripts (Spark or Python shell) on data, and it can be scheduled via triggers. For a simple JSON-to-CSV transformation on a schedule, an ETL job is the correct choice.

Exam trap

DEA-C01 often tests the misconception that a Crawler performs transformation, when in fact Crawlers only catalog schema and ETL jobs do the actual data processing.

How to eliminate wrong answers

Option A is wrong because a Glue Crawler only discovers schema and populates the Data Catalog; it does not transform data. Option B is wrong because the Glue Data Catalog is a metadata repository, not an execution engine. Option C is wrong because a Development Endpoint is an interactive environment for authoring and testing ETL scripts, not a scheduled production job.

207
Multi-Selectmedium

An e-commerce company is building a near-real-time dashboard to monitor customer clickstream data. The data is ingested via Amazon Kinesis Data Streams, transformed using AWS Lambda, and stored in Amazon S3. The team needs to query the data using Amazon Athena. Which THREE steps should be taken to optimize cost and performance? (Choose three.)

Select 3 answers
A.Use AWS Glue Data Catalog to store the table metadata.
B.Store the data in JSON format for flexibility.
C.Convert the data to Apache Parquet or ORC format.
D.Compress the data using gzip or snappy.
E.Partition the data by date in S3 (e.g., year/month/day).
AnswersC, D, E

Columnar Parquet or ORC lets Athena read only the columns each query references rather than every field, and columnar encoding compresses better. This directly reduces bytes scanned, which is the metric Athena bills on, improving both cost and latency.

Why this answer

Option C is correct because converting the clickstream data to columnar formats like Apache Parquet or ORC lets Athena read only the columns referenced in each query and applies columnar compression and predicate pushdown, dramatically reducing bytes scanned and therefore cost and latency. Option D is correct because compressing data with gzip or snappy reduces the amount of data Athena must read from S3, lowering query cost and improving performance; snappy is especially effective with Parquet/ORC since it is splittable and column-oriented. Option E is correct because partitioning the S3 data by date (e.g., year/month/day) enables Athena partition pruning so queries that filter on time only scan the relevant prefixes instead of the entire dataset.

Option A is not among the marked answers: while the AWS Glue Data Catalog is the standard metastore Athena uses, defining table metadata is a prerequisite for querying rather than a cost/performance optimization step for the data itself. Option B is not correct because storing data as JSON is row-oriented and non-columnar, which forces Athena to scan and parse entire records, increasing bytes scanned and query cost compared with Parquet or ORC.

Exam trap

A common trap is to consider AWS Glue Data Catalog as an optimization step, but it is merely a requirement; the actual optimizations are compression, partitioning, and columnar formats.

208
MCQeasy

A data engineer must load data from an Amazon S3 bucket into an Amazon Redshift cluster as part of a nightly batch pipeline. The source files are already in Parquet format and include columns that map directly to the target table. The engineer wants the fastest, most cost-effective load method that avoids staging the data through an external service. Which command should the engineer use?

A.Redshift Spectrum external table query that selects from S3 into the target table.
B.INSERT statements generated from a Python script that reads the Parquet files.
C.COPY command referencing the S3 bucket with an IAM role.
D.AWS Glue ETL job that reads from S3 and writes to Redshift using the Spark connector.
AnswerC

The COPY command loads data directly from Amazon S3 into Redshift in parallel across slices, which is the fastest and most cost-effective path for Parquet files that already match the target schema. Using an IAM role grants secure access without embedding credentials. It avoids staging through any intermediate service and is the native, recommended bulk-load mechanism for this scenario.

Why this answer

The COPY command performs a parallel bulk load directly from Amazon S3 into Redshift, using an IAM role for secure access. Because the Parquet files already match the target table, no intermediate transformation is required, making COPY the fastest and most economical option. Scripted inserts, Glue, and Spectrum add overhead or serve different purposes.

Exam trap

The trap here is reaching for a managed ETL service or scripted inserts when the native parallel COPY command is both faster and cheaper for schema-aligned S3 data.

209
MCQeasy

A small startup is building a data pipeline to ingest customer orders from a web application into Amazon Redshift for analytics. The orders are written to an Amazon RDS MySQL database. The startup wants to replicate the orders to Redshift in near-real time (within 5 minutes) with minimal operational overhead. The data volume is low, averaging 100 new orders per minute. The startup has a single data engineer who is also responsible for other tasks. What is the simplest solution?

A.Use AWS Glue with a scheduled job every 5 minutes to copy data from MySQL to Redshift
B.Use Amazon EMR with Spark streaming to read from MySQL and write to Redshift
C.Use an AWS Lambda function to query MySQL every minute and insert into Redshift
D.Use AWS Database Migration Service (DMS) with continuous replication
AnswerD

AWS DMS with continuous replication (change data capture) streams ongoing inserts from RDS MySQL to Redshift within minutes, satisfying the near-real-time requirement. It is fully managed, so the lone data engineer avoids building and operating custom replication infrastructure.

Why this answer

AWS DMS can continuously replicate from MySQL to Redshift with minimal setup and low overhead. Option A (AWS Glue) is batch-oriented and may not meet the 5-minute latency. Option B (Amazon EMR) is overkill for low data volumes.

Option C (AWS Lambda) requires custom code and may not efficiently handle the replication.

210
MCQhard

A company uses Amazon Kinesis Data Streams to ingest IoT sensor data. The data is processed by an AWS Lambda function that transforms the records and writes to an Amazon S3 bucket. Recently, the Lambda function has been failing with 'Rate exceeded' errors for the S3 PUT API calls. The data volume is 10 MB/s with average record size 2 KB. What should be done to resolve this issue?

A.Add a random prefix to the S3 object key to distribute writes across multiple prefixes
B.Switch to Amazon Kinesis Data Firehose to write to S3
C.Increase the Lambda function's reserved concurrency
D.Increase the number of Kinesis shards
AnswerA

Random prefixes increase the number of S3 partitions, raising the PUT request limit.

Why this answer

The 'Rate exceeded' error for S3 PUT API calls indicates that the Lambda function is hitting S3 request rate limits. S3 buckets have a default limit of 3,500 PUT requests per second per prefix. With a data volume of 10 MB/s and an average record size of 2 KB, the Lambda function is generating approximately 5,000 PUT requests per second (10 MB/s ÷ 2 KB), which exceeds the per-prefix limit.

Adding a random prefix to the S3 object key distributes writes across multiple prefixes, effectively increasing the aggregate request rate limit to 3,500 PUT requests per second per prefix, thereby resolving the throttling issue.

Exam trap

The trap here is that candidates often confuse Kinesis shard scaling (Option D) or Lambda concurrency (Option C) with S3 rate limits, not realizing that the bottleneck is the S3 API request rate per prefix, not the data ingestion pipeline throughput.

How to eliminate wrong answers

Option B is wrong because Amazon Kinesis Data Firehose writes to S3 in batches and can also encounter S3 rate limits if the underlying prefix is not partitioned; it does not inherently solve the per-prefix request rate limit issue. Option C is wrong because increasing the Lambda function's reserved concurrency would increase the number of concurrent invocations, which would generate even more S3 PUT requests per second, exacerbating the rate limiting problem. Option D is wrong because increasing the number of Kinesis shards increases the data ingestion parallelism but does not affect the S3 PUT request rate limit; the Lambda function would still write to the same S3 prefix at the same rate.

211
Multi-Selecthard

A company uses AWS Glue to run ETL jobs that transform data from Amazon S3 (Parquet) into a denormalized format for Amazon Redshift. The Glue job uses the DynamicFrame API. The job is failing with a 'MemoryError' when performing a join operation. The data is skewed on the join key. Which THREE actions can reduce memory usage and improve job stability? (Choose THREE.)

Select 3 answers
A.Use a broadcast join if one of the tables is small enough.
B.Use a salted join key to distribute skewed keys across partitions.
C.Increase the number of DPUs for the Glue job.
D.Repartition the data on the join key before the join operation.
E.Split the transformation into multiple Glue job steps to reduce per-step memory.
AnswersA, B, E

Avoids shuffling small table.

Why this answer

A broadcast join (using `join` with `broadcast` hint or `DynamicFrame.join(..., transformation_ctx='...')` with broadcast enabled) avoids shuffling the larger table across the cluster by copying the small table to every executor. This eliminates the memory pressure from skewed key distribution during the shuffle phase, reducing the risk of a MemoryError.

Exam trap

The trap here is that candidates often assume increasing resources (DPUs) or repartitioning will fix memory issues, but they fail to recognize that data skew on the join key is the root cause, which requires skew-aware techniques like salting or broadcast joins.

212
MCQeasy

A data engineer needs to transform JSON data from Amazon S3 into Parquet format using AWS Glue. The source files are in a bucket with thousands of small files. What is the best practice to optimize the Glue job performance?

A.Convert the JSON files to CSV before processing with Glue.
B.Enable 'Group small files' in the Glue job or use a DynamicFrame with coalesce.
C.Use an AWS Lambda function to pre-process the files.
D.Increase the number of DPUs to the maximum.
AnswerB

Coalescing merges thousands of small S3 objects into fewer, larger partitions before processing, cutting per-file overhead and task scheduling latency. This satisfies the small-files constraint by reducing the number of Spark tasks and improving Parquet write efficiency.

Why this answer

Enabling 'Group small files' in AWS Glue automatically coalesces thousands of small input files into larger partitions, reducing the number of tasks and minimizing overhead from task scheduling and S3 list operations. This is the recommended best practice for handling small files in Glue ETL jobs, as it optimizes read performance without requiring manual coalesce or repartitioning.

Exam trap

The trap here is that candidates assume more DPUs always improve performance, but for small files the bottleneck is metadata overhead, not compute capacity, so increasing DPUs without addressing file grouping leads to wasted resources and no speedup.

How to eliminate wrong answers

Option A is wrong because converting JSON to CSV adds an unnecessary preprocessing step and does not address the root cause of small file overhead; Glue can read JSON directly and convert to Parquet efficiently. Option C is wrong because using Lambda to pre-process files introduces additional cost, complexity, and potential timeout issues for large numbers of files, and does not leverage Glue's built-in optimization for small files. Option D is wrong because simply increasing DPUs does not solve the small file problem; it may even worsen performance by creating more task slots that compete for the same small files, leading to inefficient resource utilization.

213
MCQeasy

A data engineer needs to run a daily AWS Glue ETL job that transforms data in Amazon S3. The job must start at 2:00 AM UTC every day. The engineer wants to minimize operational overhead and ensure the job runs reliably. Which approach should the engineer use?

A.Configure an AWS Lambda function with a CloudWatch Events rule to start the Glue job daily at 2:00 AM UTC.
B.Use Amazon EventBridge to create a rule with a cron expression '0 2 * * ? *' that targets the AWS Glue job.
C.Use AWS Step Functions with a Wait state to delay execution until 2:00 AM UTC and then start the Glue job.
D.Create an AWS Glue trigger of type 'Schedule' with a cron expression '0 2 * * ? *' and attach it to the job.
AnswerD

AWS Glue triggers of type 'Schedule' allow defining a cron expression to run jobs on a schedule. The expression '0 2 * * ? *' runs daily at 2:00 AM UTC. This is a native Glue feature that requires no external services, minimizing operational overhead and ensuring reliable execution within the Glue environment.

Why this answer

AWS Glue provides built-in schedule triggers that use cron expressions to start jobs at specified times. Creating a trigger with the cron expression '0 2 * * ? *' and attaching it to the job is the simplest, most reliable method. It requires no additional services, reducing operational overhead and potential failure points, while ensuring the job runs daily at 2:00 AM UTC.

Exam trap

The trap here is overcomplicating the scheduling by involving external services like EventBridge or Lambda, when AWS Glue has a native trigger mechanism designed for this exact purpose.

214
MCQeasy

A company needs to ingest data from an on-premises database to Amazon S3 with minimal impact on the source database. The data volume is several TB. Which AWS service is best suited for this task?

A.AWS Direct Connect
B.AWS Snowball Edge
C.AWS Database Migration Service (DMS)
D.Amazon S3 Transfer Acceleration
AnswerC

AWS DMS performs a one-time full load plus ongoing replication by reading from the source via a replication instance, using log-based change capture rather than querying application tables. This minimises load on the on-premises database, satisfying the minimal-impact constraint for multi-terabyte ingestion into Amazon S3.

Why this answer

AWS Database Migration Service (DMS) is best suited because it can continuously replicate data from an on-premises database to Amazon S3 with minimal impact on the source. DMS uses change data capture (CDC) to capture only incremental changes after an initial full load, avoiding heavy read loads on the source database. This makes it ideal for migrating several TB of data while keeping the source operational.

Exam trap

The trap here is that candidates confuse network acceleration services (Direct Connect, Transfer Acceleration) or offline transfer devices (Snowball) with database-specific migration tools, overlooking that DMS is the only option that directly reads from a database with minimal impact via CDC.

How to eliminate wrong answers

Option A is wrong because AWS Direct Connect provides a dedicated network connection for consistent bandwidth, but it does not perform data ingestion or migration itself; it is a transport layer, not a service that reads from a database. Option B is wrong because AWS Snowball Edge is a physical device for offline data transfer, which is suitable for very large datasets (petabytes) but introduces significant latency and is not designed for minimal impact on a live database during continuous ingestion. Option D is wrong because Amazon S3 Transfer Acceleration speeds up uploads to S3 over the internet using optimized network paths, but it does not interact with the source database or handle database-specific data extraction and transformation.

215
Multi-Selectmedium

A data engineer is designing an AWS Glue ETL job that reads from an Amazon S3 bucket containing nested JSON files. The job must flatten the nested structures and write the output to Amazon Redshift. The engineer needs to ensure the job can handle schema evolution and efficiently process only new data on subsequent runs. (Choose two.)

Select 2 answers
A.Use the AWS Glue Relationalize transform to flatten nested JSON structures.
B.Configure the job to use the 'ApplyMapping' transform to rename nested fields.
C.Enable bookmarking in the AWS Glue job to track previously processed data.
D.Use the 'ResolveChoice' transform to handle data type conflicts in nested fields.
E.Set the job's maximum concurrency to 1 to prevent schema conflicts.
AnswersA, C

The Relationalize transform in AWS Glue is specifically designed to flatten nested JSON and semi-structured data into relational tables. It produces multiple tables that can be joined, which is necessary when writing to Amazon Redshift. This transform handles arrays and nested objects, making it the appropriate choice for converting nested JSON into a format suitable for a relational data warehouse.

Why this answer

Enabling job bookmarks ensures that only new data is processed on subsequent runs, which is essential for incremental ETL. The Relationalize transform flattens nested JSON into relational tables, making the data compatible with Amazon Redshift. Together, these two features address the requirements for incremental processing and flattening nested structures.

Exam trap

The trap here is confusing transforms that adjust schema (ApplyMapping, ResolveChoice) with transforms that flatten nested data (Relationalize).

216
MCQhard

A healthcare company processes patient records in near-real-time using Amazon Kinesis Data Streams. Each record contains sensitive personal health information (PHI). The data must be encrypted at rest and in transit. The company also needs to audit access to the data. The data engineer is designing the ingestion pipeline. Which combination of services and configurations meets these requirements?

A.Use Kinesis Data Firehose to deliver data to S3 with SSE-S3, and enable CloudTrail for S3.
B.Use Kinesis Data Streams with TLS and enable CloudTrail for auditing. Do not enable SSE.
C.Use Kinesis Data Streams with SSE-KMS and TLS, and enable CloudTrail for data events.
D.Use Kinesis Data Streams with SSE-KMS and TLS. Do not enable any auditing.
AnswerC

SSE-KMS encrypts stream data at rest, TLS secures it in transit, and CloudTrail data events log every API-level access to the stream. Together these satisfy the PHI encryption and access-auditing requirements for the ingestion pipeline.

Why this answer

Kinesis Data Streams supports server-side encryption with AWS KMS (SSE-KMS) for encryption at rest, and TLS is used for encryption in transit by default. Enabling CloudTrail data events captures API-level access to Kinesis streams (e.g., PutRecord, GetRecords), satisfying the audit requirement. This combination meets all three requirements: encryption at rest, encryption in transit, and auditable access.

Exam trap

DEA-C01 often tests the distinction between encryption at rest (SSE-KMS on the stream) and encryption in transit (TLS), plus the difference between CloudTrail management events (automatic) and data events (opt-in) — candidates who assume CloudTrail logs all access by default will pick the wrong answer.

How to eliminate wrong answers

Option A is wrong because it uses Kinesis Data Firehose (a delivery service, not a stream-processing service) and SSE-S3, which encrypts only the S3 destination, not the data in the Firehose buffer or during ingestion; also, CloudTrail for S3 does not audit Kinesis access. Option B is wrong because it explicitly disables SSE, failing the encryption-at-rest requirement for PHI, which is a compliance violation (HIPAA). Option D is wrong because it omits auditing entirely, failing the requirement to audit access to the data, which is mandatory for PHI under HIPAA.

217
Multi-Selecthard

A data engineer is configuring an AWS Glue ETL job to read data from an Amazon S3 bucket that contains nested JSON files. The job must flatten the nested structure and write the output to Amazon Redshift. The engineer wants to optimize the job for performance and cost. Which two actions should the engineer take? (Choose two.)

Select 2 answers
A.Increase the number of DPUs to the maximum allowed to ensure faster execution.
B.Enable job bookmarks to track processed data and avoid reprocessing.
C.Convert the JSON files to Parquet using an AWS Glue crawler before running the ETL job.
D.Use the AWS Glue DynamicFrame Relationalize transform to flatten nested JSON.
E.Use the AWS Glue ResolveChoice transform to handle data type conflicts.
AnswersB, D

Job bookmarks in AWS Glue keep track of data that has already been processed in previous job runs. When reading from S3, bookmarks use the last modified time of objects to determine new data. Enabling bookmarks prevents reprocessing of unchanged data, reducing runtime and cost, especially for incremental loads. This is a best practice for ETL jobs that run repeatedly.

Why this answer

The Relationalize transform is purpose-built to flatten nested JSON, simplifying the ETL code and improving performance. Enabling job bookmarks ensures that only new or changed data is processed on subsequent runs, reducing cost and runtime. Together, these actions optimize the job for both performance and cost.

The other options either do not address flattening, are not cost-effective, or misunderstand service capabilities.

Exam trap

The trap here is assuming that a Glue crawler can convert file formats or that simply adding more DPUs is always the best way to optimize performance without considering cost.

218
MCQeasy

A data engineer needs to ingest on-premises CSV files into Amazon S3 every hour. The files are less than 1 GB each. Which service is the most cost-effective and requires the least operational overhead?

A.AWS DataSync
B.Amazon Kinesis Data Firehose
C.AWS Snowball Edge
D.AWS Database Migration Service (DMS)
AnswerA

AWS DataSync provides a managed agent that incrementally transfers on-premises files to Amazon S3 on a schedule, meeting the hourly, sub-1 GB, low-overhead constraint. It handles scheduling, integrity verification and retries natively, unlike scripting the AWS CLI or running bespoke cron jobs.

Why this answer

AWS DataSync is the most cost-effective and least overhead option for scheduled, recurring transfers of on-premises CSV files to S3. It provides a simple agent-based setup, supports hourly scheduling, and handles files under 1GB efficiently without complex configuration. In contrast, Amazon Kinesis Data Firehose is designed for streaming data ingestion, not batch file transfers; AWS Snowball Edge is intended for large-scale offline data migrations, not hourly incremental transfers; and AWS Database Migration Service (DMS) is specialized for migrating databases, not file transfers.

219
MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format. The delivery stream is configured with a buffer size of 5 MB and a buffer interval of 60 seconds. However, the data engineer notices that S3 objects are being created with sizes much smaller than 5 MB. What is a likely cause?

A.The data is being compressed before delivery, reducing object size.
B.The incoming data rate is too low, causing the buffer interval to trigger before reaching the buffer size.
C.The data transformation lambda is splitting records into smaller ones.
D.The S3 bucket is configured with a lifecycle policy that splits objects.
AnswerB

Firehose flushes a buffer when either the 5 MB size or the 60-second interval elapses, whichever occurs first. At a low incoming data rate, the interval triggers repeatedly before the buffer fills, producing S3 objects far smaller than 5 MB.

Why this answer

Kinesis Data Firehose delivers data to S3 when either the buffer size (5 MB) or the buffer interval (60 seconds) is reached, whichever occurs first. If the incoming data rate is low, the buffer interval will expire before accumulating 5 MB of data, resulting in smaller S3 objects.

Exam trap

The trap here is that candidates may assume the buffer size is a hard minimum that must be reached before delivery, but Firehose uses an 'or' condition between buffer size and buffer interval, so low data rate causes interval-based delivery of small objects.

How to eliminate wrong answers

Option A is wrong because compression reduces the size of data after buffering, but the buffer size limit is based on the uncompressed data; compression does not cause smaller objects to be created before the buffer interval triggers. Option C is wrong because a data transformation Lambda can modify records but does not inherently split records into smaller ones; it processes records as a batch and returns them, and any splitting would be a custom logic not default behavior. Option D is wrong because S3 lifecycle policies manage object transitions or deletions after objects are created; they do not split objects during delivery.

220
MCQmedium

Refer to the exhibit. An IAM policy for an AWS Lambda function. The Lambda function is triggered by an S3 event (object created) and needs to read from a Kinesis stream. However, the function fails with access denied when trying to read from Kinesis. What is the most likely cause?

A.The Lambda function is not in the same region as the Kinesis stream
B.The Lambda function does not have permission to list S3 buckets
C.The Kinesis stream is encrypted with a customer managed KMS key, and the Lambda function lacks kms:Decrypt permission
D.The S3 bucket policy denies access to the Lambda function
AnswerC

Reading from a Kinesis stream encrypted with a customer managed KMS key requires kms:Decrypt in the Lambda execution role. The S3 event trigger and stream read permissions may be correct, but the missing KMS grant causes the access denied error.

Why this answer

When a Kinesis stream is encrypted with a customer managed KMS key, the Lambda function must have the `kms:Decrypt` permission on that key to read data from the stream. Without this permission, the Lambda function will receive an access denied error even if it has the necessary Kinesis actions (e.g., `kinesis:GetRecords`) allowed in its IAM policy. The S3 event trigger only invokes the function; it does not grant Kinesis access.

Exam trap

The DEA-C01 exam often tests the interaction between Kinesis SSE-KMS and Lambda IAM permissions, trapping candidates who assume that Kinesis read permissions alone are sufficient without considering the KMS key policy.

How to eliminate wrong answers

Option A is wrong because Lambda functions can access Kinesis streams across regions as long as the IAM permissions and network connectivity (e.g., VPC endpoints) are correctly configured; region mismatch does not inherently cause access denied. Option B is wrong because the Lambda function is triggered by an S3 event and only needs permission to read from Kinesis; listing S3 buckets is irrelevant to the Kinesis read failure. Option D is wrong because the S3 bucket policy controls access to the S3 bucket itself, not to Kinesis; the error occurs when reading from Kinesis, not when the S3 event triggers the function.

221
MCQeasy

A data engineer is setting up an Amazon Kinesis Data Firehose delivery stream to load data into Amazon Redshift. The data is coming from an application that produces JSON records. The engineer needs to transform the data to match the Redshift table schema. Which approach is the MOST cost-effective and requires the least operational overhead?

A.Use AWS Glue as a transformation step between Firehose and Redshift, with a trigger on S3.
B.Use Kinesis Data Firehose with direct PUT to Redshift and rely on Redshift's COPY command to transform.
C.Configure a Lambda function in the Firehose delivery stream to transform records before delivery.
D.Use the Kinesis Client Library (KCL) to consume the stream, transform in an EC2 instance, and then load to Redshift.
AnswerC

Firehose invokes a Lambda function inline to transform each JSON record before loading, so no intermediate storage or separate processing cluster is needed. This satisfies the least-operational-overhead and cost-effectiveness constraints, since you pay only per invocation.

Why this answer

Kinesis Data Firehose natively supports invoking a Lambda function as a transformation step within the delivery stream. This allows the engineer to write a simple Lambda function that parses the incoming JSON records and transforms them to match the Redshift table schema, all without provisioning or managing any additional infrastructure. This approach is the most cost-effective (pay per invocation) and requires the least operational overhead since Firehose handles the orchestration, retries, and delivery to Redshift automatically.

Exam trap

The trap here is that candidates often overestimate the transformation capabilities of Redshift's COPY command, mistakenly believing it can perform complex record-level transformations, when in fact it only supports basic data mapping and format parsing, not arbitrary JSON restructuring.

How to eliminate wrong answers

Option A is wrong because inserting AWS Glue as an intermediate step between Firehose and Redshift introduces unnecessary complexity, cost (Glue jobs run on a per-DPU-hour basis), and latency, as Glue is designed for batch ETL, not real-time streaming transformations. Option B is wrong because Redshift's COPY command does not perform record-level transformations; it only maps source fields to target columns and can apply basic data format conversions (e.g., JSON parsing via 'jsonpaths'), but it cannot restructure or compute new fields from the JSON payload. Option D is wrong because using the Kinesis Client Library (KCL) on an EC2 instance requires manual provisioning, scaling, and management of the EC2 fleet, which incurs significant operational overhead and cost compared to the serverless Lambda integration within Firehose.

222
MCQmedium

A company is building a data lake on Amazon S3 and wants to ingest data from multiple AWS services (CloudTrail, VPC Flow Logs, and ALB logs). The data should be stored in a central S3 bucket with a common partitioning scheme. Which service can be used to collect and centralize this data with minimal configuration?

A.Use AWS Data Pipeline to copy logs from each source S3 bucket to the central bucket.
B.Use AWS Glue to crawl the logs from each source and write to a central S3 bucket.
C.Set up Amazon Kinesis Data Firehose to ingest logs from each service and write to S3.
D.Configure each source service to deliver logs directly to the central S3 bucket.
AnswerD

CloudTrail, VPC Flow Logs and ALB each natively publish logs to S3, so pointing them at one bucket centralises delivery with no pipeline service to manage. This satisfies the minimal-configuration constraint, though each source writes its own prefix scheme.

Why this answer

CloudTrail, VPC Flow Logs, and ALB logs can each be configured to deliver logs directly to a specified S3 bucket, including a central bucket, with no intermediary service required. This approach minimizes configuration overhead and avoids data movement costs, as each service writes natively to S3 using its own built-in delivery mechanism.

Exam trap

The trap here is that candidates often overcomplicate the solution by choosing a data pipeline or ETL service (like Data Pipeline or Glue) when the simplest and most efficient method is to configure each source service to write directly to the central S3 bucket, leveraging native AWS integrations.

How to eliminate wrong answers

Option A is wrong because AWS Data Pipeline is designed for scheduled data movement and transformation between data stores, not for real-time log ingestion from multiple AWS services; it would require custom pipeline definitions and adds unnecessary complexity. Option B is wrong because AWS Glue is an ETL service for crawling, cataloging, and transforming data, not a log collection or delivery service; it cannot natively ingest logs from CloudTrail, VPC Flow Logs, or ALB logs without first having the data in S3. Option C is wrong because Amazon Kinesis Data Firehose can ingest streaming data but does not natively subscribe to CloudTrail, VPC Flow Logs, or ALB logs; these services do not send data to Firehose directly, requiring additional setup like CloudWatch Logs subscriptions or custom agents.

223
MCQeasy

A data engineer needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The network bandwidth is limited to 100 Mbps. The transfer must be completed within one week. Which service should be used?

A.AWS DataSync
B.AWS Snowball
C.Amazon CloudFront
D.AWS Database Migration Service (DMS)
AnswerB

At 100 Mbps, transferring 50 TB over the network takes roughly 46 days, far exceeding the one-week deadline. AWS Snowball ships data physically on a rugged appliance, bypassing the bandwidth constraint and completing the migration within the required timeframe.

Why this answer

AWS Snowball is the correct choice because transferring 50 TB over a 100 Mbps network would take approximately 46 days (50 TB * 8 bits/byte / 100 Mbps / 86400 seconds/day), far exceeding the one-week deadline. Snowball provides a physical storage device that can be shipped to the on-premises location, allowing data to be loaded locally and shipped to AWS, bypassing network bandwidth constraints entirely.

Exam trap

The trap here is that candidates may underestimate the time required for online transfer and choose AWS DataSync, failing to calculate that 50 TB at 100 Mbps takes over 46 days, not one week, and overlooking Snowball's physical shipping approach for offline data transfer.

How to eliminate wrong answers

Option A is wrong because AWS DataSync is designed for online data transfer over the network, and at 100 Mbps, it would take over 46 days to transfer 50 TB, which does not meet the one-week requirement. Option C is wrong because Amazon CloudFront is a content delivery network (CDN) for caching and distributing content to edge locations, not a data transfer service for ingesting large volumes of historical data into S3. Option D is wrong because AWS Database Migration Service (DMS) is specialized for migrating databases (e.g., relational, NoSQL) and does not support transferring HDFS files or large-scale file-based data to S3.

224
MCQmedium

A company uses AWS Glue to catalog data in Amazon S3. The data arrives in Parquet format, but the crawler fails to update the schema when new columns are added. What is the most likely cause?

A.The Glue Data Catalog is not configured to accept schema changes.
B.Parquet files do not support schema evolution.
C.The crawler configuration has 'Update the table definition' set to 'Ignore the change'.
D.The S3 bucket has versioning disabled.
AnswerC

If set to ignore changes, the crawler will not add new columns.

Why this answer

AWS Glue crawlers have a table-update behavior controlled by the 'Update the table definition in the Data Catalog' setting (SchemaChangePolicy.UpdateBehavior). If it is set to 'Ignore the change' (LOG), the crawler detects new columns but does not modify the table definition, so the schema appears stale. Setting it to 'Update the table definition' (UPDATE_IN_DATABASE) allows the crawler to add new columns to the catalog table.

Exam trap

DEA-C01 often tests the assumption that Parquet cannot evolve schemas or that the Data Catalog has a global setting — candidates miss that the crawler's per-run schema change policy is the actual control.

How to eliminate wrong answers

Option A is wrong because the Glue Data Catalog does accept schema changes — there is no global 'accept schema changes' toggle; the behavior is controlled per-crawler via the schema change policy. Option B is wrong because Parquet absolutely supports schema evolution (adding columns is a core feature of the format, and readers can handle missing columns with nulls). Option D is wrong because S3 bucket versioning is unrelated to Glue crawler schema detection — versioning affects object retention, not catalog updates.

225
MCQhard

A data engineer runs an AWS Glue job that reads Parquet files from Amazon S3 partitioned by year/month/day and writes to another S3 prefix. The job currently processes all historical partitions on every run, causing long runtimes and high cost. The engineer wants subsequent runs to process only new data. Which configuration should the engineer apply?

A.Configure an S3 Lifecycle rule to transition old partitions to S3 Glacier Instant Retrieval so the Glue job reads them faster.
B.Increase the number of AWS Glue DPUs allocated to the job so it can process all partitions faster.
C.Enable AWS Glue job bookmarks and ensure the job's source and target paths are stable across runs.
D.Set the Glue job's MaxConcurrentRuns parameter to 1 to prevent overlapping executions from reprocessing data.
AnswerC

Job bookmarks persist state about which partitions and files have already been processed, so reruns skip previously handled data. This directly reduces runtime and cost by limiting each run to new partitions. Bookmarks require stable source and target paths and a consistent transformation script to remain effective.

Why this answer

Job bookmarks maintain persistent state across runs, allowing Glue to skip files and partitions that were already processed in prior executions. This is the native mechanism for incremental processing in Glue ETL, directly reducing scanned data, runtime, and cost without changing the transformation logic or adding external state tracking.

Exam trap

The trap here is confusing concurrency or compute scaling controls with incremental processing, when only job bookmarks track previously processed partitions.

← PreviousPage 3 of 6 · 447 questions totalNext →

Ready to test yourself?

Try a timed practice session using only Data Ingestion Transformation questions.