Courseiva

CCNA Data Ingestion Transformation Questions

75 of 447 questions · Page 5/6 · Data Ingestion Transformation topic · Answers revealed

301
MCQmedium

A data engineer is using AWS Database Migration Service (AWS DMS) to migrate a 4 TB on-premises Oracle database to Amazon Aurora PostgreSQL. The migration must complete in a single maintenance window, and the source database cannot be taken offline for more than 30 minutes. The engineer configures a full load plus change data capture (CDC) task. During testing, the full load phase takes 14 hours. Which configuration change will most effectively reduce the time required for the full load phase?

A.Enable parallel load by increasing the number of tables loaded concurrently in the task settings.
B.Increase the Aurora PostgreSQL instance class to the largest available memory-optimized instance.
C.Enable Multi-AZ on the Aurora PostgreSQL cluster before starting the migration task.
D.Change the migration method from full load plus CDC to CDC only.
AnswerA

AWS DMS supports parallel full load, which loads multiple tables simultaneously instead of serially. For a 4 TB Oracle database with many tables, increasing the number of tables loaded concurrently in the task settings significantly reduces overall full load time by using multiple threads and network connections. This is the most effective and supported tuning mechanism for reducing full load duration without changing the source or target.

Why this answer

Parallel full load is the primary DMS tuning lever for reducing bulk migration time. By loading multiple tables concurrently, DMS saturates available network and I/O resources rather than processing tables serially. Switching to CDC only would skip existing data entirely, and scaling the target or enabling Multi-AZ does not address the serial loading bottleneck that dominates a large full load.

Exam trap

The trap here is assuming that scaling the target database or enabling high availability will speed up a DMS full load, when the real bottleneck is the serial table-by-table loading that parallel load addresses.

302
Multi-Selecteasy

A data engineer needs to ingest data from a SaaS application (Salesforce) into Amazon S3 on a daily basis. Which TWO AWS services can be used for this purpose? (Choose TWO.)

Select 2 answers
A.AWS DataSync
B.Amazon Kinesis Data Streams
C.AWS Transfer Family
D.AWS Glue
E.Amazon AppFlow
AnswersD, E

AWS Glue provides built-in Salesforce connectors and can run on a daily schedule, extracting records and writing them directly to Amazon S3. This satisfies the daily ingestion requirement without managing servers, complementing AppFlow as the second valid service.

Why this answer

Amazon AppFlow (option E) is purpose-built for ingesting data from SaaS applications like Salesforce into AWS destinations such as Amazon S3, supporting scheduled daily flows without custom code. AWS Glue (option D) can also perform this ingestion by using its Salesforce connector within Glue ETL jobs or Glue Studio, writing the extracted data to S3 on a schedule. AWS DataSync (A) is designed for migrating and syncing file data between on-premises storage and AWS, not for SaaS API ingestion.

Amazon Kinesis Data Streams (B) is a real-time streaming service that requires custom producers and does not natively connect to Salesforce. AWS Transfer Family (C) only provides managed SFTP, FTPS, and FTP endpoints into S3 or EFS, which does not address SaaS application ingestion.

Exam trap

The trap here is that candidates may confuse AWS DataSync's ability to sync from cloud sources (like EFS) with SaaS applications, but DataSync does not support Salesforce or other SaaS APIs natively.

303
MCQhard

Refer to the exhibit. A data engineer has configured an S3 event notification to send an event to an SQS queue when objects are created in the 'incoming/' prefix. The engineer wants to trigger an AWS Lambda function to process the object. However, the Lambda function is not being invoked. What is the most likely cause?

A.The Lambda function lacks permission to read from the S3 bucket.
B.Lambda is not configured as an event source for the SQS queue.
C.The SQS queue does not exist or is in a different account.
D.The S3 event notification filter prefix is incorrect.
AnswerB

An SQS queue only buffers messages; Lambda must be configured with the queue as an event source mapping to poll and invoke. Without that mapping, notifications accumulate unprocessed, so the function never runs despite S3 delivering events correctly.

Why this answer

The Lambda function is not being invoked because the SQS queue is not configured as an event source for Lambda. Even though S3 sends events to SQS, Lambda will only poll and process messages from the queue if an event source mapping (e.g., via CreateEventSourceMapping) is established. Without this mapping, the messages sit in the queue and Lambda remains idle.

Exam trap

The DEA-C01 exam often tests the distinction between sending events to a queue (S3 → SQS) and actually consuming them (SQS → Lambda), so candidates mistakenly assume that simply having S3 send to SQS will automatically trigger Lambda without an explicit event source mapping.

How to eliminate wrong answers

Option A is wrong because the Lambda function does not need permission to read from the S3 bucket directly; it only needs permission to read from the SQS queue (via the event source mapping) and to access the object in S3 once triggered. Option C is wrong because if the SQS queue did not exist or was in a different account, the S3 event notification would fail immediately (S3 would log an error), but the question states the event is sent to SQS, implying the queue exists and is accessible. Option D is wrong because the S3 event notification filter prefix 'incoming/' is correctly configured to match objects created under that prefix; if it were incorrect, no events would be sent to SQS at all.

304
MCQhard

A data engineer maintains an AWS Glue ETL job that reads JSON from Amazon S3 and writes Parquet to a second bucket. Downstream consumers report that numeric fields occasionally arrive as strings and timestamps are sometimes null. The engineer must make the job resilient to these schema variations without failing the run. Which approach should the engineer take?

A.Switch the output format to CSV so type coercion is handled by the consumer
B.Use a ResolveChoice transform to cast columns and a FillMissingValues transform to populate null timestamps
C.Enable the AWS Glue Data Catalog schema evolution option and set the job to merge schemas from the source
D.Increase the number of DPU workers and enable job bookmarks
AnswerB

ResolveChoice resolves ambiguous or mixed types such as a column that is sometimes a string and sometimes a number, applying a cast or type promotion so the output Parquet has a consistent schema. FillMissingValues then supplies defaults for null timestamps. Together they let the job complete and emit uniform Parquet instead of failing or producing inconsistent types downstream.

Why this answer

Mixed types and null timestamps are data-quality issues that must be normalized inside the transformation. ResolveChoice reconciles a column that arrives as both string and numeric into a single declared type, and FillMissingValues substitutes defaults for absent timestamps. Together they let the job emit consistent Parquet without failing on schema variance.

Exam trap

The trap here is reaching for catalog-level schema evolution or more compute when the real problem is per-record type inconsistency that only a transform such as ResolveChoice can reconcile.

305
MCQhard

A data engineer is using AWS Glue Studio to build a job that reads from an Amazon Kinesis Data Stream, performs a 5-minute tumbling window aggregation, and writes results to Amazon S3. The job must run continuously and handle late-arriving records within the window. Which configuration should the engineer use?

A.Configure the job with the Glue streaming ETL type, set the window size to 5 minutes, and enable checkpointing to track stream position
B.Use a Glue batch job with the 'groupBy' transform set to a 5-minute window and run it once per hour
C.Use an AWS Lambda function triggered by the stream with a 5-minute timeout to aggregate records and write to S3
D.Configure the job as a batch job with a schedule trigger that runs every 5 minutes and reads the stream's latest records
AnswerA

Glue streaming ETL jobs run continuously, consume from Kinesis, and support windowed aggregations with a configurable window size. Checkpointing tracks the stream position so the job resumes correctly and handles late records within the window as designed.

Why this answer

Glue streaming ETL is purpose-built for continuous consumption from Kinesis and supports windowed aggregations with checkpointing. Selecting the streaming job type with a 5-minute window and enabled checkpoints gives the continuous, late-tolerant behavior the scenario requires, unlike scheduled batch jobs or Lambda.

Exam trap

The trap here is choosing a scheduled batch job because the window is 5 minutes, when only a Glue streaming ETL job provides continuous windowed aggregation with checkpointing.

306
MCQeasy

A data engineer needs to run a Python-based transformation on each object as it lands in an Amazon S3 bucket. The objects are small (under 10 MB), arrive sporadically, and must be processed within seconds. Which approach is MOST appropriate?

A.Schedule an AWS Glue crawler and ETL job to run every 15 minutes over the bucket.
B.Run an Amazon EMR cluster with a step function that polls the bucket every minute.
C.Use Amazon Kinesis Data Firehose to deliver objects to a processing endpoint.
D.Configure an S3 event notification to invoke an AWS Lambda function that performs the transformation.
AnswerD

S3 event notifications can trigger Lambda directly on object creation, giving near-real-time, per-object processing without managing servers. Lambda handles small payloads well within its 15-minute limit, and you pay only per invocation, which matches sporadic arrivals and sub-10 MB objects perfectly.

Why this answer

S3 event notifications triggering Lambda is the standard serverless pattern for reacting to individual object uploads in near real time. It avoids polling, scales automatically, and suits small files and sporadic arrivals. Glue, Firehose, and EMR introduce batching, buffering, or cluster overhead that conflicts with the seconds-level, per-object requirement.

Exam trap

The trap here is reaching for a managed ETL service like Glue when the requirement is event-driven per-object processing with seconds-level latency.

307
MCQhard

A data engineer runs a weekly AWS Glue ETL job that processes data from Amazon DynamoDB to Amazon S3. The job reads the entire table every time, which is slow and expensive. The job needs to process only items that changed since the last run. Which solution should the engineer implement?

A.Use DynamoDB Scan with a LastEvaluatedKey to paginate and store the last scanned key to resume next time
B.Enable DynamoDB Streams and process change events with AWS Lambda to write to S3
C.Add a Global Secondary Index (GSI) on a timestamp attribute and query only new records
D.Use AWS Database Migration Service (DMS) with ongoing replication from DynamoDB to S3
AnswerB

DynamoDB Streams captures item-level changes in near real time, and Lambda consumes those events to write only modified items to Amazon S3. This satisfies the requirement to process only items changed since the last run, eliminating the slow, expensive full-table scans the weekly Glue job currently performs.

Why this answer

DynamoDB Streams captures item-level changes (inserts, updates, deletes) in near real-time. An AWS Lambda function can process these change events and write only the incremental data to Amazon S3, eliminating the need to scan the entire DynamoDB table. This approach is both cost-effective and efficient for incremental data ingestion.

Exam trap

The trap here is that candidates may think a GSI on a timestamp (Option C) is sufficient for incremental processing, but it fails to capture updates to existing items that do not change the timestamp, and it still requires a full scan of the index to find new records.

How to eliminate wrong answers

Option A is wrong because using DynamoDB Scan with LastEvaluatedKey still reads the entire table every time; it only paginates the results, not reduces the data read. Option C is wrong because a Global Secondary Index (GSI) on a timestamp attribute does not automatically track changes; you would still need to query for new records based on a timestamp, which requires storing the last processed timestamp and can miss updates to existing items. Option D is wrong because AWS Database Migration Service (DMS) with ongoing replication is designed for continuous database migration and can be complex to set up for simple incremental loads; it is overkill compared to using DynamoDB Streams with Lambda, and DMS does not natively write to S3 in a format optimized for analytics without additional transformation.

308
MCQmedium

A data engineer needs to transform JSON data into Parquet format using AWS Glue. The input data has nested fields. Which Glue feature should be used to flatten the nested structure?

A.Relationalize transform
B.DropNullFields transform
C.FindMatches transform
D.Map transform
AnswerA

Relationalize is a Glue transform that converts nested, semi-structured data into a set of flat relational tables, unnesting arrays and structs into separate tables linked by join keys. It directly addresses flattening nested JSON before writing Parquet.

Why this answer

The Relationalize transform is the correct choice because it is specifically designed to flatten nested JSON structures (such as arrays and structs) into a set of related tables (DataFrames) that can be written as Parquet. This transform recursively extracts nested fields, creating separate DataFrames for each level of nesting, which is essential for converting complex JSON into a flat, columnar Parquet format suitable for analytics.

Exam trap

The trap here is that candidates may confuse the Map transform (which can flatten JSON with custom code) with a built-in flattening feature, but AWS Glue's Relationalize is the dedicated, no-code solution for this specific task, and the exam expects you to know the exact purpose of each transform.

How to eliminate wrong answers

Option B (DropNullFields transform) is wrong because it only removes fields with null values from the schema, not flattening nested structures. Option C (FindMatches transform) is wrong because it is used for fuzzy matching and deduplication of records, not for schema transformation. Option D (Map transform) is wrong because it applies a custom function to each row or column but does not inherently flatten nested JSON; it requires manual coding to handle nesting, whereas Relationalize automates the process.

309
MCQmedium

A data engineer is using AWS Database Migration Service (AWS DMS) to migrate a large on-premises Oracle database to Amazon Aurora PostgreSQL. The migration must minimize downtime, so the engineer needs to capture ongoing changes while the initial full load runs. Which AWS DMS task configuration should the engineer use?

A.Full load only, followed by a separate full load task after cutover.
B.Change data capture (CDC) only, without an initial full load.
C.Full load plus CDC, but disable the target schema creation.
D.Full load plus change data capture (CDC) in a single task.
AnswerD

A full load plus CDC task first copies existing rows and then continuously applies ongoing changes captured from the source transaction logs, keeping the target in sync until cutover. This minimizes downtime because the target is current at the moment of switchover, which is exactly what the low-downtime requirement demands.

Why this answer

To minimize downtime during a database migration, AWS DMS should run a task that performs a full load and then continues with change data capture. The full load seeds the target with existing rows, and CDC applies ongoing inserts, updates, and deletes from the source logs, keeping Aurora PostgreSQL synchronized until the cutover window, which can then be brief.

Exam trap

The trap here is thinking that a full load alone keeps the target current, when in fact changes made during and after the load require change data capture to be applied.

310
MCQeasy

A data engineer needs to schedule an AWS Glue ETL job to run every hour and process new data that arrives in an S3 bucket. The job should only process files that have been added since the last run. Which approach should the engineer use to track which files have been processed?

A.Configure S3 Event Notifications to trigger the Glue job on each new object creation.
B.Enable job bookmarks in the Glue ETL job.
C.Store the last processed timestamp in a DynamoDB table and query it at the start of the job.
D.Use S3 Inventory to list all objects and filter by last modified date in the job.
AnswerB

Job bookmarks persist state across runs, letting Glue track previously processed S3 objects by path and last-modified timestamp. This satisfies the hourly incremental requirement: each run reads only files added since the prior execution, avoiding full reprocessing. No external tracking table or custom filtering logic is needed.

Why this answer

AWS Glue job bookmarks automatically track the last processed data, allowing the job to incrementally process only new files since the last run. This is the native, built-in mechanism for stateful incremental processing in Glue ETL, eliminating the need for external tracking.

Exam trap

The trap here is that candidates often choose Option A (S3 Event Notifications) because it seems like a direct trigger for new files, but they overlook that the requirement is for an hourly batch job that tracks processed files across runs, not a per-file event-driven trigger.

How to eliminate wrong answers

Option A is wrong because S3 Event Notifications trigger the job per object creation, which can lead to duplicate processing if multiple files arrive within the hour and does not inherently track which files have been processed across runs. Option C is wrong because storing the last processed timestamp in DynamoDB requires custom implementation and maintenance, and it does not handle edge cases like file overwrites or partitions as reliably as Glue bookmarks. Option D is wrong because S3 Inventory provides periodic snapshots (daily or weekly) and is not designed for real-time or hourly incremental processing; filtering by last modified date in the job would require full scans and manual state management.

311
MCQeasy

A company wants to schedule a nightly batch job to copy data from an on-premises PostgreSQL database to Amazon S3. The solution must minimize operational overhead. Which AWS service should be used?

A.AWS Glue
B.AWS Data Pipeline
C.Amazon EMR
D.AWS Database Migration Service (AWS DMS) with ongoing replication
AnswerA

AWS Glue provides serverless, managed ETL with built-in scheduling and JDBC connectivity to on-premises PostgreSQL, so no infrastructure is provisioned or maintained. This meets the nightly batch copy requirement to Amazon S3 while minimising operational overhead.

Why this answer

AWS Glue is the correct choice because it provides a fully managed ETL service that can connect to on-premises PostgreSQL via JDBC, extract data, and write it to Amazon S3 with minimal configuration. Glue's built-in scheduler can run the job nightly, eliminating the need to manage servers or orchestration infrastructure, which directly meets the requirement to minimize operational overhead.

Exam trap

The trap here is that candidates often confuse AWS DMS (designed for continuous replication) with batch data movement, overlooking that DMS's ongoing replication feature is not intended for scheduled batch jobs and adds unnecessary overhead for a simple nightly copy.

How to eliminate wrong answers

Option B (AWS Data Pipeline) is wrong because, while it can copy data from on-premises databases to S3, it requires managing a task runner on-premises and has higher operational overhead compared to Glue's serverless model. Option C (Amazon EMR) is wrong because it is designed for big data processing using Hadoop/Spark clusters and introduces significant overhead for cluster management, making it overkill for a simple nightly batch copy job. Option D (AWS DMS with ongoing replication) is wrong because it is primarily intended for database migration and continuous replication, not for scheduled batch jobs; using it for nightly batch copies would incur unnecessary complexity and cost for ongoing change data capture.

312
Multi-Selecthard

A data engineer is troubleshooting an AWS Glue job that reads from Amazon S3 and writes to Amazon Redshift. The job runs successfully but 5% of records are missing after the load. The engineer suspects data consistency issues. Which THREE actions could help diagnose and resolve the problem? (Choose THREE.)

Select 3 answers
A.Use the Redshift COPY command with a manifest file to load data.
B.Increase the number of DPUs for the Glue job.
C.Enable Glue job bookmarks to track processed files.
D.Use a staging table in Redshift with a transaction to commit.
E.Review the job's CloudWatch Logs for any error messages.
AnswersA, C, E

A manifest file explicitly lists every S3 object to load, so COPY fails loudly on missing or unreadable files rather than silently skipping them. This surfaces the 5% discrepancy and prevents partial loads, addressing the suspected consistency issue.

Why this answer

Option A is correct because using the Redshift COPY command with a manifest file explicitly lists every S3 object to be loaded, preventing silent omissions of files that can occur with prefix-based loads and making missing records easier to detect. Option C is correct because Glue job bookmarks track which S3 files have already been processed, so enabling them prevents files from being skipped or reprocessed and helps identify gaps in the input data that cause missing records. Option E is correct because reviewing the job's CloudWatch Logs surfaces errors, warnings, and skipped-record messages from Glue and Redshift, which is the primary diagnostic step for understanding why 5% of records are missing.

Option B does not belong because increasing DPUs only scales compute capacity and does not address data consistency or missing records. Option D does not belong because a staging table with a transaction ensures atomicity of the load but does not by itself diagnose or resolve records being dropped during extraction or copy.

Exam trap

The trap here is that candidates often assume performance tuning (increasing DPUs) or database-level transactions (staging tables) can fix data ingestion gaps, when the actual problem is incomplete or inconsistent file discovery from the source (S3).

313
MCQmedium

A data engineer is using AWS Glue ETL to transform a large dataset in S3. The job processes 2 TB of data daily and currently runs for 6 hours. The engineer wants to reduce runtime without changing the transformation logic. What is the best approach?

A.Reduce the number of DPUs to minimize overhead.
B.Use the Spark UI to analyze bottlenecks and rewrite code.
C.Increase the number of Glue DPUs or enable auto-scaling.
D.Switch from Spark to Python shell.
AnswerC

Adding DPUs or enabling auto-scaling increases the number of concurrent executors processing partitions in parallel, so the same 2 TB workload completes faster. This reduces the six-hour runtime without altering transformation logic, satisfying the constraint of unchanged code.

Why this answer

Increasing the number of DPUs or enabling auto-scaling directly allocates more distributed processing capacity to the AWS Glue job, which reduces runtime for large datasets by parallelizing the workload across more resources. Since the transformation logic is fixed and the job is already running on Spark, adding compute capacity is the most straightforward way to speed up processing without code changes.

Exam trap

The trap here is that candidates may think reducing DPUs reduces overhead and speeds up the job, but in distributed systems, more parallelism (more DPUs) reduces runtime for large datasets, while reducing DPUs increases it.

How to eliminate wrong answers

Option A is wrong because reducing DPUs would decrease parallelism and likely increase runtime, not reduce it, as the job already takes 6 hours on 2 TB of data. Option B is wrong because using the Spark UI to analyze bottlenecks and rewriting code would change the transformation logic, which the question explicitly prohibits. Option D is wrong because switching from Spark to Python shell would remove distributed processing entirely, making the job unable to handle 2 TB of data efficiently and likely causing it to fail or run far longer.

314
MCQeasy

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3. The data volume is about 500 GB initially, with daily incremental updates of 10 GB. The pipeline must minimize operational overhead. Which AWS service should be used for the initial and incremental loads?

A.AWS Database Migration Service (DMS) with change data capture (CDC) to Amazon S3.
B.AWS Glue with a JDBC connection and incremental crawl.
C.Amazon Kinesis Data Firehose with a custom producer.
D.AWS Data Pipeline with a SQL activity and HiveCopyActivity.
AnswerA

AWS DMS with change data capture performs the initial full load and then continuously replicates ongoing changes from Oracle to Amazon S3, so incremental updates need no custom scripting or scheduled jobs, satisfying the minimal-operational-overhead requirement.

Why this answer

AWS DMS with CDC is the correct choice because it supports continuous replication from Oracle to Amazon S3 with minimal overhead. It handles both the initial 500 GB full load and ongoing 10 GB daily increments via change data capture, without requiring custom code or complex pipeline management.

Exam trap

The trap here is that candidates often choose AWS Glue for its serverless nature, but Glue's incremental crawl only updates the Data Catalog, not the data itself, and it cannot capture row-level changes from a database without full reloads.

How to eliminate wrong answers

Option B is wrong because AWS Glue with an incremental crawl is designed for cataloging schema changes, not for capturing row-level changes from a database; it would require full table scans for each incremental load, which is inefficient for 10 GB daily updates. Option C is wrong because Amazon Kinesis Data Firehose requires a custom producer to stream data from Oracle, which adds operational overhead and does not natively support CDC or initial bulk loads from a database. Option D is wrong because AWS Data Pipeline with a SQL activity and HiveCopyActivity is a legacy service that lacks native CDC support for Oracle, requiring custom scripting for incremental loads and increasing operational complexity.

315
Multi-Selecthard

Which THREE factors should be considered when choosing between Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose for real-time data ingestion? (Choose three.)

Select 3 answers
A.The need for a fully managed delivery destination
B.Whether the application requires custom data processing logic
C.The ability to compress data before storage
D.The latency requirements for data delivery
E.The maximum throughput supported per shard
AnswersA, B, D

Firehose can directly deliver to S3, Redshift, etc., while Streams requires a consumer.

Why this answer

Amazon Kinesis Data Firehose is a fully managed service that automatically delivers streaming data to destinations like Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Splunk, making it ideal when you need a fully managed delivery destination without managing the ingestion pipeline. In contrast, Amazon Kinesis Data Streams requires you to build and manage consumers to process and deliver data, so if you need a fully managed destination, Firehose is the correct choice.

Exam trap

The trap here is that candidates often assume compression or throughput limits are unique to one service, but both services support compression and throughput is a scaling detail of Streams, not a direct comparison factor for choosing between the two.

316
MCQeasy

A company wants to ingest real-time streaming data from thousands of IoT devices into AWS for immediate processing. Which service is designed for ingesting large volumes of streaming data with low latency?

A.Amazon Simple Storage Service (S3)
B.AWS Database Migration Service (DMS)
C.Amazon Simple Queue Service (SQS)
D.Amazon Kinesis Data Streams
AnswerD

Amazon Kinesis Data Streams ingests thousands of concurrent device feeds with sub-second latency, satisfying the immediate-processing constraint. Unlike Kinesis Data Firehose, which micro-batches to destinations, Data Streams delivers records to consumers within milliseconds, so downstream analytics act on events as they arrive rather than after buffering.

Why this answer

Amazon Kinesis Data Streams is purpose-built for real-time streaming data ingestion at scale. Option A (S3) is object storage, not designed for streaming ingestion. Option B (DMS) is for database migration, not real-time data streaming.

Option C (SQS) is a message queue service that is pull-based and not optimized for high-throughput streaming with low latency. Only option D (Kinesis Data Streams) meets all requirements for ingesting large volumes of streaming data with low latency.

317
MCQhard

A company uses AWS Glue to run ETL jobs that process data from Amazon S3 and load into Amazon Redshift. The jobs have recently started failing with 'Out of Memory' errors. The data volume has increased 3x in the past month. Which is the MOST effective solution to resolve this issue without redesigning the job?

A.Use Amazon Athena instead of Glue for the transformation.
B.Increase the number of Glue workers (DPUs) for the job.
C.Rewrite the job to use Spark SQL instead of PySpark.
D.Increase the number of partitions in the input S3 data.
AnswerB

Increasing the number of Glue workers (DPUs) adds parallel executors and memory capacity, letting the existing job process the tripled data volume without code changes. Vertical scaling of worker type alone is less effective than adding workers for memory-bound shuffles.

Why this answer

Glue ETL jobs run on a Spark cluster sized by the number of DPUs (workers). Out-of-memory errors during a 3x data volume increase indicate the existing worker count cannot hold the partitions and shuffle data in memory. Increasing the number of Glue workers adds executor memory and parallelism, which is the most direct fix without redesigning the job.

Exam trap

DEA-C01 often tests the misconception that changing the query language or input partitioning fixes OOM, when the direct lever for Spark memory is the number and size of Glue workers.

How to eliminate wrong answers

Option A is wrong because switching to Athena is a redesign of the processing architecture and is not a drop-in fix for an existing Glue job. Option C is wrong because rewriting PySpark as Spark SQL does not inherently reduce memory pressure and still requires code changes. Option D is wrong because increasing S3 partitions can help parallelism but does not guarantee more memory per executor and may even increase overhead if not matched with more workers.

318
MCQmedium

A data engineer needs to load data from an on-premises Oracle database to Amazon S3 daily. The table is 500 GB and grows by 50 MB per day. The load must capture only new and changed rows since the last run. Which solution is MOST cost-effective and requires the least maintenance?

A.Write a custom Python script on EC2 to query the Oracle redo logs and upload to S3
B.Export the entire table to CSV daily using a script and upload to S3
C.Use AWS Glue ETL job with a JDBC connection and a timestamp filter
D.Use AWS Database Migration Service (DMS) with ongoing replication (CDC)
AnswerD

DMS supports CDC and can capture only changes, minimizing cost and effort.

Why this answer

AWS DMS with ongoing replication (CDC) is the most cost-effective and low-maintenance solution because it continuously captures only new and changed rows from the Oracle source using its built-in CDC mechanism (reading redo logs), without requiring custom scripting or full table exports. It automatically handles schema changes, resumability, and incremental loading to S3, minimizing operational overhead and data transfer costs.

Exam trap

The trap here is that candidates often choose AWS Glue with a timestamp filter (Option C) because it seems simpler, but they overlook that Glue still performs a full table scan via JDBC to apply the filter, which is inefficient for large tables and does not provide true CDC from redo logs, unlike DMS's native log-based replication.

How to eliminate wrong answers

Option A is wrong because writing a custom Python script on EC2 to parse Oracle redo logs is complex to implement, requires deep Oracle internals knowledge, and demands ongoing maintenance for log format changes and error handling, making it neither cost-effective nor low-maintenance. Option B is wrong because exporting the entire 500 GB table daily to CSV is extremely inefficient, wastes significant compute and network resources, and incurs high S3 storage costs for unchanged data, failing the 'only new and changed rows' requirement. Option C is wrong because while AWS Glue with a timestamp filter can capture incremental changes, it requires the source table to have a reliable, monotonically increasing timestamp column and still performs a full JDBC scan of the table to filter rows, which is inefficient for a 500 GB table and does not natively support change data capture from redo logs.

319
MCQhard

Refer to the exhibit. A CloudFormation stack outputs the Glue job name and S3 bucket names. The Glue job transforms CSV files from the raw bucket to Parquet in the processed bucket. However, the Glue job is failing with an error that it cannot write to the processed bucket. What is the most likely cause?

A.The Glue job does not have permission to write to the processed bucket
B.The raw data bucket is in a different region
C.The Glue job is not using the correct worker type
D.The Glue job is using an incorrect file format
AnswerA

CloudFormation outputs merely expose values; they grant no IAM permissions. The Glue job's execution role therefore lacks s3:PutObject on the processed bucket, producing the access-denied write failure. The fix is attaching a policy permitting writes to that bucket.

Why this answer

The most likely cause is that the Glue job's IAM role lacks the necessary permissions (e.g., s3:PutObject, s3:ListBucket) on the processed bucket. AWS Glue jobs require an IAM role with policies that grant write access to the target S3 bucket; without these permissions, the job fails with a write error. This is a common misconfiguration when the role is scoped only to read from the raw bucket.

Exam trap

The DEA-C01 exam often tests the misconception that S3 write failures are caused by region mismatches or file format issues, but the actual trap is that candidates overlook the IAM permission layer and attribute the error to non-permission factors like worker type or data format.

How to eliminate wrong answers

Option B is wrong because cross-region access to S3 buckets is fully supported and does not cause a write permission error; the error message specifically indicates a write failure, not a connectivity or region mismatch. Option C is wrong because the worker type (e.g., G.1X, G.2X) affects memory and compute capacity, not S3 write permissions; an incorrect worker type would cause performance or OOM issues, not a bucket write error. Option D is wrong because using an incorrect file format (e.g., specifying Parquet when the output is CSV) would cause a format conversion error, not a permission-denied write error; the error message explicitly states it cannot write to the bucket, pointing to an access control issue.

320
MCQmedium

A data engineer is building an AWS Glue ETL job that reads records from an Amazon Kinesis Data Stream and writes them to Amazon S3 in Parquet format. The job must checkpoint its progress so that it can resume without reprocessing data after a failure. Which AWS Glue mechanism should the engineer configure to track the stream position?

A.Use AWS Glue Studio's auto-generated script and enable the 'retry on failure' option in the job's trigger.
B.Enable job bookmarks with the transformation_ctx parameter set on the create_data_frame_from_options call.
C.Set the Glue job's MaxConcurrentRuns parameter to 1 and let the job restart from the beginning on failure.
D.Configure a Kinesis Data Stream consumer with enhanced fan-out and rely on the stream's 24-hour retention to replay records.
AnswerB

Job bookmarks persist the state of the last processed record per source. When reading from Kinesis, Glue stores the stream sequence number and shard information, so a re-run resumes from the checkpoint. Setting transformation_ctx uniquely identifies the source in the bookmark store, preventing conflicting state when a job reads multiple streams.

Why this answer

Glue job bookmarks maintain state between runs, storing the last processed Kinesis sequence number per shard. With transformation_ctx set on the read operation, the job can resume from where it stopped after a failure, avoiding duplicate writes to S3. The other options address throughput, concurrency, or retry orchestration, none of which track record-level progress.

Exam trap

The trap here is assuming that Kinesis stream retention or retry triggers provide checkpointing, when only Glue job bookmarks persist the last processed position.

321
MCQhard

A company ingests JSON data from an S3 bucket into a Glue ETL job. The data contains nested structures and arrays. The team wants to flatten the data into a tabular format for analysis in Athena. Which Glue transformation is appropriate?

A.Map
B.Relationalize
C.Filter
D.DropNullFields
AnswerB

Relationalize converts nested JSON and arrays into a set of flat, related tables, which is precisely what Athena needs for tabular SQL queries. Other transforms such as ApplyMapping only rename or retype fields and cannot unnest arrays, so they would leave the nested structures intact.

Why this answer

The Relationalize transformation is specifically designed to flatten nested JSON and arrays into a tabular format suitable for Athena. Option A (Map) applies a function to each record but does not flatten structures. Option C (Filter) selects rows based on a condition.

Option D (DropNullFields) removes null fields but does not address nested structures.

322
MCQhard

A company runs a daily batch ETL job using AWS Glue. The job processes 500 GB of data from Amazon RDS to Amazon S3. The job currently uses a single DPU and takes 6 hours to complete. The team wants to reduce runtime to under 1 hour without increasing costs significantly. Which approach should they use?

A.Change the job type from Python to Spark.
B.Use multiple Glue jobs triggered sequentially.
C.Increase the RDS instance size to improve read throughput.
D.Use AWS Glue Spark job with 100 workers.
AnswerD

More workers enable parallelism, reducing runtime.

Why this answer

AWS Glue Spark jobs can parallelize data processing across multiple workers, dramatically reducing runtime. With 100 workers, the job can process the 500 GB dataset in parallel, achieving sub-1-hour runtime while keeping costs relatively low since Glue charges per DPU-second and the total DPU-seconds may be similar to the original 6-hour single-DPU job.

Exam trap

The trap here is that candidates might think increasing parallelism (Option D) is too expensive, but Glue's pay-per-DPU-second model means a job with 100 workers running for 1 hour costs roughly the same as 1 worker running for 100 hours, so the total cost is similar, not significantly higher.

How to eliminate wrong answers

Option A is wrong because changing from Python to Spark alone does not add parallelism; the job still runs on a single DPU unless the number of workers is increased. Option B is wrong because running multiple Glue jobs sequentially would increase total runtime, not reduce it, as each job would still process data serially. Option C is wrong because the bottleneck is Glue's processing capacity, not RDS read throughput; increasing RDS instance size would not significantly reduce Glue job runtime since the job already reads 500 GB over 6 hours, and the read rate is not the limiting factor.

323
Multi-Selecthard

A company uses AWS DMS to replicate data from an Amazon RDS for MySQL database to Amazon S3. Which TWO configurations are required to enable continuous change data capture (CDC) from MySQL?

Select 2 answers
A.Ensure the S3 bucket is in the same AWS Region as the source database
B.Grant REPLICATION CLIENT and REPLICATION SLAVE privileges to the DMS user
C.Enable binary logging (binlog) on the MySQL source database
D.Enable versioning on the target S3 bucket
E.Configure the MySQL source to be Multi-AZ
AnswersB, C

DMS requires REPLICATION CLIENT to read binlog metadata and REPLICATION SLAVE to stream binlog events as a replica. Granting both to the DMS user satisfies the stem's continuous CDC requirement, since MySQL denies replication access without these specific global privileges.

Why this answer

Option B is correct because AWS DMS requires the MySQL endpoint user to have REPLICATION CLIENT (to read binlog metadata/status) and REPLICATION SLAVE (to read the binary log events) privileges so it can capture ongoing changes for CDC. Option C is correct because continuous change data capture depends on the MySQL source having binary logging enabled (log_bin=ON) with a suitable binlog_format (e.g., ROW) so DMS can read committed changes from the binlog. Option A is not required, since DMS can replicate to an S3 bucket in a different Region than the source database.

Option D is not required, as S3 bucket versioning is unrelated to DMS CDC and is not needed for change capture. Option E is not required, because Multi-AZ is a high-availability deployment choice for the source and does not enable or affect DMS change data capture.

Exam trap

DEA-C01 often tests the specific prerequisites for MySQL CDC with DMS. Candidates might think that Multi-AZ or S3 versioning is required, but the key requirements are binlog and replication privileges. It's also common to confuse REPLICATION CLIENT with other privileges like SELECT.

324
MCQmedium

A data engineer is configuring an AWS Glue crawler against an Amazon S3 path that contains CSV files with inconsistent column counts across files. The crawler keeps creating multiple tables for the same data and the engineer wants a single table with a merged schema. Which crawler configuration should the engineer change?

A.Increase the crawler's 'Maximum number of tables' limit in the crawler configuration.
B.Enable the crawler's 'Update all new and existing partitions with metadata from the table' option.
C.Set the crawler's 'Create a single schema for each S3 path' option to true.
D.Change the crawler's classifier to a custom classifier that forces all columns to string type.
AnswerC

This option, sometimes called 'CombineCompatibleSchemas', tells the crawler to merge compatible schemas that share the same S3 path into one table. When CSV files have varying column counts but compatible types, enabling this setting produces a single table with the union of columns instead of multiple tables, which is exactly what the engineer needs.

Why this answer

The 'Create a single schema for each S3 path' crawler setting merges compatible schemas found under the same S3 path into one table. For CSV files with inconsistent column counts, enabling this option produces a single table with the union of compatible columns, eliminating the multiple tables the crawler was creating.

Exam trap

The trap here is assuming that a custom classifier or partition metadata option controls table grouping, when only the single-schema-per-S3-path setting merges compatible schemas into one table.

325
MCQeasy

A company needs to transform JSON data from an S3 bucket into a structured format for Amazon Redshift. The transformation should be done serverlessly. Which service should be used?

A.AWS Glue
B.Amazon EMR
C.Amazon Athena
D.AWS Lambda
AnswerA

AWS Glue is serverless, so no cluster management is needed, and its ETL jobs transform JSON from S3 into a structured schema that loads directly into Amazon Redshift, satisfying the serverless transformation constraint in the stem.

Why this answer

AWS Glue is the correct choice because it is a fully managed, serverless ETL service designed specifically for transforming and preparing data for analytics, including converting JSON to structured formats like Parquet or ORC for Amazon Redshift. It can crawl the S3 source, infer schemas, and run Spark-based transformation jobs without provisioning any infrastructure, aligning perfectly with the serverless requirement.

Exam trap

The trap here is that candidates often confuse Amazon Athena's serverless SQL querying capability with ETL transformation, but Athena cannot transform or write data into a different format for Redshift—it only reads and queries data in place.

How to eliminate wrong answers

Option B (Amazon EMR) is wrong because it requires provisioning and managing EC2 clusters, which is not serverless; it is a managed Hadoop framework but still involves underlying infrastructure. Option C (Amazon Athena) is wrong because it is a serverless query engine for analyzing data directly in S3 using SQL, not a transformation service for converting JSON to a structured format for Redshift. Option D (AWS Lambda) is wrong because it is designed for short-running, event-driven functions (max 15-minute execution time) and is not suitable for large-scale ETL transformations on big datasets, which typically require longer-running jobs.

326
MCQhard

A data engineer maintains an AWS Glue job that incrementally processes new files in Amazon S3 using job bookmarks. After a schema change in the source data added a new column, the engineer updated the Glue Data Catalog table. Subsequent job runs still process only previously seen files and ignore newly arrived objects. The engineer verifies that new files exist in the prefix and that the bookmark state was not reset. Which factor most likely explains why new files are being skipped?

A.Amazon S3 eventual consistency delays object visibility, so bookmarks never see files written within the same hour.
B.Job bookmarks only support Amazon S3 sources when the files are in Parquet format, so JSON files are silently skipped.
C.The Glue job's IAM role lacks s3:ListBucket permission, so the job cannot detect newly arrived objects.
D.Job bookmarks use the source path and a transformation context key; changing the job script or the catalog table can invalidate the bookmark state.
AnswerD

Glue job bookmarks track processed data using a key derived from the source path, transformation context, and related job parameters. Editing the script or altering the catalog table can change that context, causing the bookmark to behave unexpectedly and skip new files. Resetting or recreating the bookmark after such changes restores correct incremental processing.

Why this answer

Job bookmarks persist state keyed to the source path, transformation context, and job parameters. Modifying the job script or the referenced Data Catalog table alters that context, which can cause the bookmark to skip newly arrived objects despite their presence. Resetting the bookmark or recreating the job restores expected incremental behavior.

Format, IAM, and S3 consistency are not the cause here.

Exam trap

The trap here is assuming bookmarks only look at file timestamps, when they actually key on transformation context that script or catalog edits can invalidate.

327
MCQmedium

A company is using Amazon Kinesis Data Streams to ingest real-time clickstream data from a website. The data is consumed by an Amazon Kinesis Data Analytics for Apache Flink application that performs real-time analytics. The Flink application writes its results to an Amazon S3 bucket. The company has noticed that the Flink application is experiencing high checkpoint failure rates, causing delays. The CloudWatch metrics show that the checkpoint size is large and increasing. The data engineer needs to reduce the checkpoint size. Which action should the data engineer take?

A.Decrease the checkpoint interval to reduce the amount of state accumulated.
B.Reduce the parallelism of the Flink application.
C.Increase the state time-to-live (TTL) configuration to retain state longer.
D.Enable incremental checkpointing in the Flink application to only write changes since the last checkpoint.
AnswerD

Incremental checkpointing writes only the state changes since the previous checkpoint rather than the full state, so checkpoint size stops growing with application state. This directly addresses the large and increasing checkpoint size causing the high failure rates.

Why this answer

Enabling incremental checkpointing in the Flink application is the correct action because it only writes the changes since the last checkpoint, significantly reducing checkpoint size. This is especially effective when state is large and growing, as it avoids rewriting the entire state with each checkpoint.

Exam trap

DEA-C01 often tests Flink checkpoint tuning. Candidates may think that reducing checkpoint interval or parallelism helps, but the trap is not knowing that incremental checkpointing is the key to reducing checkpoint size for large state.

How to eliminate wrong answers

Option A is wrong because decreasing the checkpoint interval would increase checkpoint frequency, potentially exacerbating the issue by creating more checkpoints, though each might be smaller; it does not address the root cause of large state size. Option B is wrong because reducing parallelism would decrease throughput and may not reduce checkpoint size; it could even increase per-task state. Option C is wrong because increasing state TTL would retain state longer, making the state larger and worsening the problem.

328
MCQhard

A data engineer runs an AWS Glue Studio job that reads JSON from Amazon S3 and writes to a partitioned Parquet table. The job currently runs for six hours. Profiling shows that a small number of partitions contain millions of rows while most contain a few hundred. Which change will MOST improve runtime?

A.Increase the number of DPUs allocated to the Glue job from 10 to 100.
B.Apply a salting technique to the skewed keys before the write, or repartition on a composite key.
C.Enable job bookmarks and re-run the job to skip already-processed files.
D.Switch the job's worker type from G.1X to G.2X to get more memory per executor.
AnswerB

Salting appends a random suffix to hot keys so rows spread across many tasks, and repartitioning on a composite key achieves similar balance. Both directly address the uneven task durations causing the long tail. This is the standard remedy for data skew in Spark and therefore in AWS Glue jobs.

Why this answer

Data skew, not compute capacity, is causing the long runtime because a few partitions hold most of the rows. Salting hot keys or repartitioning on a composite key spreads those rows across many tasks so no single task becomes a straggler. Increasing DPUs, changing worker type, or enabling bookmarks does not change how rows are distributed within a run.

Exam trap

The trap here is assuming that more DPUs or bigger workers will fix any slow job, when skew requires redistributing data rather than adding compute.

329
MCQmedium

A data engineer is building an AWS Glue Studio visual ETL job that reads JSON files from Amazon S3, applies a transformation, and writes to Amazon Redshift. During a test run, the job fails with an error indicating that the dynamic frame could not be written because the target table schema does not match the incoming data. The engineer needs to ensure the job automatically reconciles schema differences such as missing columns and data type mismatches during the write. Which action should the engineer take?

A.Set the job bookmarks option to 'Enable' so Glue tracks schema changes between runs and applies them to the target.
B.Change the write node's 'Handling of schema differences' to 'Reconcile schema' so missing columns are added and type mismatches are cast.
C.Enable the 'Auto schema reconciliation' option in the Glue job's Redshift connection parameters.
D.Increase the number of DPUs allocated to the Glue job so the write can complete with a larger executor pool.
AnswerB

In AWS Glue Studio, the Redshift write node exposes a 'Handling of schema differences' property. Setting it to 'Reconcile schema' lets the writer add missing columns and coerce compatible data types to match the target table. This directly addresses the failure described, since the dynamic frame will be aligned to the Redshift table schema before the COPY operation is issued.

Why this answer

The Redshift write node in AWS Glue Studio provides a 'Handling of schema differences' setting. Selecting 'Reconcile schema' lets the writer add missing columns and cast compatible types so the dynamic frame matches the target table. This is the supported mechanism for automatically aligning incoming data with an existing Redshift schema, and it resolves the described failure without manual DDL changes.

Exam trap

The trap here is assuming that Glue connection settings or job bookmarks control runtime schema behavior, when schema reconciliation is a property of the write node itself.

330
MCQhard

A data engineer is building an AWS Glue ETL job that reads from an AWS Glue Data Catalog table backed by Amazon S3. The job must process only records added since the last successful run to reduce cost and runtime. The source data is partitioned by year, month, and day. Which approach should the engineer use?

A.Use the AWS Glue Data Catalog's update time and a custom script that queries the catalog for recently changed partitions before reading S3.
B.Set the job's maximum concurrency to 1 and rely on the S3 object's last modified timestamp in the script to filter records.
C.Configure the job to use a pushdown predicate that filters on the year, month, and day partition columns using the current date.
D.Enable job bookmarks in the AWS Glue job and use the default transformation_ctx on the source DynamicFrame.
AnswerD

AWS Glue job bookmarks track the state of previously processed data. When enabled and a transformation context is set on the source, the job reads only new or changed data since the last run. For partitioned S3 sources, bookmarks use partition metadata to determine what is new, which directly satisfies the requirement to process only newly added records.

Why this answer

AWS Glue job bookmarks persist state across runs and, when a transformation context is set on the source, cause the job to read only data that has not been processed before. For partitioned S3 sources, bookmarks track new partitions and files, which precisely matches the requirement to process only records added since the last successful run while reducing cost and runtime.

Exam trap

The trap here is confusing a pushdown predicate that filters by current date with true incremental state tracking across job runs.

331
Multi-Selectmedium

A data engineer is designing an ingestion pipeline that uses AWS Glue to read from an Amazon RDS for PostgreSQL database. The job must read only rows changed since the previous run and must not scan the entire table each night. The source table has a last_updated timestamp column that is updated on every write. (Choose two.)

Select 2 answers
A.Configure the Glue JDBC connection with a pushdown predicate that filters on last_updated greater than a stored high-water mark.
B.Set the Glue job's '--enable-auto-scaling' argument so the job adds workers as the source table grows.
C.Create the DynamicFrame with the 'additional_options' parameter set to {'hashfield': 'id'} to enable hash-based change detection.
D.Enable job bookmarks on the Glue job so it tracks the last processed state of the JDBC source between runs.
E.Use the Glue ResolveChoice transformation with the 'cast:double' option on the timestamp column.
AnswersA, D

A pushdown predicate is sent to the source database as part of the SQL query, so RDS filters rows before returning them to Glue. Filtering on last_updated greater than a persisted high-water mark reads only changed rows, minimizes network transfer, and satisfies the incremental requirement without full-table scans.

Why this answer

Incremental JDBC ingestion in Glue relies on a pushdown predicate that filters on a high-water mark column like last_updated, combined with job bookmarks that persist state between runs. The predicate pushes work to RDS and reduces transfer, while bookmarks ensure the job resumes from the previous position without reprocessing. Transformations and worker-tuning options do not affect which rows are read.

Exam trap

The trap here is assuming that enabling job bookmarks alone is enough, when a pushdown predicate is also required to restrict the SQL read on the source.

332
Multi-Selectmedium

A company is using AWS Glue to run ETL jobs that transform data from S3 to Redshift. The jobs are failing intermittently with out-of-memory errors. Which THREE actions can help resolve this issue? (Choose THREE.)

Select 3 answers
A.Increase the number of DPUs allocated to the Glue job
B.Use S3 Select to filter data before reading into the Glue job
C.Use Spark's 'coalesce' function to reduce the number of partitions
D.Optimize the transformation logic to use less memory, for example by filtering early
E.Use a larger worker type, such as G.2X
AnswersA, D, E

More DPUs provide more memory and compute resources.

Why this answer

Increasing the number of DPUs allocated to the Glue job provides more memory and compute resources for the Spark executors, directly addressing out-of-memory errors by allowing larger datasets to be processed without exceeding heap limits. This is a standard scaling approach for memory-intensive ETL workloads in AWS Glue.

Exam trap

The trap here is that candidates often confuse reducing data volume (S3 Select) with increasing memory capacity, or mistakenly believe coalescing partitions always reduces memory usage, when in fact it can concentrate data and exacerbate OOM errors.

333
MCQhard

A company has a 100 TB dataset stored on-premises in a Hadoop cluster. They want to ingest this data into Amazon S3 for processing with AWS Glue. The company has a limited time window and a slow internet connection. Which strategy is MOST appropriate?

A.Use AWS Snowball Edge to physically ship the data to AWS.
B.Use AWS DataSync over the existing internet connection.
C.Use Amazon S3 Transfer Acceleration to speed up the upload.
D.Use AWS Direct Connect to establish a high-bandwidth connection.
AnswerA

Snowball Edge bypasses the slow internet connection by physically shipping 100 TB, meeting the limited time window. Network transfer of that volume over a slow link would take far too long, so physical shipment is the appropriate strategy.

Why this answer

AWS Snowball Edge is the most appropriate strategy because the dataset is 100 TB, the time window is limited, and the internet connection is slow. Snowball Edge provides a physical storage device that can be shipped to AWS, bypassing network bandwidth constraints entirely. This approach is designed for large-scale data transfers (typically over 10 TB) where network transfer would be impractical or exceed the available time window.

Exam trap

The trap here is that candidates may overestimate the effectiveness of network acceleration techniques (like Transfer Acceleration or Direct Connect) for extremely large datasets, failing to recognize that physical shipping is the only viable option when bandwidth and time are severely constrained.

How to eliminate wrong answers

Option B is wrong because AWS DataSync relies on the existing internet connection, which is slow and would take an excessively long time to transfer 100 TB, likely exceeding the limited time window. Option C is wrong because Amazon S3 Transfer Acceleration uses edge locations and optimized network paths, but it still depends on the underlying internet connection speed; a slow connection will remain a bottleneck, and it is not designed for petabyte-scale offline transfers. Option D is wrong because AWS Direct Connect requires establishing a dedicated network connection, which involves significant lead time for setup and does not solve the immediate problem of a slow internet connection; it also still transfers data over a network, which for 100 TB would be time-consuming even at high bandwidth.

334
MCQmedium

A company uses AWS Glue to process data from multiple sources. The data is stored in an Amazon S3 data lake. The company needs to transform the data using a custom Python library that is not available in the default Glue environment. What is the MOST efficient way to make this library available to the Glue jobs?

A.Manually install the library on each node in the Glue cluster by editing the bootstrap script.
B.Upload the library as a .whl file to Amazon S3 and reference it in the Glue job's --additional-python-modules parameter.
C.Create a custom Docker image with the library and use it in AWS Glue for Ray.
D.Use a shell command in the Glue job script to run 'pip install <library>' before the job runs.
AnswerB

Glue's --additional-python-modules parameter installs wheels from S3 into the job's Python environment at runtime, satisfying the need for a custom library absent from the default Glue image without building a custom connector or repackaging the job.

Why this answer

AWS Glue supports adding custom Python libraries by uploading a .whl file to Amazon S3 and referencing it via the `--additional-python-modules` job parameter. This method is the most efficient as it requires no manual node configuration, no custom Docker images, and no runtime pip installs, ensuring the library is automatically distributed to all worker nodes before the job executes.

Exam trap

The trap here is that candidates may think running 'pip install' directly in the script (Option D) is acceptable, but AWS explicitly recommends using the `--additional-python-modules` parameter for efficiency and reliability, as runtime pip installs can fail due to network timeouts or missing build dependencies.

How to eliminate wrong answers

Option A is wrong because manually editing bootstrap scripts to install the library on each node is inefficient, error-prone, and not scalable; Glue manages cluster lifecycle automatically, so manual node-level modifications are not recommended and can be lost on auto-scaling events. Option C is wrong because AWS Glue for Ray is a specific runtime for distributed Python and Ray-based workloads, not a general-purpose Glue ETL job; using a custom Docker image for Ray adds unnecessary complexity and is not the standard approach for standard Glue ETL jobs. Option D is wrong because running 'pip install' inside the Glue job script is inefficient, adds runtime overhead, may fail due to network restrictions or permissions, and is not the intended way to manage dependencies in Glue; the library must be pre-packaged and referenced via the job parameters.

335
MCQeasy

A data engineering team needs to transform CSV files stored in Amazon S3 into Parquet format using AWS Glue. The files are partitioned by date and are updated hourly. Which AWS Glue feature should be used to automatically detect the schema and partition structure?

A.AWS Glue Crawler
B.AWS Glue DataBrew
C.AWS Lake Formation
D.Amazon Athena
AnswerA

AWS Glue Crawler scans the S3 data, infers the CSV schema, and registers partition structure in the Glue Data Catalog. Scheduled hourly, it keeps metadata current as new date partitions arrive, enabling the ETL job to read and convert files to Parquet.

Why this answer

AWS Glue Crawler is the correct choice because it automatically scans data in S3, infers the schema (including data types), and detects the partition structure (e.g., date-based partitions like year/month/day) by examining the folder hierarchy. It then populates the AWS Glue Data Catalog with metadata, enabling ETL jobs to read the data without manual schema definition.

Exam trap

AWS often tests the distinction between tools that discover metadata (Crawler) versus tools that consume or transform data (Athena, DataBrew), leading candidates to pick Athena because it can query partitioned data, but it cannot automatically detect the partition structure without a pre-existing catalog.

How to eliminate wrong answers

Option B (AWS Glue DataBrew) is wrong because it is a visual data preparation tool for cleaning and normalizing data, not for automatic schema or partition detection. Option C (AWS Lake Formation) is wrong because it provides centralized security and governance for data lakes, but it does not perform schema discovery or partition detection itself. Option D (Amazon Athena) is wrong because it is a query engine that can read data from the Glue Data Catalog, but it does not automatically detect schemas or partitions; it relies on existing catalog metadata.

336
MCQhard

A data pipeline ingests JSON data from an S3 bucket using AWS Glue. The JSON files contain nested structures, and the team wants to flatten them for analysis in Amazon Athena. Which Glue transformation is most appropriate?

A.Filter
B.Join
C.Map
D.Relationalize
AnswerD

Flattens nested JSON into separate tables.

Why this answer

Relationalize is specifically designed to flatten nested JSON into relational tables. Option A (Map) applies a function to each record. Option B (Filter) removes records.

Option C (Join) combines datasets.

337
MCQeasy

A company needs to ingest streaming data from thousands of IoT devices into Amazon S3 for long-term storage and analytics. The data arrives continuously at a rate of 5 MB per second and must be stored in a compressed format to reduce storage costs. The solution should be highly available and require minimal management. Which AWS service should the company use?

A.Amazon Kinesis Data Streams with an AWS Lambda consumer that writes to Amazon S3.
B.Amazon Kinesis Data Firehose with a delivery stream to Amazon S3, enabling compression.
C.Amazon Managed Streaming for Apache Kafka (Amazon MSK) with a Kafka Connect S3 sink connector.
D.AWS Glue streaming ETL job that reads from Amazon Kinesis Data Streams and writes to Amazon S3.
AnswerB

Kinesis Data Firehose is a fully managed service that can ingest streaming data, automatically compress it using GZIP, and deliver it to S3. It requires no code, scales automatically, and provides high availability. It meets the requirements of continuous ingestion, compression, and minimal management, making it the ideal choice.

Why this answer

Amazon Kinesis Data Firehose is designed for streaming data ingestion to destinations like S3. It automatically handles scaling, buffering, compression, and encryption. With compression enabled, data is stored in GZIP format, reducing storage costs.

It requires no infrastructure management, making it the simplest and most cost-effective solution for continuous IoT data ingestion to S3.

Exam trap

The trap here is assuming that Kinesis Data Streams with Lambda is equivalent to Firehose, but Firehose provides built-in compression and delivery to S3 without custom code.

338
MCQmedium

A data engineer needs to ingest streaming data from thousands of devices sending JSON messages via HTTP POST. The data should be stored in Amazon S3 with minimal latency and also be available for real-time analytics. Which combination of services is MOST appropriate?

A.Amazon DynamoDB with DynamoDB Streams and Lambda.
B.Amazon SQS and AWS Lambda to write to S3.
C.AWS Lambda directly writing to S3 via API Gateway.
D.Amazon API Gateway, Amazon Kinesis Data Streams, and Kinesis Data Firehose.
AnswerD

API Gateway accepts the devices' HTTP POST requests, Kinesis Data Streams provides low-latency ingestion for real-time analytics consumers, and Firehose micro-batches the same stream into S3. This satisfies both the minimal-latency S3 storage and concurrent real-time analytics constraints.

Why this answer

Amazon API Gateway provides a scalable, managed HTTP endpoint that can receive JSON POST requests from thousands of devices and proxy them directly into Amazon Kinesis Data Streams. Kinesis Data Streams ingests the data with low latency and makes it available for real-time analytics via Kinesis Client Library (KCL) applications, AWS Lambda, or Kinesis Data Analytics. Kinesis Data Firehose then delivers the stream to Amazon S3 with minimal buffering (as low as 60 seconds), satisfying both the real-time and storage requirements.

Exam trap

DEA-C01 often tests the misconception that SQS or DynamoDB Streams can replace Kinesis for real-time streaming ingestion, confusing queue-based decoupling with stream-based replayable analytics.

How to eliminate wrong answers

Option A is wrong because DynamoDB Streams captures item-level changes from a DynamoDB table, not raw HTTP POST payloads from devices; it also does not natively deliver to S3 and is not designed for high-volume streaming ingestion. Option B is wrong because SQS is a queue, not a streaming service—it lacks the ordered, replayable, low-latency stream semantics needed for real-time analytics, and Lambda polling SQS adds latency and complexity. Option C is wrong because Lambda writing directly to S3 via API Gateway bypasses a durable, replayable stream, offers no built-in buffering or real-time analytics integration, and can hit Lambda concurrency limits under thousands of concurrent device connections.

339
Multi-Selecthard

A company uses AWS Glue to transform data stored in S3. The Glue job runs daily and processes data in the range of hundreds of GB. The data engineer wants to optimize the job for cost and performance. Which THREE actions should be taken? (Choose THREE.)

Select 3 answers
A.Store intermediate data in HDFS on Amazon EMR
B.Increase the number of DPUs for the job
C.Reduce the number of DPUs to save cost
D.Use columnar data formats such as Parquet
E.Partition the data by date or other high-cardinality columns
AnswersB, D, E

More DPUs can reduce runtime, improving cost if job runs shorter.

Why this answer

Increasing the number of DPUs (Data Processing Units) for an AWS Glue job can improve performance by enabling parallel processing of large datasets (hundreds of GB). AWS Glue allocates resources in increments of DPUs, where each DPU provides 4 vCPU and 16 GB of memory; scaling out DPUs reduces execution time, which can lower overall cost if the job runs fewer minutes, balancing cost and performance.

Exam trap

The trap here is that candidates mistakenly think reducing DPUs always saves cost, but AWS Glue bills by DPU-hour, so longer runtimes from fewer DPUs can actually increase cost, and the question explicitly asks for both cost and performance optimization.

340
Multi-Selectmedium

Which TWO options are valid methods to ingest on-premises relational database data into Amazon S3 for analytics? (Choose 2.)

Select 2 answers
A.AWS Snowball Edge
B.AWS Glue ETL job with JDBC connection to source
C.Amazon Kinesis Data Streams with Direct Put
D.AWS Database Migration Service (DMS) with S3 target
E.Amazon AppFlow
AnswersB, D

Glue can read from JDBC and write to S3.

Why this answer

AWS Glue ETL jobs can connect to on-premises relational databases via JDBC, extract data, and write it directly to Amazon S3 in formats like Parquet or ORC. This is a fully managed, serverless approach suitable for batch ingestion and transformation of structured data for analytics.

Exam trap

The trap here is that candidates confuse AWS Glue ETL (which uses JDBC for batch extraction) with Amazon Kinesis (which is for streaming), or they overlook that AWS DMS is a dedicated service for database migration and replication to S3, while Snowball Edge is for offline bulk transfer, not live ingestion.

341
MCQmedium

The exhibit shows an AWS CLI command and its output. A data engineer wants to copy only objects larger than 10 MB from the S3 bucket to another bucket for processing. Which approach should be used to automate this task?

A.Use S3 replication rules to replicate objects above 10 MB
B.Use AWS CLI with a script to filter and copy objects
C.Use S3 Inventory to generate a list and then copy
D.Use AWS Lambda with S3 event notifications
AnswerB

The CLI can filter by size and copy objects using a script.

Why this answer

The command lists objects larger than 10 MB. To automate copying, a script using AWS CLI with the --query parameter can filter and copy. Using S3 Batch Operations allows performing actions on a list of objects.

The correct approach is to use AWS CLI with a script that iterates over the filtered list and uses aws s3 cp. S3 replication is for continuous sync, not one-time copy. Lambda with S3 events triggers only on new objects, not existing ones.

S3 Inventory provides metadata but not direct copy.

342
MCQeasy

A data engineer needs to ingest streaming data from thousands of IoT devices into AWS for real-time processing. The data volume peaks at 5 GB/min. Which AWS service should be used as the ingestion endpoint?

A.Amazon Kinesis Data Streams
B.AWS Glue
C.Amazon S3
D.AWS Lambda
AnswerA

Kinesis Data Streams ingests high-throughput streaming records with per-shard capacity that scales to thousands of producers, and delivers sub-second latency for real-time processing. It satisfies the 5 GB/min peak by adding shards, unlike S3 transfer or batch-oriented endpoints.

Why this answer

Amazon Kinesis Data Streams is designed for real-time data ingestion at scale, supporting throughput of up to 1 MB/s or 1,000 records/s per shard. With a peak of 5 GB/min (~83 MB/s), you can horizontally scale by adding shards to meet the required throughput, making it the ideal ingestion endpoint for high-volume streaming IoT data.

Exam trap

The trap here is that candidates often confuse AWS Glue's streaming ETL capability (which reads from a stream but does not ingest) with a direct ingestion endpoint, or they assume S3's high durability makes it suitable for real-time ingestion, ignoring its lack of streaming semantics and low-latency write guarantees.

How to eliminate wrong answers

Option B (AWS Glue) is wrong because it is a serverless ETL service for batch data processing and cataloging, not a real-time streaming ingestion endpoint; it cannot handle continuous, high-velocity data streams. Option C (Amazon S3) is wrong because it is an object storage service that does not provide real-time ingestion or streaming capabilities; data must be written via API calls or batch uploads, and it lacks the low-latency, ordered replay features needed for streaming. Option D (AWS Lambda) is wrong because it is a compute service for running code in response to events, not a dedicated ingestion endpoint; it has a maximum invocation payload limit of 256 KB and is not designed to buffer or scale for sustained 5 GB/min throughput.

343
Multi-Selectmedium

A data engineer is configuring an AWS Glue ETL job that reads from an Amazon S3 bucket containing nested JSON and writes to a partitioned Parquet table. The engineer wants to reduce job cost and improve read performance. (Choose two.)

Select 2 answers
A.Enable job bookmarks to avoid reprocessing previously processed files.
B.Convert the source JSON to Parquet before the Glue job runs.
C.Increase the number of workers to the maximum allowed for the job.
D.Use the Glue crawler to infer the schema and store it in the Data Catalog.
E.Partition the output Parquet table by low-cardinality columns used in filters.
AnswersA, E

Job bookmarks track which S3 objects have already been processed and skip them on subsequent runs. For incremental data landing in S3, this avoids re-reading and re-transforming old files, directly reducing DPU hours and cost. It is a supported Glue feature and a common cost-control measure for recurring jobs.

Why this answer

Job bookmarks cut cost by skipping already-processed S3 objects, which is important for incremental pipelines. Partitioning the output Parquet table by low-cardinality filter columns enables partition pruning, reducing I/O and improving read performance for downstream queries. Crawlers, worker scaling, and pre-conversion do not directly deliver both cost reduction and read performance for this job configuration.

Exam trap

The trap here is assuming that adding workers or running a crawler reduces cost, when bookmarks and output partitioning are the levers that actually cut DPU time and scan volume.

344
Multi-Selectmedium

A company is building a data lake on Amazon S3 and needs to ingest data from multiple sources. Which of the following AWS services can be used to ingest and transform data in near real-time? (Select TWO.)

Select 2 answers
A.AWS Glue
B.Amazon Kinesis Data Firehose
C.Amazon Athena
D.AWS Step Functions
E.Amazon Simple Queue Service (SQS)
AnswersA, B

Can be used for ETL jobs triggered by S3 events.

Why this answer

AWS Glue is correct because it provides a serverless ETL (Extract, Transform, Load) service that can ingest data from various sources and transform it in near real-time using its streaming ETL capabilities. Glue can consume data from Amazon Kinesis Data Streams or Apache Kafka, apply transformations using Apache Spark, and write the results to Amazon S3 or other destinations, making it suitable for near real-time data ingestion and transformation.

Exam trap

The trap here is that candidates often confuse Amazon Athena (a query engine) with an ingestion service, or assume SQS alone can perform transformations, when neither service is designed for near real-time data ingestion and transformation into a data lake.

345
MCQeasy

A data engineer needs to run a transformation on streaming data using SQL-like queries without managing servers, and the output must be written to Amazon S3 in near real time. The source is an Amazon Kinesis Data Stream. Which AWS service is the MOST appropriate to perform the transformation?

A.AWS Glue with a Python shell job triggered hourly by Amazon EventBridge.
B.AWS Lambda with a function triggered by Kinesis records and writing directly to S3.
C.Amazon Managed Service for Apache Flink with a SQL application.
D.Amazon EMR running Apache Spark Structured Streaming on a persistent cluster.
AnswerC

Amazon Managed Service for Apache Flink supports SQL-based stream processing on Kinesis Data Streams and can write results to Amazon S3, while the service manages the underlying infrastructure. It is designed for continuous, near-real-time transformations and removes the burden of provisioning and scaling servers, which matches the SQL-like, serverless-management requirement and the S3 output destination.

Why this answer

Amazon Managed Service for Apache Flink provides a managed environment for running SQL-based stream processing applications against Kinesis Data Streams and can deliver results to Amazon S3 in near real time. Because AWS operates the underlying infrastructure, the engineer avoids cluster management while still getting continuous, low-latency transformation, which is precisely what the scenario requires.

Exam trap

The trap here is treating AWS Lambda or a scheduled Glue job as equivalent to a managed streaming SQL engine, when only Flink provides continuous SQL processing without server management.

346
MCQeasy

A company wants to ingest streaming data from Apache Kafka into Amazon S3 for long-term storage and analytics. The data is in JSON format and must be delivered to S3 with minimal effort and no custom code. Which AWS service should the data engineer use?

A.AWS Glue streaming ETL job.
B.Amazon Kinesis Data Streams with an AWS Lambda consumer.
C.Amazon Managed Service for Apache Flink.
D.Amazon Kinesis Data Firehose with a Kafka source.
AnswerD

Kinesis Data Firehose can ingest data directly from Apache Kafka (via the Firehose HTTP endpoint or the Kafka connector) and deliver it to Amazon S3 without any custom code. It automatically handles scaling, buffering, and delivery, making it the most appropriate choice for minimal effort and no code.

Why this answer

Kinesis Data Firehose is a fully managed service that can read from Apache Kafka and deliver to Amazon S3 without requiring custom code. It handles scaling, buffering, and format conversion. Other options like Kinesis Data Streams, Glue, or Managed Flink require writing and maintaining code, which does not meet the no-code requirement.

Exam trap

The trap here is confusing Kinesis Data Streams with Kinesis Data Firehose; Data Streams requires custom consumers, while Firehose provides managed delivery.

347
MCQhard

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The data is then processed by a Kinesis Data Analytics application running SQL queries. The analytics application is falling behind and processing records with increasing latency. The stream has 4 shards, and the average record size is 5 KB. What is the MOST effective way to improve processing latency?

A.Increase the number of shards in the Kinesis stream to 8.
B.Enable enhanced fan-out on the Kinesis stream for the analytics application.
C.Increase the parallelism of the Kinesis Data Analytics application.
D.Increase the retention period of the Kinesis stream to 7 days.
AnswerC

More parallelism allows the application to process more records per unit time.

Why this answer

Increasing the parallelism of the Kinesis Data Analytics application (e.g., by increasing the number of in-application streams or ParallelismPerKPU) allows it to consume from the stream faster, reducing latency. Option A is wrong because 5 KB is well below the 1 MB/s shard limit, so increasing shards is unnecessary. Option B is wrong because enhanced fan-out is for consumers that need low latency, but does not help if the application is CPU-bound.

Option D is wrong because increasing the retention period does not affect processing speed; it only keeps data longer.

348
MCQeasy

A data engineer needs to ingest data from an Amazon Kinesis Data Stream into an Amazon S3 bucket for long-term storage. The data is in JSON format and must be delivered within 60 seconds of arrival. The engineer wants a fully managed solution that requires minimal code. Which service should the engineer use?

A.AWS Glue ETL job scheduled to run every minute to pull from Kinesis.
B.AWS Lambda to read from the Kinesis stream and write to Amazon S3.
C.Amazon Kinesis Client Library (KCL) application running on Amazon EC2.
D.Amazon Kinesis Data Firehose with an Amazon S3 destination.
AnswerD

Amazon Kinesis Data Firehose is a fully managed service that can ingest data from Kinesis Data Streams and deliver it to Amazon S3. It requires no code to set up and can buffer data for up to 60 seconds or based on size. It automatically handles scaling, and can convert JSON to Parquet or other formats if needed. This meets the requirements.

Why this answer

Amazon Kinesis Data Firehose is the fully managed service designed to deliver streaming data from Kinesis Data Streams to destinations like Amazon S3. It requires no code, can buffer for up to 60 seconds, and handles scaling automatically. The other options involve custom code or are not suited for near-real-time streaming ingestion.

Exam trap

The trap here is assuming that any service that can read from Kinesis and write to S3 is equally managed, when Firehose is purpose-built for this with minimal code.

349
MCQhard

A data engineer is using AWS Glue to process a large dataset where a small number of partitions contain disproportionately more rows than others, causing some executors to run much longer than others and the job to take hours. The engineer wants to redistribute the data across partitions before a join operation to improve performance. Which technique should the engineer apply?

A.Increase the number of AWS Glue DPUs allocated to the job so that more executors are available to process the skewed partitions.
B.Use coalesce to reduce the number of partitions, which merges partitions without a full shuffle and balances the data.
C.Enable the AWS Glue job bookmarks feature so that only new data is processed on subsequent runs, reducing the data volume.
D.Call repartition on the DataFrame using a join key column before the join, which triggers a full shuffle and distributes rows evenly across partitions.
AnswerD

Repartitioning on the join key before the join forces a shuffle that redistributes rows so that each partition contains a roughly equal share of data for that key. This directly addresses the skew that causes some executors to process far more rows than others. While a full shuffle has a cost, it is the standard remedy for skewed joins and can dramatically reduce overall job time when skew is severe.

Why this answer

Repartitioning on the join key before the join causes a shuffle that spreads rows evenly across partitions, which is the direct fix for skewed partitions. Increasing DPUs does not help because the largest partition remains a single bottleneck, coalesce can worsen skew by merging without redistribution, and job bookmarks only reduce data across runs. The key is to redistribute data within the run before the expensive join.

Exam trap

The trap here is believing that adding more compute capacity, such as more DPUs, automatically resolves data skew, when the bottleneck is the uneven distribution of rows rather than total capacity.

350
Multi-Selectmedium

A company is ingesting real-time clickstream data into Amazon S3 using Amazon Kinesis Data Firehose. The data is semi-structured and the company wants to transform the data into Parquet format and partition it by year, month, day, and hour. Which TWO steps should be taken to achieve this? (Choose TWO.)

Select 2 answers
A.Set up an Amazon S3 event notification to trigger an AWS Lambda function that partitions the data after delivery.
B.Enable dynamic partitioning in Kinesis Data Firehose and specify the partition keys as year, month, day, hour extracted from the data.
C.Use an AWS Glue Crawler to infer the schema and automatically partition the data in S3.
D.Create an AWS Lambda function that transforms incoming records to Parquet and attach it to the Firehose delivery stream as a data transformation.
E.Configure Kinesis Data Firehose to convert the data to Parquet format using a schema from the AWS Glue Data Catalog.
AnswersB, D

Correct. Dynamic partitioning extracts partition keys from the data and creates S3 prefixes accordingly.

Why this answer

Kinesis Data Firehose's dynamic partitioning feature allows you to specify partition keys (year, month, day, hour) extracted from the incoming data, and Firehose will automatically create the corresponding S3 prefix structure (e.g., year=2024/month=01/day=15/hour=10/) during delivery. Option D is correct because to convert semi-structured data to Parquet format, you can attach an AWS Lambda function as a data transformation to Firehose, which converts each record to Parquet before delivery to S3. Option E is incorrect because while Kinesis Data Firehose does support converting data to Parquet format using a schema from the AWS Glue Data Catalog, this approach requires a pre-defined Glue schema and is less flexible for semi-structured data.

Moreover, the question does not mention any existing Glue Data Catalog, and the Lambda transformation in option D is a more direct and customizable method for the transformation needed.

Exam trap

AWS often tests the misconception that dynamic partitioning alone handles format conversion, but in reality, dynamic partitioning only manages the S3 prefix structure, while Parquet conversion requires a separate Lambda transformation or the use of Firehose's built-in Parquet conversion with a compatible input format.

351
MCQhard

A data engineer runs an AWS Glue job that writes Parquet files to Amazon S3. The job frequently fails with an error indicating too many small files are being written, causing slow downstream Athena queries. The engineer wants to reduce the number of output files without changing the transformation logic. Which action should the engineer take?

A.Set the Glue job's output to use coalesce or repartition before writing.
B.Enable bookmarks on the Glue job to track processed files.
C.Change the output format from Parquet to CSV to reduce file count.
D.Increase the number of DPUs allocated to the Glue job.
AnswerA

Using coalesce or repartition in the Spark code before the write step controls the number of output partitions, directly reducing small files. This preserves transformation logic while consolidating output, which is the standard remedy for too many small Parquet files in Glue jobs.

Why this answer

Controlling output partitions with coalesce or repartition before writing is the direct way to reduce the number of small Parquet files in an AWS Glue job, improving downstream Athena performance without altering transformation logic.

Exam trap

The trap here is thinking more DPUs or bookmarks fix small files, when only output partition control (coalesce/repartition) changes file count.

352
Multi-Selecthard

Which THREE factors should a data engineer consider when choosing between AWS Glue and Amazon EMR for a data transformation job? (Choose three.)

Select 3 answers
A.The ability to output results to Amazon S3
B.The support for Apache Spark
C.The level of control over the execution environment and dependencies
D.The need for a serverless vs. cluster-based environment
E.The cost model: pay per DPU for Glue vs. per instance for EMR
AnswersC, D, E

Glue is serverless and abstracts infrastructure, while EMR gives cluster-level control over instance types, libraries and dependencies. Needing custom execution environments or specific package versions therefore favours EMR, making control level a genuine selection factor.

Why this answer

When choosing between AWS Glue and Amazon EMR for a data transformation job, key considerations include: the level of control over the execution environment (EMR offers more customization, while Glue is managed), the deployment model (Glue is serverless, EMR is cluster-based), and the cost structure (Glue charges per DPU, EMR charges per EC2 instance). Options A and B are not differentiating factors because both services support Apache Spark and can output to S3.

353
MCQeasy

A data engineer needs to run a transformation in AWS Glue where each record must be processed independently and the output schema is known ahead of time. The transformation should operate on a DynamicFrame and return a DynamicFrame. Which Glue transform is designed for this row-by-row operation?

A.DropNullFields
B.Relationalize
C.Map
D.ResolveChoice
AnswerC

The Map transform applies a user-supplied function to every record in a DynamicFrame and returns a new DynamicFrame with the transformed records. It preserves the one-to-one record relationship, making it ideal when each record is processed independently and the output schema is predetermined. It is the canonical Glue transform for this pattern.

Why this answer

The Map transform is purpose-built for applying a function to each record and returning a DynamicFrame, which matches the need for independent row processing with a known output schema. Relationalize, ResolveChoice, and DropNullFields are schema or structure utilities, not general row-level mapping transforms.

Exam trap

The trap here is confusing schema-normalization transforms such as ResolveChoice or DropNullFields with the Map transform, which is the only one that applies custom per-record logic.

354
MCQhard

A data engineer is using AWS Glue Studio to create a job that joins data from two Amazon S3 sources: a large fact table and a small dimension table. The job performs a join and then writes the result to Amazon S3 in Parquet format. The engineer notices that the job is running slowly and consuming many DPUs. Which optimization technique should the engineer apply to improve performance?

A.Use a broadcast join to replicate the small dimension table to all worker nodes.
B.Partition the fact table by join key before the join.
C.Convert the dimension table to CSV format to reduce file size.
D.Increase the number of DPUs to the maximum to speed up the join.
AnswerA

A broadcast join is an optimization technique where the smaller table is replicated to all worker nodes, allowing the join to be performed locally without shuffling the large fact table. This reduces network overhead and improves performance for joins where one table is significantly smaller. AWS Glue supports broadcast joins when one side of the join is small enough to fit in memory.

Why this answer

A broadcast join is the appropriate optimization when joining a large fact table with a small dimension table. It avoids shuffling the large table by replicating the small table to all nodes, reducing network traffic and improving speed. The other options either do not address the join inefficiency, may increase cost, or degrade performance by using a less efficient format.

Exam trap

The trap here is assuming that adding more DPUs is always the best way to improve performance, without considering algorithmic optimizations like broadcast joins.

355
MCQhard

A company uses AWS DMS to migrate a 2 TB Oracle database to Amazon RDS for PostgreSQL. The migration is taking longer than expected. The task status shows 'Full load in progress' with a low 'Table throughput (rows/s)'. Which action would MOST improve throughput?

A.Enable Multi-AZ on the DMS replication instance
B.Change the target table preparation mode to 'Do nothing'
C.Increase the number of parallel tasks in the DMS task settings
D.Increase the number of shards in the source database
AnswerC

Parallel tasks allow concurrent loading of tables, increasing throughput.

Why this answer

The low 'Table throughput (rows/s)' during the full load phase indicates that the DMS task is not processing tables with enough parallelism. Increasing the number of parallel tasks in the DMS task settings allows the replication instance to load multiple tables concurrently, which directly improves throughput by utilizing available CPU and memory resources more efficiently.

Exam trap

The trap here is that candidates confuse 'parallel tasks' with 'Multi-AZ' or 'target table preparation mode', assuming that high availability or skipping table preparation will speed up data transfer, when in fact only increasing parallelism directly addresses low row throughput during full load.

How to eliminate wrong answers

Option A is wrong because enabling Multi-AZ on the DMS replication instance provides high availability and failover support, but does not increase throughput during full load; it may even reduce performance due to synchronous replication overhead. Option B is wrong because changing the target table preparation mode to 'Do nothing' only affects how DMS handles existing tables (e.g., truncate or drop), not the speed of data transfer; it does not address low row throughput. Option D is wrong because increasing the number of shards in the source database is a source-side change that does not directly affect DMS's ability to read and load data faster; DMS's throughput is limited by its own parallelism settings, not the source shard count.

356
MCQmedium

A company uses AWS Glue ETL to transform data from Amazon RDS for MySQL to Amazon S3. The Glue job reads from a JDBC connection. The job runs once daily and processes all records, but the data volume is growing. Which change would improve performance and reduce costs?

A.Increase the number of DPUs for the Glue job
B.Switch to a Glue Python shell job
C.Use a higher JDBC fetch size
D.Enable Glue job bookmarking and set the job to process only new data
AnswerD

Job bookmarks persist state from prior runs, so the JDBC source reads only rows added or changed since the last successful run rather than the full table. This cuts JDBC read volume, shuffle and S3 write costs, satisfying the growing-data-volume constraint while keeping the daily schedule.

Why this answer

Enabling Glue job bookmarking allows the job to process only new or changed data since the last run, rather than reprocessing the entire dataset. This reduces both the data volume read from the JDBC source and the transformation time, directly improving performance and lowering costs by minimizing DPU usage.

Exam trap

The trap here is that candidates often assume increasing parallelism (Option A) is the universal fix for performance, overlooking the fact that reducing the data volume processed (Option D) is a more fundamental and cost-effective optimization.

How to eliminate wrong answers

Option A is wrong because increasing the number of DPUs for the Glue job would increase parallelism and potentially speed up execution, but it does not address the root cause of reprocessing all records daily; it would only scale the cost linearly without reducing the data volume processed. Option B is wrong because a Glue Python shell job is designed for lightweight, single-node Python scripts and cannot handle JDBC connections or large-scale data transformations; it lacks the distributed processing capabilities of a full Glue ETL job. Option C is wrong because using a higher JDBC fetch size can improve the efficiency of reading rows from MySQL by reducing round trips, but it still processes all records every run and does not eliminate the overhead of scanning the entire table daily.

357
MCQeasy

A data engineer needs to run a transformation on a large dataset stored in Amazon S3 using AWS Glue Studio. The transformation is a simple column rename and filter that can be expressed visually. The engineer wants to minimize development time and avoid writing PySpark code. Which approach should the engineer use?

A.Use AWS Glue DataBrew to create a recipe that renames columns and filters rows, then run a DataBrew job.
B.Use an AWS Glue crawler to infer the schema, then query the data with Amazon Athena and save the results to a new S3 location.
C.Develop an AWS Glue ETL script in PySpark using the DynamicFrame API, then run it as a job with the 'glueetl' job type.
D.Create a Glue Studio visual job, add a source node for the S3 bucket, add a Transform node for the rename and filter, and add a target node for the output location.
AnswerD

Glue Studio provides a visual job editor with drag-and-drop nodes for sources, transforms, and targets. A visual job can implement column renames and filters without writing code, and Glue generates the underlying PySpark script automatically. This directly meets the requirement to minimize development time and avoid manual PySpark coding, while still running on the Glue Spark engine for scale.

Why this answer

Glue Studio's visual job editor lets engineers build ETL pipelines by connecting source, transform, and target nodes without writing code. For a simple column rename and filter, this is the fastest path because Glue generates the Spark code and manages execution. The other options either require manual coding, use a different tool not integrated with Glue Studio jobs, or use query-based approaches that do not provide the same visual job experience.

Exam trap

The trap here is assuming that any visual data preparation tool, such as DataBrew, is interchangeable with Glue Studio's visual job editor for building ETL jobs.

358
MCQeasy

A company needs to ingest real-time clickstream data from a web application into Amazon S3 for analytics. The data must be available within minutes of generation. Which AWS service should be used to capture and deliver this streaming data?

A.Amazon RDS
B.Amazon Kinesis Data Firehose
C.AWS Glue
D.Amazon Simple Queue Service (SQS)
AnswerB

Kinesis Data Firehose is a fully managed delivery stream that ingests streaming records and buffers them to S3 on a configurable interval, achieving near-real-time availability within minutes without consumer code. It matches the minutes-latency requirement for clickstream ingestion.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to capture, transform, and load streaming data into Amazon S3, Redshift, Elasticsearch, or Splunk in near real-time (typically within 60 seconds). It directly addresses the requirement for ingesting real-time clickstream data and delivering it to S3 within minutes, without requiring custom code or manual scaling.

Exam trap

The trap here is confusing Amazon Kinesis Data Streams (which requires custom consumers and is not directly integrated with S3) with Amazon Kinesis Data Firehose (which is purpose-built for automated delivery to destinations like S3), leading candidates to overlook the 'within minutes' requirement and choose a service that needs additional components.

How to eliminate wrong answers

Option A (Amazon RDS) is wrong because it is a relational database service for transactional workloads, not designed for streaming data ingestion or direct delivery to S3; it would require additional ETL processes to move data to S3. Option C (AWS Glue) is wrong because it is a serverless ETL service for batch data processing and cataloging, not for real-time streaming capture; it can process data from S3 but does not ingest streaming data directly. Option D (Amazon Simple Queue Service (SQS)) is wrong because it is a message queue service for decoupling application components, not a streaming data delivery service; it does not automatically write data to S3 and requires custom consumers to do so.

359
MCQmedium

A data engineer runs an AWS Glue ETL job that reads semi-structured JSON from Amazon S3, flattens nested arrays, and writes Parquet to a partitioned S3 location. Job runs are becoming expensive because Glue reprocesses all historical partitions on every run. The engineer wants subsequent runs to process only newly arrived data. Which approach should the engineer take with the LEAST operational overhead?

A.Schedule the job with a shorter interval so each run has less data to process.
B.Move the source data to an Amazon RDS table and use a Glue JDBC connection with a watermark column.
C.Create an AWS Glue job bookmark and enable it on the S3 data source, then let the job read only new objects on each run.
D.Add a WHERE clause to the DynamicFrame that filters records by the current date at runtime.
AnswerC

Job bookmarks persist state from prior runs and let the Glue job identify newly added S3 objects, so only incremental data is processed. This is the native, low-overhead mechanism for incremental processing in Glue ETL and directly solves the reprocessing cost problem without custom tracking.

Why this answer

Glue job bookmarks maintain run state and cause the reader to process only new or changed S3 objects since the last successful run. This eliminates repeated scanning of historical partitions with no custom code, unlike runtime filters or schedule changes, which still read all objects. Moving data to RDS is a much heavier architectural change than needed.

Exam trap

The trap here is assuming a date filter in the transform prevents historical data from being read, when filtering happens after the source objects are already listed and scanned.

360
MCQhard

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams. The stream has 10 shards, and the data volume is expected to grow by 50% over the next month. The engineer needs to ensure that the pipeline can scale without manual intervention. Which approach should be used?

A.Set up a CloudWatch Alarm to trigger a Lambda function to add shards
B.Use an Auto Scaling group to add more shards
C.Switch the Kinesis stream to on-demand capacity mode
D.Configure the stream to use a Lambda function that scales shards
AnswerC

On-demand capacity mode automatically scales shard throughput in response to traffic, removing the need to manually reshard as volume grows by 50%. It satisfies the no-manual-intervention constraint directly, unlike provisioned mode, which requires monitoring and explicit shard splits or merges to match demand.

Why this answer

Kinesis Data Streams on-demand capacity mode automatically scales shard capacity up or down based on observed throughput, eliminating the need to manually provision or add shards. It handles the 50% growth without any CloudWatch alarms, Lambda functions, or Auto Scaling groups. This is the only option that provides fully managed, hands-off scaling.

Exam trap

The trap here is confusing EC2 Auto Scaling concepts with Kinesis shard scaling, or assuming Lambda can programmatically add shards; the exam tests whether you know that only on-demand mode provides native automatic scaling for Kinesis Data Streams.

How to eliminate wrong answers

Option A is wrong because CloudWatch alarms plus Lambda shard addition is a custom, manual-provisioning workaround that requires you to manage scaling logic and shard counts yourself, not native auto-scaling. Option B is wrong because Auto Scaling groups apply to EC2 instances, not Kinesis shards; Kinesis does not integrate with ASGs. Option D is wrong because a Lambda function cannot natively scale Kinesis shards; shard scaling is controlled by the service (on-demand) or by explicit UpdateShardCount API calls, not by Lambda.

361
MCQeasy

A company wants to migrate on-premises data to Amazon S3 using AWS DataSync. The data is stored on an NFS file server and the total volume is 50 TB. The network bandwidth between the on-premises data center and AWS is 1 Gbps (gigabit per second). What is the primary factor that will determine the total time required for the initial data transfer?

A.The available network bandwidth between on-premises and AWS
B.The number of S3 buckets used as the destination
C.The average file size in the dataset
D.The IOPS (I/O operations per second) of the on-premises NFS server
AnswerA

Available network bandwidth is the limiting factor: at 1 Gbps, roughly 125 MB/s theoretical throughput caps the transfer, so 50 TB needs many hours regardless of DataSync tuning. Disk read speed, S3 request rates and agent CPU rarely saturate a 1 Gbps link, making bandwidth the primary determinant of total elapsed time.

Why this answer

AWS DataSync transfers data over the network, so the primary constraint is the available bandwidth between the on-premises NFS server and AWS. With 50 TB of data and a 1 Gbps link, the theoretical minimum transfer time is approximately 50 TB * 8 / 1 Gbps = 400,000 seconds (~111 hours), but real-world throughput is lower due to protocol overhead, latency, and competing traffic. The network bandwidth directly dictates the maximum data transfer rate, making it the dominant factor for the initial transfer duration.

Exam trap

The trap here is that candidates may focus on the NFS server's IOPS or file size, assuming storage performance is the bottleneck, but the question explicitly provides a 1 Gbps bandwidth figure, signaling that network throughput is the key limiting factor for the initial transfer.

How to eliminate wrong answers

Option B is wrong because the number of S3 buckets does not affect transfer speed; DataSync can write to multiple buckets, but the throughput is still limited by the network pipe. Option C is wrong because while average file size can impact the number of file operations, DataSync uses parallel streams and can handle small files efficiently; the total data volume and bandwidth are the primary drivers, not file size. Option D is wrong because the NFS server's IOPS is rarely the bottleneck for a bulk transfer over a 1 Gbps link; DataSync reads files sequentially and the network bandwidth is typically the limiting factor, not the storage I/O performance.

362
MCQmedium

A data engineer is building an AWS Glue ETL job that reads from an Amazon DynamoDB table and writes to Amazon S3. The DynamoDB table has a large number of items, and the engineer needs to ensure the job reads the data efficiently without consuming too much provisioned throughput. Which method should the engineer use to read from DynamoDB?

A.Use Amazon Kinesis Data Streams to stream changes from DynamoDB to S3, then read from S3.
B.Use AWS Glue's native DynamoDB connector with the read throughput parameter set to a high value.
C.Use AWS Glue's DynamoDB connector with the 'dynamodb.scan' option to scan the table.
D.Use the DynamoDB export to S3 feature to export the table to S3, then read from S3 in the Glue job.
AnswerD

DynamoDB export to S3 is a serverless, efficient way to export large tables without consuming read capacity. The export creates a consistent snapshot in S3, which Glue can then read in parallel. This method avoids impacting the table's provisioned throughput and is recommended for large-scale reads.

Why this answer

DynamoDB export to S3 allows you to export a full table snapshot to S3 without consuming read capacity, making it ideal for large-scale ETL reads. The Glue job can then read the exported data from S3 in parallel, achieving high throughput without impacting the DynamoDB table's performance.

Exam trap

The trap here is assuming that using the native DynamoDB connector with high throughput is the best way, when it can actually degrade performance by consuming provisioned capacity.

363
MCQeasy

A data engineer needs to ingest log files from multiple EC2 instances into Amazon S3. The logs are written to local disk on each instance. The engineer wants a simple agent-based solution that can collect, compress, and upload logs to S3 with minimal configuration. The solution must support incremental uploads (only new log lines) and handle log rotation. What should the engineer use?

A.Install and configure Amazon CloudWatch Agent to collect logs and send them to Amazon CloudWatch Logs, then use a subscription filter to export logs to S3.
B.Use AWS CLI cp command with --recursive in a cron job to copy logs to S3 every minute.
C.Install AWS DataSync agent on each EC2 instance to sync logs to S3 daily.
D.Use an S3 sync command from the AWS CLI scheduled every hour.
AnswerA

Kinesis Agent tails log files, compresses, and sends to CloudWatch Logs; export to S3 can be automated.

Why this answer

Amazon CloudWatch Agent is a lightweight agent that can tail log files, compress them on the fly, and send them to CloudWatch Logs. From CloudWatch Logs, a subscription filter can export the logs to Amazon S3, supporting incremental uploads and log rotation. Option B (AWS CLI cp) is manual and does not handle incremental uploads efficiently.

Option C (AWS DataSync) is designed for bulk data transfers, not real-time log ingestion. Option D (S3 sync) is also not real-time and lacks agent-based tailing and compression.

364
MCQmedium

A data engineer is using AWS Glue DataBrew to profile a dataset stored in Amazon S3. The profile shows that a column named country contains values such as 'US', 'usa', 'United States', and 'U.S.A.' The engineer needs to standardize these values to a single canonical form before loading the data into Amazon Redshift. Which DataBrew transformation should the engineer apply?

A.Use the 'Remove special characters' transform to strip punctuation.
B.Use the 'Replace value or pattern' transform to replace each variant with 'United States'.
C.Use the 'Change case' transform to convert all values to uppercase.
D.Use the 'Flag duplicate values' transform to identify repeated entries.
AnswerB

The 'Replace value or pattern' transform in DataBrew lets the engineer map each observed variant to a canonical value, handling 'US', 'usa', 'United States', and 'U.S.A.' in one recipe step. It matches the profiling output directly and produces a deterministic result, which is exactly what standardization requires before loading into Redshift.

Why this answer

Standardizing a column with several representations of the same entity requires explicit value mapping, which the 'Replace value or pattern' transform provides. Case conversion, punctuation removal, and duplicate flagging each address only part of the problem or change nothing in the stored values. The replace transform is the only option that yields one canonical value for all variants.

Exam trap

The trap here is treating case normalization or punctuation cleanup as sufficient standardization when the column contains entirely different spellings of the same entity.

365
MCQeasy

A data engineer needs to ingest streaming data from an IoT fleet into Amazon S3 for near-real-time analytics. The data volume is approximately 5 GB per hour, and each event is less than 1 KB. Which AWS service should be used as the ingestion endpoint?

A.AWS IoT Core
B.AWS DataSync
C.Amazon AppFlow
D.Amazon Kinesis Data Streams
AnswerA

AWS IoT Core ingests high-volume, small-payload device telemetry and can route it directly to Amazon S3 via rules, meeting the 5 GB per hour near-real-time requirement. Its native MQTT support suits sub-1 KB events from a fleet, unlike services designed for batch or large-object transfer.

Why this answer

AWS IoT Core is purpose-built for ingesting data from IoT devices, supporting MQTT, HTTP, and WebSocket protocols. It can handle millions of devices and high-throughput, small-message payloads (each event <1 KB) and integrates directly with Amazon S3 via IoT Core rules, making it the ideal ingestion endpoint for near-real-time analytics on streaming IoT data.

Exam trap

The trap here is that candidates often default to Amazon Kinesis Data Streams for any streaming workload, overlooking that AWS IoT Core is the specialized, fully managed service designed specifically for IoT device ingestion, with native MQTT support and direct S3 integration via rules.

How to eliminate wrong answers

Option B (AWS DataSync) is wrong because it is designed for one-time or scheduled bulk data transfers between on-premises storage and AWS, not for continuous, near-real-time streaming ingestion from IoT devices. Option C (Amazon AppFlow) is wrong because it is a fully managed integration service for transferring data between SaaS applications (e.g., Salesforce, Slack) and AWS, not for ingesting IoT device telemetry streams. Option D (Amazon Kinesis Data Streams) is wrong because while it can ingest streaming data, it is a generic stream processing service that requires additional configuration (e.g., Kinesis Data Firehose) to write to S3, and it is not the dedicated IoT ingestion endpoint; AWS IoT Core is the recommended first-hop for IoT data.

366
MCQeasy

A data engineer needs to ingest data from an Amazon Kinesis Data Stream into an Amazon S3 bucket. The data must be delivered in near real time, and the engineer wants to minimize operational overhead by using a fully managed service that can also transform the data format from JSON to Parquet. Which AWS service should be used?

A.AWS Glue ETL job
B.Amazon Kinesis Data Firehose
C.Amazon Kinesis Client Library (KCL) application
D.AWS Database Migration Service (AWS DMS)
AnswerB

Amazon Kinesis Data Firehose is a fully managed service that can read from Kinesis Data Streams, transform data using AWS Lambda or built-in format conversion, and deliver to Amazon S3. It supports converting JSON to Parquet using the built-in record format conversion feature, which requires a AWS Glue table schema. This minimizes operational overhead and provides near real-time delivery.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is fully managed, can ingest from Kinesis Data Streams, and supports built-in conversion of JSON to Parquet before delivering to Amazon S3. It requires minimal operational effort and provides near real-time delivery. The other options either require custom code, are batch-oriented, or do not support the required source and transformations.

Exam trap

The trap here is confusing Kinesis Data Firehose with Kinesis Data Streams or Kinesis Client Library, or assuming AWS Glue can handle real-time ingestion with format conversion out of the box.

367
MCQeasy

A company uses AWS Database Migration Service (DMS) to continuously replicate data from an on-premises Oracle database to Amazon S3. The data is stored as CSV files. The downstream team requires the data to be in Apache Parquet format. Which change should the data engineer make to the DMS task?

A.Modify the DMS task to use Apache Parquet as the target table preparation mode.
B.Add an S3 lifecycle rule to convert CSV to Parquet.
C.Change the DMS task to use full load instead of continuous replication.
D.Configure a Lambda function to transform data after DMS writes to S3.
AnswerA

DMS can write directly in Parquet format.

Why this answer

AWS DMS supports specifying Apache Parquet as the target data format for Amazon S3 targets directly within the task configuration. By setting the 'Data format' to 'Parquet' in the S3 target endpoint or task settings, DMS automatically converts the replicated data into Parquet files, eliminating the need for post-processing. This is the most efficient and native approach to meet the downstream team's requirement.

Exam trap

The trap here is that candidates may assume DMS only supports CSV for S3 targets, overlooking the built-in Parquet option, and instead choose a complex workaround like Lambda or lifecycle rules.

How to eliminate wrong answers

Option B is wrong because S3 lifecycle rules can transition objects between storage classes or expire them, but they cannot convert file formats (e.g., CSV to Parquet). Option C is wrong because changing from continuous replication to full load would stop ongoing data synchronization and does not address the format conversion requirement. Option D is wrong because while a Lambda function could transform CSV to Parquet, it introduces unnecessary complexity, latency, and cost compared to the native DMS capability; DMS can directly write Parquet without additional services.

368
Multi-Selecthard

A data engineer is configuring an AWS Glue ETL job that reads from an Amazon S3 source with many small JSON files and writes to Amazon S3 in Parquet. The job runs slowly and produces many tiny output files. The engineer wants to improve throughput and reduce the number of output files without changing the source data layout. (Choose two.)

Select 2 answers
A.Call coalesce or repartition on the DataFrame before writing the Parquet output.
B.Enable the AWS Glue groupFiles option to coalesce input files during the read.
C.Increase the number of Glue DPUs allocated to the job.
D.Enable AWS Glue job bookmarks to skip previously processed files.
E.Convert the output format from Parquet to CSV to reduce file count.
AnswersA, B

Repartitioning or coalescing the DataFrame controls how many partitions exist at write time, and each partition produces one output file. Reducing partitions before the write consolidates output into fewer, larger Parquet files, which improves downstream query performance and directly addresses the many-tiny-output-files problem.

Why this answer

Small input files cause per-object overhead that slows reads, and the output file count equals the number of write-time partitions. Grouping input files with the groupFiles option consolidates reads, while repartitioning or coalescing the DataFrame before writing reduces the number of output partitions. Together these improve throughput and produce fewer, larger Parquet files without altering the source layout.

Exam trap

The trap here is assuming that adding more DPUs will consolidate output files, when output file count is determined by write-time partitions, not by worker capacity.

369
MCQeasy

A data engineer needs to ingest data from an Amazon S3 bucket into an Amazon Redshift table on a daily schedule. The data is in CSV format and the schema matches. Which service is simplest for this batch ingestion?

A.Amazon Redshift COPY command
B.AWS Glue ETL job with JDBC connection
C.AWS Data Pipeline
D.Amazon Athena CREATE TABLE AS SELECT
AnswerA

The Redshift COPY command loads CSV files directly from Amazon S3 into a table in parallel, parsing and distributing data across slices natively. It requires no cluster provisioning or transformation code, making it the simplest option for scheduled batch ingestion with matching schemas.

Why this answer

The Amazon Redshift COPY command is the simplest and most efficient method for batch loading data from Amazon S3 into Redshift when the schema matches and the data is in CSV format. It leverages Redshift's massively parallel processing (MPP) architecture to read data directly from S3, automatically handling compression, encryption, and error logging without requiring any intermediate services or custom code.

Exam trap

The trap here is that candidates may overcomplicate the solution by choosing AWS Glue or Data Pipeline, forgetting that Redshift's native COPY command is purpose-built for high-speed, parallel batch ingestion from S3 with minimal configuration.

How to eliminate wrong answers

Option B is wrong because AWS Glue ETL with JDBC connection introduces unnecessary complexity and overhead for a simple schema-matching CSV load; Glue is better suited for complex transformations or semi-structured data, not for a direct COPY operation. Option C is wrong because AWS Data Pipeline is a legacy orchestration service that requires defining pipelines, schedules, and activities, adding operational overhead compared to the single COPY command. Option D is wrong because Amazon Athena CREATE TABLE AS SELECT (CTAS) writes query results to a new table in S3, not into Redshift; it cannot directly ingest data into a Redshift table.

370
MCQmedium

A data engineer needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The on-premises network has a 1 Gbps link to AWS. The transfer must complete within 5 days. Which solution is MOST cost-effective and meets the requirements?

A.Use Amazon S3 Transfer Acceleration to speed up the transfer over the internet.
B.Use AWS DataSync to transfer the data over the existing network link.
C.Use AWS Snowball Edge to physically transfer the data.
D.Use AWS Direct Connect to establish a dedicated network connection.
AnswerC

AWS Snowball Edge is a physical device used for offline data transfer. It can handle large volumes faster than network transfer and is cost-effective for this scenario.

Why this answer

AWS Snowball Edge is a physical device that can transfer large amounts of data faster than a network link, especially with a 1 Gbps link that would take about 4.6 days for 50 TB (theoretical max, but actual throughput will be lower due to overhead). Snowball Edge can transfer 50 TB in a few days and is cost-effective for large data volumes. Option A (Amazon S3 Transfer Acceleration) speeds up transfers but still limited by network bandwidth.

Option B (AWS DataSync) is efficient for online transfers but may not meet the 5-day deadline over 1 Gbps. Option D (AWS Direct Connect) would require additional setup and cost, and still limited by the 1 Gbps link.

371
MCQeasy

A data engineer needs to ingest streaming data from an Amazon Kinesis Data Stream into an Amazon S3 bucket. The data must be delivered in near real-time and stored in Parquet format for efficient querying. The engineer wants to minimize custom code. Which solution should the engineer use?

A.Use Amazon Kinesis Data Firehose to deliver the stream to S3, and enable record format conversion to Parquet using an AWS Glue table.
B.Use Amazon Kinesis Data Analytics to process the stream and write Parquet to S3 using a Flink application.
C.Use AWS Lambda to read from the Kinesis Data Stream and write Parquet files to S3 using the AWS SDK for pandas (awswrangler).
D.Use AWS Glue streaming ETL job to read from the Kinesis Data Stream and write Parquet to S3.
AnswerA

Kinesis Data Firehose can read directly from a Kinesis Data Stream, buffer records, and deliver them to S3. It supports record format conversion from JSON to Parquet using a schema defined in the AWS Glue Data Catalog. This requires no custom code and provides near real-time delivery, meeting all requirements with minimal effort.

Why this answer

Kinesis Data Firehose provides a fully managed way to deliver streaming data from Kinesis Data Streams to S3. It supports automatic conversion to Parquet using a schema from the AWS Glue Data Catalog, requiring no custom code. This meets the near real-time and Parquet requirements while minimizing development effort, unlike solutions that require writing Lambda functions, Flink applications, or Glue scripts.

Exam trap

The trap here is overlooking Firehose's built-in record format conversion feature and assuming that custom code is needed to convert JSON to Parquet.

372
MCQhard

A data engineer maintains an AWS Glue ETL job that reads JSON files from Amazon S3, applies transformations using a DynamicFrame, and writes Parquet to another S3 bucket. The job currently reads the entire source prefix on every run, but the source is now partitioned by year/month/day and only new partitions need processing. The engineer wants to process only new partitions without changing the job's transformation logic. Which change should the engineer make?

A.Enable job bookmarks on the Glue job so it tracks previously processed S3 objects and skips them on subsequent runs
B.Create a Glue crawler that catalogs only the new partitions and point the job at the crawler's output table
C.Configure the job to use a smaller number of DPUs so each run scans less data
D.Add a pushdown predicate to the DynamicFrame that filters on the year, month, and day partition columns using hardcoded values
AnswerA

Glue job bookmarks persist state about which S3 objects and partitions were processed in prior runs, so subsequent runs read only new data. This requires no change to the transformation logic and directly addresses incremental processing of newly arriving partitions.

Why this answer

Job bookmarks are the native Glue mechanism for incremental processing. When enabled, Glue tracks the state of previously processed S3 objects and partitions, so each run reads only new data. This meets the requirement to process only new partitions without rewriting the transformation logic or manually specifying dates.

Exam trap

The trap here is confusing partition discovery (crawlers and catalog updates) with partition filtering at read time, when only job bookmarks give automatic incremental reads.

373
MCQeasy

A data engineer must load data from an Amazon DynamoDB table into an Amazon S3 data lake nightly. The table is approximately 800 GB and the nightly window is tight. The engineer wants a fully managed, serverless option that exports the table to S3 without consuming DynamoDB read capacity or writing custom code. Which solution meets these requirements?

A.Run an AWS Database Migration Service task with DynamoDB as the source and Amazon S3 as the target on a nightly schedule.
B.Use the native Amazon DynamoDB export to Amazon S3 feature, which exports a point-in-time snapshot without consuming read capacity.
C.Use AWS Glue with a DynamoDB connection and the dynamodb.read.throughput.percent option set to 100.
D.Enable DynamoDB Streams, attach an AWS Lambda function, and write each stream record to Amazon S3 as it arrives.
AnswerB

DynamoDB's native export to Amazon S3 produces a full point-in-time snapshot in DynamoDB JSON or Amazon Ion format directly to an S3 bucket. It is serverless, requires no code, and does not consume provisioned or on-demand read capacity, making it ideal for a large nightly export within a tight window.

Why this answer

Amazon DynamoDB's native export to Amazon S3 creates a full point-in-time snapshot of the table directly in an S3 bucket without consuming read capacity and without requiring custom code or replication infrastructure. It is serverless and scales to large tables, which fits the tight nightly window and the no-capacity-impact constraint.

Exam trap

The trap here is assuming DynamoDB Streams can produce a full historical export, when streams only capture changes that occur after the stream is enabled.

374
MCQmedium

A data engineer is building an AWS Glue ETL job that reads a large Amazon S3 dataset of nested JSON files and must flatten the nested arrays into separate rows for downstream analytics. The engineer needs the most efficient, code-free way to apply this transformation within the Glue job. Which approach should the engineer use?

A.Configure the Glue crawler to automatically flatten nested JSON during cataloging.
B.Use the AWS Glue Studio visual editor and add a Flatten transformation to the job.
C.Write a custom Python shell script in Glue to recursively flatten the JSON.
D.Use an Apache Spark filter transformation to explode the nested arrays.
AnswerB

The Flatten transformation in AWS Glue Studio un-nests nested structures such as arrays and structs into separate rows without writing custom code, which matches the requirement for a code-free, efficient transformation of nested JSON. It is designed exactly for this scenario and integrates with the visual job editor.

Why this answer

The Flatten transformation in AWS Glue Studio is purpose-built to un-nest arrays and structs into rows without custom code, directly satisfying the need for an efficient, code-free flattening step in a Glue ETL job on nested JSON.

Exam trap

The trap here is assuming a crawler or filter can reshape nested data, when only a dedicated Flatten transformation un-nests arrays into rows.

375
MCQhard

A data engineering team is troubleshooting a slow AWS Glue ETL job that reads from an Amazon DynamoDB table and writes to Amazon S3 in Parquet format. The job processes 50 GB of data. Which action would most effectively improve job performance?

A.Use S3 Select to push down filters
B.Reduce the batch size in the DynamoDB connector
C.Increase the number of DPUs
D.Change output to JSON format to reduce overhead
AnswerC

Adding DPUs scales the number of Apache Spark executors, giving more parallel workers to read the 50 GB DynamoDB table and write Parquet to S3. This directly addresses the compute-bound bottleneck, unlike tuning partition counts or connection settings.

Why this answer

Increasing the number of DPUs (Data Processing Units) for the AWS Glue job directly allocates more distributed computing resources (CPU, memory, and network bandwidth) to parallelize the read from DynamoDB and the write to S3. Since the job processes 50 GB of data, the bottleneck is likely the throughput of the Glue Spark cluster, and adding DPUs increases parallelism, reducing overall execution time.

Exam trap

The trap here is that candidates often confuse S3 Select as a universal filter mechanism or assume that reducing batch size always improves performance, when in fact it increases API overhead and latency in distributed systems like Glue.

How to eliminate wrong answers

Option A is wrong because S3 Select is used to filter data within S3 objects (e.g., CSV or JSON files) and cannot be applied to a DynamoDB source; the filter pushdown must happen at the DynamoDB API level using expressions, not S3 Select. Option B is wrong because reducing the batch size in the DynamoDB connector would decrease the number of items read per request, increasing the number of API calls and likely worsening performance due to higher latency and throttling risk. Option D is wrong because changing output to JSON format would increase file size and write overhead compared to Parquet (which is columnar and compressed), thus degrading performance, not improving it.

← PreviousPage 5 of 6 · 447 questions totalNext →

Ready to test yourself?

Try a timed practice session using only Data Ingestion Transformation questions.