Courseiva

CCNA Data Ingestion Transformation Questions

72 of 447 questions · Page 6/6 · Data Ingestion Transformation topic · Answers revealed

376
MCQeasy

A company wants to ingest streaming data from IoT devices into Amazon S3 using Amazon Kinesis Data Firehose. The data must be transformed from JSON to Parquet format before landing in S3. What is the SIMPLEST way to achieve this?

A.Configure Kinesis Data Firehose with a built-in Parquet converter.
B.Use an AWS Lambda function as a data transformation in Kinesis Data Firehose to convert JSON to Parquet.
C.Use Kinesis Data Firehose to deliver data directly to S3 in JSON format and run a nightly Glue job to convert to Parquet.
D.Use Kinesis Data Analytics to convert the data to Parquet before sending to Firehose.
AnswerA

Kinesis Data Firehose offers a native record format conversion feature that transforms incoming JSON to Parquet using a schema from the AWS Glue Data Catalog, with no custom processing required. This directly satisfies the stem's requirement to convert JSON to Parquet before landing in S3, and remains the simplest option since Firehose handles the transformation itself.

Why this answer

Amazon Kinesis Data Firehose has a built-in Parquet conversion feature that uses an AWS Glue schema to convert incoming JSON data to Parquet format. This is the simplest approach because it requires no custom code or additional services; you only need to provide a schema and enable the conversion in the Firehose delivery stream configuration. Option B (using Lambda) is more complex, as it requires writing and maintaining a custom transformation function.

Exam trap

Candidates often overlook the native Parquet conversion capability in Kinesis Data Firehose and assume a Lambda function is required. The built-in conversion using an AWS Glue schema is actually simpler and supported directly.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Firehose does not have a built-in Parquet converter; it can deliver data in Parquet format only if the input data is already in a format that can be converted (e.g., via a Lambda transformation or by using a schema from AWS Glue), but there is no native 'Parquet converter' toggle. Option C is wrong because it introduces unnecessary complexity and latency by storing JSON in S3 first and then running a nightly Glue job, which is not the simplest approach and does not meet the requirement for real-time transformation before landing. Option D is wrong because Kinesis Data Analytics is designed for real-time analytics and stream processing, not for format conversion; it would add unnecessary overhead and complexity compared to using Firehose's built-in Lambda transformation.

377
MCQmedium

A data engineer is using AWS Glue Studio to build an ETL job that reads semi-structured JSON from Amazon S3. The source files contain nested arrays and inconsistent keys, and the engineer wants the job to automatically infer the schema at runtime without a Data Catalog table. Which transform or configuration should the engineer use to read the data most reliably?

A.Create a DynamicFrame with the from_options method and set the format to "json" with the recurse parameter enabled.
B.Use a Spark DataFrame read with spark.read.schema() and a hard-coded StructType that matches all nested fields.
C.Configure the job to use the ApplyMapping transform with a manually defined mapping for every nested attribute before reading.
D.Run an AWS Glue crawler against the S3 prefix and then use the catalog table in the job with a from_catalog read.
AnswerA

DynamicFrame.from_options with format "json" and recurse=true handles nested and inconsistent JSON by flattening and inferring the schema dynamically, which suits semi-structured data without a catalog table. It is the standard programmatic approach in AWS Glue for reading JSON from S3 while accommodating schema variability at runtime.

Why this answer

Reading semi-structured JSON from S3 without a Data Catalog entry is best done with DynamicFrame.from_options, specifying the JSON format and enabling recursion so nested arrays and inconsistent keys are handled during inference. This avoids the rigidity of hard-coded schemas and the catalog dependency of crawler-based reads.

Exam trap

The trap here is assuming a Glue crawler or a hard-coded Spark schema is required to read JSON into Glue, when DynamicFrame.from_options with recurse handles runtime inference directly.

378
MCQeasy

A data engineer needs to run a one-time AWS Glue ETL job that reads data from an Amazon S3 bucket and writes transformed Parquet files to another S3 bucket. The job does not need a schedule and should be run immediately after creation. Which method should the engineer use to start the job?

A.Configure a scheduled trigger with a cron expression that runs once.
B.Create a trigger with type ON_DEMAND and activate it.
C.Create a workflow and add the job to it, then start the workflow.
D.Use the AWS Glue console or AWS CLI to start the job run directly.
AnswerD

AWS Glue jobs can be started immediately by using the StartJobRun API, the AWS CLI command start-job-run, or the console's Run job button. This is the most straightforward method for a one-time, unscheduled job run and does not require any trigger configuration.

Why this answer

For a one-time, unscheduled AWS Glue job, the simplest way to start it is to invoke StartJobRun through the console, AWS CLI, or SDK. Triggers and workflows are used for scheduling or orchestration and are not needed when the job should run immediately after creation.

Exam trap

The trap here is assuming that a trigger or workflow is always required to start an AWS Glue job, when a direct start is sufficient for on-demand runs.

379
MCQmedium

A company needs to ingest data from multiple SaaS applications (e.g., Salesforce, Marketo) into Amazon S3 for analytics. The data sources have different schemas and update frequencies. Which AWS service should be used to build this ingestion pipeline with minimal code?

A.AWS Data Pipeline
B.AWS Glue
C.Amazon Kinesis Data Firehose
D.Amazon AppFlow
AnswerD

Amazon AppFlow provides managed, low-code connectors for SaaS sources such as Salesforce and Marketo, handling differing schemas and update schedules natively. It satisfies the minimal-code requirement by removing custom ingestion scripts, delivering data directly into Amazon S3 for analytics.

Why this answer

Amazon AppFlow (Option D) is the correct answer because it is purpose-built for ingesting data from SaaS applications like Salesforce and Marketo into Amazon S3 with minimal code. It supports various source connectors, handles schema variations, and allows scheduling based on update frequencies. AWS Data Pipeline (A) requires more manual configuration and code for connectors.

AWS Glue (B) has connectors but is more complex for this use case, often requiring additional ETL scripting. Amazon Kinesis Data Firehose (C) is designed for streaming data, not batch extraction from SaaS APIs.

380
MCQeasy

A company wants to ingest data from multiple SaaS applications into Amazon S3 using a fully managed service that supports schema discovery and transformation. Which AWS service should they use?

A.Amazon Kinesis Data Firehose
B.Amazon AppFlow
C.AWS Glue
D.AWS Data Pipeline
AnswerB

Amazon AppFlow is the fully managed ingestion service that connects SaaS sources such as Salesforce and SAP to Amazon S3, performing schema discovery and optional transformation during the flow. It satisfies the no-code, managed requirement without building custom extraction pipelines.

Why this answer

Amazon AppFlow is a fully managed integration service that enables you to securely transfer data between SaaS applications and AWS services like Amazon S3. It supports schema discovery and transformation, making it ideal for ingesting data from multiple SaaS apps without writing code.

Exam trap

The trap is confusing AppFlow with AWS Glue, as both can move data, but AppFlow is specifically for SaaS integration with built-in connectors and transformations.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Firehose is designed for streaming data and does not natively connect to SaaS applications or perform schema discovery. Option C is wrong because AWS Glue is a serverless ETL service that can connect to SaaS via custom connectors, but it is not fully managed for SaaS integration and requires more setup. Option D is wrong because AWS Data Pipeline is a legacy service for orchestrating data workflows, not specifically for SaaS ingestion.

381
MCQeasy

A data engineer needs to move 50 TB of existing data from an on-premises data center into Amazon S3 as a one-time migration. The data center has a 1 Gbps internet connection that is shared with production traffic, and the migration must complete within two weeks without disrupting production. Which approach should the engineer use?

A.Use Amazon S3 Transfer Acceleration with multipart uploads from the data center.
B.Use AWS Snowball Edge Storage Optimized devices to ship the data to AWS.
C.Use AWS Transfer Family with SFTP to upload the files to Amazon S3.
D.Use AWS DataSync over the internet to copy the data directly to Amazon S3.
AnswerB

Snowball Edge Storage Optimized devices are designed for large one-time migrations where network transfer is impractical. Shipping devices avoids consuming the shared 1 Gbps link and can move 50 TB well within two weeks. The data is copied to the device on-premises and imported into S3 after AWS receives it, meeting both the timing and non-disruption requirements.

Why this answer

A 50 TB one-time migration over a shared 1 Gbps link cannot reliably finish in two weeks without harming production traffic. AWS Snowball Edge Storage Optimized devices move the data physically, sidestepping the bandwidth constraint entirely. Network-based options such as DataSync, Transfer Family, or Transfer Acceleration all depend on the same limited connection and therefore do not meet the stated requirements.

Exam trap

The trap here is focusing on transfer speed features like Transfer Acceleration while ignoring that the shared 1 Gbps link is the real bottleneck.

382
MCQhard

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The job uses the write_dynamic_frame.from_jdbc_conf method. The engineer notices that the job is slow and sometimes fails due to connection timeouts. Which action should the engineer take to improve performance and reliability?

A.Increase the number of DPUs for the Glue job to provide more parallelism.
B.Enable job bookmarks to avoid reprocessing previously loaded data.
C.Use the Amazon Redshift COPY command via a staging area in Amazon S3 instead of JDBC.
D.Increase the JDBC connection timeout and retry settings in the Glue job script.
AnswerC

The COPY command is the most efficient way to load large datasets into Redshift. It parallelizes the load and uses Redshift's massively parallel processing. Writing to S3 first and then using COPY reduces JDBC overhead and connection timeouts. This is the recommended best practice for bulk loads into Redshift from Glue.

Why this answer

For large data loads into Amazon Redshift, the COPY command is significantly faster and more reliable than JDBC. AWS Glue can write transformed data to Amazon S3 and then invoke the COPY command to load it into Redshift. This approach leverages Redshift's parallel processing, reduces connection overhead, and avoids timeouts.

Other options do not address the fundamental performance issue.

Exam trap

The trap here is assuming that increasing DPUs or tweaking timeouts will solve JDBC write performance issues, when the real solution is to use the Redshift COPY command.

383
MCQeasy

A company uses AWS Glue ETL jobs to transform data stored in Amazon S3. The job reads data in Parquet format, applies transformations, and writes the output back to S3 in Parquet format. The team wants to improve the job's performance and reduce costs. Which action is MOST effective?

A.Change the input format from Parquet to CSV to simplify parsing.
B.Coalesce the input data into a single large file before processing.
C.Use column pruning and predicate pushdown to read only necessary columns and filter data early.
D.Increase the number of workers to maximum allowed.
AnswerC

Parquet is columnar, so column pruning and predicate pushdown let Glue read only required columns and filter rows at the storage layer, cutting bytes scanned. This directly reduces both shuffle volume and cost, the stated objectives.

Why this answer

Column pruning and predicate pushdown reduce the amount of data read from S3 by Spark-based AWS Glue ETL jobs. By reading only the necessary columns and filtering rows early in the scan, I/O and memory usage decrease, directly improving performance and reducing costs.

Exam trap

The trap here is that candidates often confuse 'coalesce' (reducing partitions) with 'repartition' (increasing parallelism) and assume fewer files always improve performance, ignoring that Glue ETL benefits from parallel reads across many small files when using columnar formats.

How to eliminate wrong answers

Option A is wrong because changing from Parquet to CSV would increase data size and parsing overhead, degrading performance and increasing costs. Option B is wrong because coalescing input into a single file eliminates parallelism, causing a single executor to process all data, which increases runtime and resource contention. Option D is wrong because increasing workers to the maximum allowed without addressing data skew or I/O bottlenecks can lead to excessive cost with diminishing returns, and may hit service limits or shuffle overhead.

384
MCQmedium

A data engineer is configuring an AWS Glue ETL job that reads from an Amazon S3 bucket and writes transformed records to an Amazon Redshift cluster. The job must load data in parallel and use an Amazon Redshift IAM role for authentication. Which connection option should the engineer configure in the Glue job to enable parallel loading and IAM-based authentication?

A.Use a JDBC connection with the Redshift cluster endpoint, database name, and a database user credential stored in AWS Secrets Manager.
B.Configure an AWS Glue connection of type JDBC with the Redshift JDBC URL and set the 'redshift_tmp_dir' parameter to an S3 path, using a database password stored in AWS Glue.
C.Create an AWS Glue connection of type Amazon Redshift, specify the cluster, database, and an IAM role with permissions to access both S3 and Redshift, and enable the 'Use Redshift COPY' option.
D.Use an AWS Glue connection of type Amazon S3 and configure the Redshift cluster endpoint in the job's script using boto3.
AnswerC

An AWS Glue connection of type Amazon Redshift with an attached IAM role allows Glue to use the Redshift COPY command with temporary S3 staging, which parallelizes data loading across Redshift slices. The IAM role provides authentication without embedding database credentials. Enabling the Redshift COPY option directs Glue to stage data in S3 and issue a COPY command, which is the documented way to achieve parallel, IAM-authenticated loads from Glue to Redshift.

Why this answer

The correct configuration uses an AWS Glue connection of type Amazon Redshift with an IAM role and the Redshift COPY option enabled. This lets Glue stage data in Amazon S3 and issue a COPY command, which parallelizes loading across Redshift slices and authenticates via IAM instead of database credentials. Other connection types may provide connectivity but do not combine parallel COPY-based loading with IAM role authentication in the way Glue's native Redshift integration does.

Exam trap

The trap here is assuming that any JDBC connection to Redshift enables parallel loading and IAM authentication, when only the native Amazon Redshift connection type with the COPY option provides both.

385
MCQhard

Refer to the exhibit. A data engineer runs an AWS Glue ETL job that writes output to an S3 bucket. The job fails with the error shown. What is the most likely cause?

A.The IAM role used by the Glue job lacks the s3:PutObject permission for the output bucket
B.The Glue job attempted to write data in an unsupported format
C.The S3 bucket does not exist
D.The output file name contains invalid characters
AnswerA

Glue assumes the job's IAM role to write output, so a missing s3:PutObject permission on the target bucket produces an access-denied failure at the write stage. Granting that action to the role resolves it, satisfying the stem's constraint of the S3 write error shown in the exhibit.

Why this answer

The error shown in the exhibit indicates an access denied or permission failure when the AWS Glue ETL job attempts to write its output to the S3 bucket. The most likely cause is that the IAM role assigned to the Glue job does not include the s3:PutObject permission for the target bucket, which is required to upload objects. Without this permission, the job cannot complete the write operation, resulting in the failure.

Exam trap

The trap here is that candidates may focus on the data format or bucket existence, but the error message explicitly points to an access permission issue, which is a common misconfiguration in IAM roles for Glue jobs.

How to eliminate wrong answers

Option B is wrong because AWS Glue supports writing data in multiple formats (e.g., Parquet, ORC, JSON, CSV) and the error message does not indicate an unsupported format issue; such a problem would typically produce a different error related to format conversion. Option C is wrong because if the S3 bucket did not exist, the error would be a 'NoSuchBucket' or '404 Not Found' error, not an access denied error. Option D is wrong because while invalid characters in file names can cause errors, the error message shown is specifically about access permissions, not about invalid object key syntax.

386
MCQhard

A data engineer is using AWS Glue Studio to build a job that reads from Amazon S3, applies a filter, and writes to Amazon Redshift. The engineer needs to ensure the job can be rerun safely without creating duplicate rows in Redshift if a previous run partially succeeded. Which design choice best meets this requirement?

A.Enable AWS Glue job bookmarks and rely on them to prevent duplicate writes to Redshift on rerun.
B.Configure the Redshift write to use a pre-action and post-action that truncate the target table before each load.
C.Set the Glue job's write mode to append and add a unique constraint on the Redshift target table to reject duplicates.
D.Write the output to a staging table in Redshift and use a MERGE or upsert operation keyed on a business key to apply changes idempotently.
AnswerD

Staging plus MERGE keyed on a business key makes the load idempotent: rerunning the job re-applies the same changes without creating duplicates. This is a standard pattern for safe reruns in Redshift, and it isolates failures to the staging step before the final merge.

Why this answer

Loading into a staging table and then merging into the target on a business key is the canonical idempotent pattern for Redshift. Reruns re-execute the same merge deterministically, so partial failures do not leave duplicates. This decouples the risky load from the final apply step and supports safe retries.

Exam trap

The trap here is assuming that Glue job bookmarks or Redshift unique constraints guarantee idempotent writes, when only an explicit merge on a business key does.

387
MCQeasy

A company wants to ingest streaming data from thousands of IoT devices into AWS for real-time analytics. Which AWS service is best suited for this purpose?

A.Amazon S3
B.AWS Lambda
C.Amazon RDS
D.Amazon Kinesis Data Streams
AnswerD

Amazon Kinesis Data Streams ingests high-throughput streaming data from thousands of concurrent producers with low latency, supporting real-time analytics downstream. Its shard-based scaling handles the device volume, unlike batch-oriented services such as AWS Glue or S3-based ingestion.

Why this answer

Amazon Kinesis Data Streams is purpose-built for ingesting and processing streaming data at scale from thousands of sources. It can capture and store terabytes of data per hour from IoT devices, enabling real-time analytics with millisecond latencies. The service provides durable, ordered data streams that can be consumed by multiple applications simultaneously.

Exam trap

The trap here is that candidates often confuse batch-oriented services like S3 or compute services like Lambda with the dedicated streaming ingestion layer required for real-time data, overlooking that Kinesis Data Streams provides the necessary buffering, ordering, and replay capabilities.

How to eliminate wrong answers

Option A is wrong because Amazon S3 is an object storage service designed for static data, not for real-time streaming ingestion; it lacks the low-latency, ordered delivery and concurrent consumer support required for streaming IoT data. Option B is wrong because AWS Lambda is a serverless compute service that can process events but is not designed as a primary ingestion buffer for high-throughput streaming data; it has a maximum invocation duration of 15 minutes and cannot natively store or replay streaming data. Option C is wrong because Amazon RDS is a relational database service for transactional workloads and structured queries, not for high-velocity, unbounded streaming data ingestion; it would create bottlenecks and cannot handle the throughput and ordering requirements of thousands of IoT devices.

388
MCQeasy

A data engineer needs to ingest data from a relational database (MySQL) into Amazon S3 for analytics. The database is 500 GB and the job must run daily with incremental updates. Which AWS service is BEST suited for this task?

A.Amazon EMR with Apache Sqoop.
B.Amazon Kinesis Data Firehose with a database source.
C.AWS Database Migration Service (DMS) with a replication task.
D.AWS Glue ETL job with a JDBC connection.
AnswerC

AWS DMS with a replication task performs ongoing change data capture from MySQL to Amazon S3, handling the 500 GB initial load plus daily incremental updates. This satisfies the incremental requirement, whereas one-off tools like Snowball or Glue crawlers cannot continuously replicate ongoing database changes.

Why this answer

AWS DMS with a replication task is the best choice because it is specifically designed for continuous, incremental data replication from relational databases like MySQL to Amazon S3. DMS supports ongoing replication (change data capture) to capture incremental changes without custom scripting, and it can handle the initial 500 GB load efficiently. Other services either lack native incremental support or require additional configuration for this use case.

Exam trap

The trap here is that candidates often choose AWS Glue ETL (Option D) because it is familiar for data transformation, but they overlook that Glue lacks native incremental replication from databases, whereas DMS is purpose-built for this exact scenario with minimal overhead.

How to eliminate wrong answers

Option A is wrong because Amazon EMR with Apache Sqoop is a batch-oriented tool that does not natively support continuous incremental updates; it requires manual scripting for change data capture and is more complex to manage for daily incremental runs. Option B is wrong because Amazon Kinesis Data Firehose does not natively connect to a relational database as a source; it ingests streaming data from producers like Kinesis Data Streams or SDK, not directly from MySQL. Option D is wrong because AWS Glue ETL with a JDBC connection is designed for batch ETL jobs and does not have built-in change data capture for incremental updates; it would require custom logic to track changes, making it less suitable for daily incremental ingestion.

389
MCQhard

A data engineer runs an AWS Glue job that reads a large partitioned Parquet dataset from Amazon S3 and writes aggregated results to another S3 prefix. The job runs daily and currently reprocesses the entire dataset each time, which is becoming expensive. The engineer wants subsequent runs to process only data added since the last successful run, based on the job's state. Which feature should the engineer enable?

A.AWS Glue job bookmarks
B.AWS Glue job retry with a smaller max retry count
C.AWS Glue Data Catalog partition indexes
D.Amazon S3 Inventory reports on the source prefix
AnswerA

Job bookmarks persist state about which S3 objects or partitions have already been processed during prior runs. On subsequent runs, Glue reads only new or changed data, which reduces runtime and cost. This directly satisfies the requirement to process only data added since the last successful run, using the job's own state tracking.

Why this answer

AWS Glue job bookmarks maintain state across runs, recording which S3 objects, partitions, or JDBC rows were processed. When enabled, a subsequent run reads only new or modified data, avoiding full reprocessing. This is the built-in mechanism for incremental Glue jobs, unlike partition indexes, S3 Inventory, or retry settings, which do not track processed data across runs.

Exam trap

The trap here is confusing catalog performance features such as partition indexes with state tracking, when only job bookmarks persist which data a Glue job has already processed between runs.

390
MCQmedium

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed in real-time using custom Python code before being stored in Amazon S3. Which AWS service should be used to perform this transformation?

A.Amazon EMR with Spark Streaming
B.AWS Lambda function triggered by Kinesis Data Streams
C.Kinesis Data Analytics for Apache Flink
D.Kinesis Data Firehose with custom data transformation
AnswerC

Kinesis Data Analytics for Apache Flink is the correct choice because it allows running custom Apache Flink applications that support custom Python code via the Apache Flink Python API, enabling real-time data transformation.

Why this answer

Amazon EMR with Spark Streaming, which is optimized for large-scale batch and stream processing but is not the simplest or most direct service for this specific use case. Option B is AWS Lambda, which can be used for simple transformations but has limitations on execution time and complexity. Option D is Kinesis Data Firehose with custom data transformation, which supports only built-in transformations or Lambda functions, not arbitrary custom Python code directly.

Option C, Kinesis Data Analytics for Apache Flink, is correct because it allows running custom Apache Flink applications, which support custom Python code via the Apache Flink Python API, for real-time data transformation.

391
Multi-Selecthard

A company needs to ingest data from multiple SaaS applications (Salesforce, Marketo) into Amazon S3 for analytics. The data volume is moderate (~100 GB per day). The pipeline must handle schema changes, deduplicate records, and provide low latency (under 1 hour). Which THREE services should be used? (Choose THREE.)

Select 3 answers
A.Amazon AppFlow
B.Amazon EventBridge
C.Amazon Kinesis Data Streams
D.AWS Glue DataBrew
E.AWS Database Migration Service (DMS)
AnswersA, B, D

AppFlow can ingest data from SaaS applications like Salesforce and Marketo.

Why this answer

Amazon AppFlow is the correct choice because it is a fully managed integration service specifically designed to transfer data from SaaS applications like Salesforce and Marketo to AWS services such as Amazon S3. It supports incremental transfers, handles schema changes automatically via its schema evolution feature, and can achieve sub-hour latency for moderate data volumes (~100 GB/day) without custom coding.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Streams as a universal ingestion service, but it lacks native SaaS connectors and schema evolution handling, making it unsuitable for this specific use case compared to AppFlow.

392
Multi-Selecthard

A company is migrating a legacy on-premises ETL pipeline to AWS. The pipeline processes daily batch files from an FTP server. The data must be transformed using complex business logic before being loaded into Amazon Redshift. Which THREE AWS services should be used for this migration?

Select 3 answers
A.Amazon Athena
B.Amazon Redshift
C.AWS Glue
D.Amazon Kinesis Data Streams
E.AWS Transfer Family
AnswersB, C, E

Amazon Redshift serves as the target data warehouse, satisfying the requirement to load transformed data. Its massively parallel processing architecture handles analytical queries over the daily batch volumes, and native integration with AWS Glue and Amazon S3 enables efficient bulk loads via COPY commands after transformation completes.

Why this answer

AWS Transfer Family (E) is correct because it provides a fully managed FTP/FTPS/SFTP endpoint that can land the legacy daily batch files directly into Amazon S3, replacing the on-premises FTP server without changing the source's protocol. AWS Glue (C) is correct because it is a serverless ETL service whose Spark-based jobs and Glue Data Catalog can implement the complex business logic transformations on the batch data before loading. Amazon Redshift (B) is correct because it is the target data warehouse where the transformed data is loaded for analytics, and Glue can write to it via the Redshift JDBC/ODBC connection or the Redshift data source.

Amazon Athena (A) is not appropriate because it is an interactive query service over S3, not an ETL engine for applying complex transformation logic. Amazon Kinesis Data Streams (D) is not appropriate because it is designed for real-time streaming ingestion, whereas this pipeline processes daily batch files.

Exam trap

The trap is confusing batch ETL with streaming (Kinesis) or picking Athena as a transformation service; candidates may also forget Transfer Family for FTP ingestion and choose only Glue and Redshift.

393
MCQeasy

A data engineer needs to ingest data from multiple SaaS applications (Salesforce, Marketo) into Amazon S3 for a data lake. The data volumes are moderate and the sync needs to be scheduled daily. Which AWS service is most appropriate for this task?

A.AWS Glue
B.Amazon AppFlow
C.AWS Database Migration Service (DMS)
D.Amazon Kinesis Data Firehose
AnswerB

Amazon AppFlow provides managed connectors for SaaS sources such as Salesforce and Marketo, with scheduled flows delivering data into Amazon S3. It handles moderate volumes and daily synchronisation without custom ingestion code, matching the stated constraints.

Why this answer

Amazon AppFlow is purpose-built for securely transferring data between SaaS applications (like Salesforce and Marketo) and AWS services (like S3). It supports scheduled, incremental data syncs with built-in connectors, making it the most appropriate choice for moderate-volume daily ingestion into a data lake.

Exam trap

The trap here is that candidates often confuse AWS Glue's ETL capabilities with direct SaaS ingestion, overlooking that Glue requires a custom connector or script to pull from APIs, whereas AppFlow provides native, managed connectors.

How to eliminate wrong answers

Option A is wrong because AWS Glue is an ETL service designed for batch data transformation and cataloging, not for direct ingestion from SaaS applications; it lacks native connectors for Salesforce or Marketo. Option C is wrong because AWS DMS is intended for migrating databases (e.g., Oracle, MySQL) to AWS, not for pulling data from SaaS APIs. Option D is wrong because Amazon Kinesis Data Firehose is optimized for streaming data ingestion (e.g., from IoT or logs) and does not provide native SaaS connectors or scheduled sync capabilities.

394
Matchingmedium

Match each AWS data migration tool to its primary function.

Drag a concept onto its matching description — or click a concept then click the description.

Concepts
Matches

Migrate databases with minimal downtime

Physical device for large data transfer

Online data transfer between on-prem and AWS

Fast uploads over long distances

Combine data across sources into views

Why these pairings

AWS DMS migrates databases with minimal downtime; AWS Snowball transfers large datasets physically; AWS DataSync automates online data transfer. Common confusions include swapping tool functions or attributing schema conversion to DataSync instead of SCT.

395
MCQhard

A data engineer is building an AWS Glue ETL job that reads from an Amazon S3 bucket containing nested JSON files and must flatten the nested structures before writing to Amazon Redshift. The job uses the Glue DynamicFrame API. The engineer wants the transformation to run as a single pass without an intermediate shuffle. Which operation should the engineer use?

A.ResolveChoice
B.ApplyMapping
C.Relationalize
D.DropNullFields
AnswerC

Relationalize flattens nested DynamicFrames into a set of related flat DynamicFrames in a single pass, producing one frame per nested array or struct. It is designed exactly for unnesting JSON before loading into a relational target such as Redshift, and it avoids a full shuffle because it processes the frame as a single traversal.

Why this answer

Relationalize is the Glue DynamicFrame transform specifically designed to unnest nested JSON into multiple flat frames in one traversal. It produces a root frame and one frame per nested array, each with a join key back to the parent. This matches the requirement to flatten nested structures before loading into Redshift without an intermediate shuffle.

Exam trap

The trap here is confusing ResolveChoice, which resolves type ambiguity, with Relationalize, which unnest nested structures; only Relationalize performs the flattening required by the scenario.

396
MCQmedium

A data engineer runs an AWS Glue for Apache Spark job that writes partitioned Parquet output to Amazon S3. The engineer notices thousands of small output files in each partition, which slows downstream Amazon Athena queries. The job reads from a large S3 source and uses default partitioning. Which change should the engineer make to reduce the number of small files?

A.Set the Glue job's output to a single partition using coalesce(1) before writing.
B.Enable Glue job bookmarks and increase the number of DPUs allocated to the job.
C.Convert the output to CSV so fewer files are created per partition.
D.Use repartition or a controlled partition count before writing, or enable Glue's grouping of output files.
AnswerD

Repartitioning to a sensible number of partitions, or using Glue's built-in grouping of output files, consolidates many small files into fewer larger ones while preserving parallelism. This reduces per-file overhead and improves Athena query performance by lowering the number of objects scanned and listed. It scales with data volume, unlike forcing a single partition, and directly targets the small-file cause.

Why this answer

The number of output files equals the number of partitions written, so consolidating partitions with repartition or using Glue's output file grouping reduces small files without sacrificing parallelism. Coalescing to one partition kills scalability, bookmarks and DPUs do not affect file count, and CSV does not change partitioning while harming query efficiency.

Exam trap

The trap here is assuming that changing the output format or adding compute reduces file count, when file count is governed by the number of written partitions.

397
MCQmedium

A company is streaming IoT sensor data to Amazon Kinesis Data Streams. The data is JSON with a schema that changes occasionally. They want to load the data into Amazon S3 in Parquet format partitioned by date and sensor_id. Which approach is MOST cost-effective and operationally efficient?

A.Use Amazon EMR to read from Kinesis Data Streams and write to S3 in Parquet format.
B.Use a Lambda function to transform records to Parquet and write to S3.
C.Use a custom Kinesis Client Library application on EC2 to buffer and write Parquet files to S3.
D.Use Amazon Kinesis Data Firehose with a schema from AWS Glue Data Catalog to convert to Parquet and enable dynamic partitioning by date and sensor_id.
AnswerD

Amazon Kinesis Data Firehose can directly convert incoming JSON data to Parquet using a schema from AWS Glue Data Catalog, and supports dynamic partitioning by date and sensor_id without custom code. It is fully managed, making it the most cost-effective and operationally efficient.

Why this answer

Amazon Kinesis Data Firehose is the most cost-effective and operationally efficient solution because it natively supports converting JSON to Parquet using a schema from the AWS Glue Data Catalog and can dynamically partition data by date and sensor_id. Firehose handles buffering, compression, and delivery to S3 without requiring custom code or infrastructure management.

Exam trap

DEA-C01 often tests the misconception that custom code (Lambda, KCL) is needed for Parquet conversion and partitioning, but Firehose provides built-in features for these tasks.

How to eliminate wrong answers

Option A is wrong because Amazon EMR requires provisioning and managing clusters, which is operationally heavy and less cost-effective for a simple streaming ETL task. Option B is wrong because a Lambda function would need custom code to buffer, transform to Parquet, and manage partitioning, increasing complexity and potential errors. Option C is wrong because a custom Kinesis Client Library application on EC2 requires managing instances, writing code for buffering and Parquet conversion, and handling scaling, which is not operationally efficient.

398
MCQmedium

Refer to the exhibit. A data engineer is troubleshooting an AWS Lambda function that reads from an S3 bucket and writes to a Kinesis Data Stream. The Lambda function fails with an AccessDeniedException when calling the kinesis:PutRecords API. Which change is needed to the IAM policy?

A.Add s3:PutObject permission to the policy
B.Change the resource ARN for Kinesis to a wildcard
C.Change the resource ARN for Kinesis to include the correct stream name
D.Add kinesis:PutRecords permission to the policy
AnswerC

The Lambda execution role's policy must scope the Kinesis action to the exact stream ARN, since `kinesis:PutRecords` is evaluated against the stream resource, not the account or wildcard. A mismatched stream name in the resource ARN yields AccessDeniedException even when the action itself is permitted, satisfying the stem's requirement to correct the policy.

Why this answer

The Lambda function's IAM policy grants kinesis:PutRecords permission but the resource ARN does not match the target Kinesis Data Stream. The AccessDeniedException occurs because IAM evaluates the resource ARN against the stream's ARN and denies access on mismatch. The correct fix is to change the resource ARN to include the correct stream name (option C).

While using a wildcard (option B) would also resolve the error, it is not the recommended approach because it violates the principle of least privilege. Option A is incorrect because the error is from Kinesis, not S3. Option D is incorrect because the kinesis:PutRecords action is already present in the policy.

Exam trap

The DEA-C01 exam often tests the misconception that adding the missing API action (kinesis:PutRecords) is the fix, but here the action is already present and the error stems from an incorrect resource ARN. The exam expects the specific stream name to be used, aligning with AWS best practices for least privilege.

How to eliminate wrong answers

Option A is wrong because the error is from Kinesis, not S3; adding s3:PutObject does not address the kinesis:PutRecords AccessDeniedException. Option C is wrong because the policy already includes a specific stream name, but it is incorrect; changing to a wildcard is the fix, not specifying a different name. Option D is wrong because the policy already includes kinesis:PutRecords permission; the issue is the resource ARN restriction, not a missing action.

399
Multi-Selectmedium

A company wants to use AWS Glue to transform data stored in Amazon S3. The data is partitioned by date and includes both CSV and Parquet files. The transformation should be optimized for cost and performance. Which THREE actions should the data engineer take? (Choose THREE.)

Select 3 answers
A.Run a crawler to update the schema before each job run.
B.Use partition pruning by filtering on the date column in the ETL script.
C.Use job bookmarks to process only new data.
D.Increase the number of DPUs to the maximum allowed.
E.Convert all files to Parquet format before processing.
AnswersB, C, E

Filtering on the date partition column lets Glue read only relevant S3 prefixes rather than the full dataset. This reduces bytes scanned and DPU hours, directly meeting the cost and performance optimisation constraint in the stem.

Why this answer

Option B is correct because partition pruning by filtering on the date column lets AWS Glue read only the relevant S3 partitions instead of scanning the entire dataset, which directly reduces I/O, runtime, and cost. Option C is correct because Glue job bookmarks persist state between runs so the job processes only new or changed data since the last run, avoiding reprocessing of already-transformed partitions and lowering both execution time and DPU consumption. Option E is correct because converting CSV files to Parquet enables columnar storage and compression, so Glue reads less data and benefits from predicate pushdown, improving performance and reducing cost compared with row-based CSV.

Option A is not appropriate because running a crawler before every job run adds unnecessary cost and time, and the schema can be supplied directly or updated only when it changes. Option D is not appropriate because maximizing DPUs increases cost rather than optimizing it, and the required performance can be achieved through pruning, bookmarks, and columnar formats.

Exam trap

DEA-C01 often tests cost optimization by tempting candidates with 'more DPUs = faster' (Option D) or 'crawl every run' (Option A), when the correct answers are the data-reduction techniques — partition pruning, job bookmarks, and columnar format conversion.

400
MCQeasy

A media company is building a data pipeline to ingest user activity logs from multiple sources into Amazon S3. The logs are JSON files generated every minute. The company wants to use Amazon Athena to query the logs with minimal latency and cost. The current approach is to use AWS Kinesis Data Firehose to deliver the logs to S3 with a prefix like 'logs/2024/01/01/00/file.json'. However, when running Athena queries, the team notices high query costs because Athena scans all files in the 'logs/' prefix even when querying for a specific date. What should the team do to reduce the amount of data scanned by Athena?

A.Create an Athena view that filters by date.
B.Increase the number of partitions by using a more granular prefix like 'logs/2024/01/01/00/00/'.
C.Convert the JSON files to Apache Parquet format using AWS Glue ETL jobs.
D.Create a Hive-style partition structure in S3 with keys like 'year=2024/month=01/day=01/hour=00/' and update the Glue Data Catalog accordingly.
AnswerD

Hive-style partitioning splits data into year/month/day/hour prefixes, letting Athena prune irrelevant partitions via partition projection or the Glue Data Catalog. Queries for one date then scan only that partition rather than every file under logs/, cutting both cost and latency.

Why this answer

Athena reduces data scanned by using partition pruning, which requires a Hive-style partition structure in S3 (e.g., year=2024/month=01/day=01/hour=00/) registered in the Glue Data Catalog. When queries filter on partition columns, Athena only reads the relevant partitions instead of scanning the entire logs/ prefix. This directly addresses the high query cost caused by scanning all files.

Exam trap

DEA-C01 often tests the difference between S3 prefix organization and true Hive-style partitioning — candidates pick 'more granular prefix' thinking it enables pruning, but without Glue Data Catalog partition registration, Athena still scans all files.

How to eliminate wrong answers

Option A is wrong because an Athena view is just a saved query — it does not change how data is stored or pruned, so it still scans all files. Option B is wrong because a more granular prefix like logs/2024/01/01/00/00/ is not a Hive-style partition; without partition metadata in the Glue Data Catalog, Athena cannot prune and will still scan everything. Option C is wrong because converting to Parquet reduces data scanned per file (columnar compression) but does not eliminate scanning irrelevant date partitions; it's a complementary optimization, not the primary fix for date-based pruning.

401
MCQmedium

A company runs a nightly ETL job using AWS Glue. The job reads data from a JDBC connection to an on-premises MySQL database. The job fails with an error indicating that the connection pool is exhausted. What is the most likely cause and solution?

A.The database is not reachable due to network issues. Check VPC and security groups.
B.The Glue job is hitting the AWS Glue connection pool limit. Increase the Glue connection pool size.
C.The database credentials are expired. Rotate the password in AWS Secrets Manager.
D.The Glue job is using too many executors, exhausting the database connections. Reduce the number of DPUs or increase the database max connections.
AnswerD

Each Glue executor opens its own JDBC connections, so a high DPU count multiplies concurrent sessions against the on-premises MySQL instance until its connection limit is hit. Reducing DPUs lowers that parallelism, directly addressing the exhausted connection pool constraint; raising MySQL's max_connections is the complementary fix.

Why this answer

AWS Glue jobs distribute work across multiple executors, each of which opens its own JDBC connection to the source database. When the number of executors (controlled by DPUs) exceeds the database's configured maximum connections, the database connection pool is exhausted, causing the error. Reducing the number of DPUs or increasing the database's max_connections setting resolves the issue.

Exam trap

The trap here is that candidates confuse a database-side connection pool exhaustion with an AWS Glue service limit or network issue, leading them to incorrectly choose options about Glue connection pools or VPC configurations.

How to eliminate wrong answers

Option A is wrong because a network connectivity issue (e.g., VPC or security group misconfiguration) would typically result in a timeout or 'cannot connect' error, not a 'connection pool exhausted' error. Option B is wrong because AWS Glue does not have a configurable 'connection pool size' for JDBC connections; the pool exhaustion is on the database side, not Glue's internal pool. Option C is wrong because expired credentials would cause an authentication failure (e.g., 'Access denied for user'), not a connection pool exhaustion error.

402
MCQmedium

A company uses AWS Glue ETL jobs to transform data in S3. The job runs successfully but takes longer than expected. The data is in Parquet format and partitioned by date. Which change would most improve performance without increasing cost?

A.Repartition the data by a different column.
B.Convert Parquet to CSV for faster serialization.
C.Increase the number of DPUs for the job.
D.Enable pushdown predicates to filter partitions early.
AnswerD

Pushdown predicates let AWS Glue push partition and column filters to the S3 data source, so only matching partitions and row groups are read. With date partitioning, this prunes irrelevant partitions early, cutting I/O and runtime without adding compute capacity or cost.

Why this answer

Pushdown predicates allow AWS Glue to filter data at the storage layer (e.g., S3 partition pruning) before reading it into memory. Since the data is partitioned by date, enabling pushdown predicates reduces the amount of data scanned, which directly decreases job runtime without requiring additional DPUs or changing the data format.

Exam trap

The trap here is that candidates often assume performance issues are solved by adding more resources (DPUs) or changing file formats, when the real bottleneck is reading unnecessary data due to lack of partition pruning.

How to eliminate wrong answers

Option A is wrong because repartitioning by a different column would likely increase shuffle overhead and may not align with the existing partition structure, potentially worsening performance. Option B is wrong because converting Parquet to CSV would increase data size and I/O due to CSV's lack of compression and columnar storage, making the job slower and more expensive. Option C is wrong because increasing DPUs would raise cost without addressing the root cause (scanning unnecessary partitions), and the question explicitly asks for a change that does not increase cost.

403
MCQmedium

A data engineer needs to catalog a growing S3 data lake. New CSV files land in s3://analytics/raw/orders/ with a partition structure year=YYYY/month=MM/day=DD/. The engineer must create an AWS Glue Data Catalog table that automatically recognizes these partitions and requires no crawler runs for future dates. Which approach meets these requirements?

A.Create a Glue crawler with a schedule of every hour to discover new partitions.
B.Create the table with CREATE EXTERNAL TABLE and specify PARTITIONED BY (year string, month string, day string), then enable partition projection in the table properties.
C.Create a table using an AWS Glue crawler once, then run MSCK REPAIR TABLE on the table after each new partition is written.
D.Create the table with CREATE EXTERNAL TABLE and add each partition manually with ALTER TABLE ADD PARTITION as files arrive.
AnswerB

Partition projection on an Athena/Glue table lets the engine compute partition locations mathematically from the defined pattern and date range instead of consulting the catalog for each partition. New day, month, or year folders are therefore queryable immediately with no crawler or repair step, which is exactly the automatic, low-maintenance behavior the scenario requires.

Why this answer

Partition projection makes Athena and Glue compute partition metadata from the table's own properties, so newly arriving year/month/day folders are queryable without a crawler or repair command. The other choices all require periodic human or scheduled action after each new partition lands, which violates the explicit no-crawler-runs requirement.

Exam trap

The trap here is assuming the Data Catalog must always be refreshed by a crawler or MSCK REPAIR TABLE before new partitions are visible, when partition projection removes that dependency entirely.

404
Multi-Selectmedium

A data engineer is designing an ingestion pipeline that uses Amazon Kinesis Data Firehose to deliver streaming records into an Amazon S3 bucket. The records arrive as JSON, and downstream consumers require Parquet with a stable schema. The engineer must configure the Firehose delivery stream so records are converted to Parquet before landing in S3. (Choose two.)

Select 2 answers
A.Enable Amazon S3 Object Lambda access points on the destination bucket to convert objects to Parquet on read.
B.Enable record format conversion on the Firehose delivery stream and select Apache Parquet as the output format.
C.Attach an AWS Glue table as the schema reference for the record format conversion configuration.
D.Configure an AWS Lambda function as the delivery stream's transformation to rewrite each record into Parquet bytes.
E.Set the S3 destination prefix to include a .parquet file extension so Firehose writes columnar files.
AnswersB, C

Firehose supports record format conversion from JSON to Parquet. Enabling it and selecting Apache Parquet as the output format instructs Firehose to convert records in flight before writing to S3. Without this setting, Firehose writes the raw JSON payload, so downstream consumers would not receive Parquet as required.

Why this answer

Firehose record format conversion converts JSON to Parquet in flight, and it requires a schema reference. The supported schema source is an AWS Glue table in the Data Catalog that describes the incoming JSON structure. Enabling conversion with Apache Parquet output and pointing to a matching Glue table together produce Parquet files in S3 for downstream analytics.

Exam trap

The trap here is assuming a Lambda transform or a file extension change can produce Parquet, when Firehose requires its built-in record format conversion with a Glue table schema reference.

405
MCQmedium

A data engineer needs to transform JSON data from Amazon S3 into Parquet using AWS Glue. The JSON is nested and contains arrays. The engineer wants to flatten the nested structure and write the result to S3 partitioned by a 'region' field. Which combination of Glue transforms should the engineer use?

A.Use the 'ResolveChoice' transform to flatten nested data, then write with partition keys.
B.Use the 'Relationalize' transform to flatten the nested JSON, then write the resulting DynamicFrame with partition keys set to 'region'.
C.Use the 'DropNullFields' transform to remove nulls, then write with partition keys.
D.Use the 'Unbox' transform to extract nested fields, then use 'ApplyMapping' to rename columns, and write with partition keys.
AnswerB

Relationalize flattens nested structures and arrays into separate relational tables, producing a DynamicFrame collection. The engineer can select the desired table, set partition keys to 'region', and write to S3 in Parquet. This is the standard Glue approach for flattening nested JSON before writing partitioned output, and it avoids manual schema manipulation.

Why this answer

Relationalize is designed to flatten nested JSON and arrays into relational tables, producing a collection of DynamicFrames. The engineer can then select the relevant table, set the 'region' field as a partition key, and write Parquet to S3. This is the intended Glue transform for nested-to-relational conversion.

Exam trap

The trap here is confusing Unbox, which extracts a single nested field, with Relationalize, which fully flattens nested structures and arrays into multiple relational tables.

406
MCQmedium

A data engineer is troubleshooting a Lambda function that reads from the Kinesis stream 'my-data-stream'. The Lambda function is able to read data but occasionally fails with 'KMS.AccessDeniedException'. What is the most likely cause?

A.The Lambda function's execution role does not have kms:Decrypt permission for the KMS key.
B.The retention period is too short; increase it.
C.The stream has too few shards; increase shard count.
D.The Lambda function is not authorized to consume from Kinesis streams.
AnswerA

Kinesis encrypts stream data with a customer managed KMS key, so every GetRecords call requires kms:Decrypt. The intermittent KMS.AccessDeniedException means the function's execution role lacks that permission on the key policy, blocking decryption of returned records even though the read itself succeeds.

Why this answer

The KMS.AccessDeniedException indicates that the Lambda function's execution role lacks the kms:Decrypt permission for the AWS KMS key used to encrypt the Kinesis stream. When a Kinesis stream is encrypted with a customer managed KMS key, the consumer (Lambda) must have explicit decrypt permissions on that key to read the data records.

Exam trap

The trap here is that candidates may confuse KMS permissions with Kinesis stream permissions, assuming the error is about stream consumption authorization rather than decryption of encrypted data.

How to eliminate wrong answers

Option B is wrong because a short retention period would cause data to expire, not produce a KMS.AccessDeniedException. Option C is wrong because insufficient shards would cause throttling or throughput issues, not a KMS access error. Option D is wrong because the Lambda function is already able to read data (as stated), so it has Kinesis consumption authorization; the error is specifically about KMS decryption, not stream-level permissions.

407
MCQeasy

A data engineer must load a 2 GB uncompressed CSV file from Amazon S3 into Amazon Redshift using the COPY command. The cluster is a two-node ra3.xlplus cluster, and the load is running far slower than expected. The engineer wants the fastest reliable improvement without changing the cluster. What should the engineer do?

A.Convert the CSV to GZIP before loading and keep it as one file.
B.Add the COMPUPDATE OFF and STATUPDATE OFF parameters to the COPY command.
C.Increase the cluster to four nodes so more slices can read the same file simultaneously.
D.Split the single CSV file into multiple files and load them in parallel with a single COPY command.
AnswerD

Amazon Redshift parallelizes COPY across slices, but a single file can only be read by one slice at a time, so a lone 2 GB file serializes the load. Splitting it into multiple files lets each slice read its own file concurrently, dramatically improving throughput. This is the standard remedy and requires no cluster change.

Why this answer

Redshift distributes a COPY workload across slices, but each input file is read by only one slice. A single 2 GB CSV therefore runs on one slice while the rest idle. Splitting the file into several smaller files, ideally a multiple of the slice count, allows all slices to read concurrently, which is the fastest reliable fix that does not alter the cluster.

Exam trap

The trap here is thinking that more cluster nodes or compression will speed up a single-file COPY, when the real limiter is that one file maps to one slice.

408
MCQhard

A data engineer is using AWS Glue DataBrew to profile a dataset in Amazon S3. The dataset contains a column 'customer_id' that should be unique. The engineer runs a profile job and notices that the 'customer_id' column has a uniqueness metric of 98%. The engineer needs to identify the duplicate values. Which DataBrew feature should the engineer use to display the duplicate values?

A.Use the 'Column profile' view to see a histogram of values.
B.Use the 'Value frequencies' view in the profile job results to see the most frequent values.
C.Use the 'Column statistics' view in the profile job results to see the count of duplicates.
D.Use the 'Data quality rules' feature to create a rule that flags duplicates.
AnswerB

In AWS Glue DataBrew profile results, the 'Value frequencies' view displays the most common values in a column along with their counts. For a column like 'customer_id' with duplicates, this view will show which IDs appear more than once, allowing the engineer to identify the duplicate values. It is the correct feature for this purpose.

Why this answer

AWS Glue DataBrew profile jobs include a 'Value frequencies' view that lists the most frequent values in a column with their counts. For a column that should be unique, this view directly reveals which customer_id values are duplicated. The other options provide summary statistics or rules but do not display the specific duplicate values.

Exam trap

The trap here is assuming that the uniqueness metric or column statistics will list the duplicate values, when in fact you need to examine the value frequencies to see the actual duplicates.

409
MCQhard

A data engineer is building an AWS Glue ETL job that reads from an Amazon S3 bucket containing nested JSON files. The job must flatten the nested structure and write the output to Amazon Redshift. The engineer notices that some JSON records have missing fields and inconsistent schemas. Which AWS Glue feature should be used to handle these inconsistencies and ensure the job does not fail?

A.Use a Glue DataFrame with a predefined schema and set the job to ignore malformed records.
B.Use Amazon Kinesis Data Firehose to transform the JSON before the Glue job reads it.
C.Use AWS Glue crawlers to infer the schema and then manually edit the Data Catalog table to add missing columns.
D.Use a Glue DynamicFrame with the resolveChoice transform to handle schema inconsistencies.
AnswerD

DynamicFrame and resolveChoice are designed to handle schema variability in semi-structured data. They allow you to specify how to resolve ambiguous types or missing fields, such as casting to a specific type or dropping nulls. This prevents job failures due to inconsistent schemas and is the recommended approach for nested JSON with varying structures.

Why this answer

AWS Glue DynamicFrames are specifically built to handle semi-structured data with evolving schemas. The resolveChoice transform lets you resolve ambiguous or conflicting schema types, such as choosing a specific data type or dropping nulls. This ensures the ETL job can process nested JSON with missing fields without failing, making it the correct choice for this scenario.

Exam trap

The trap here is thinking that a predefined schema or manual catalog edits can handle runtime schema variations, but they do not provide the flexibility needed for inconsistent JSON.

410
MCQmedium

A company is ingesting streaming data into Kinesis Data Streams. The consumer application experiences high latency due to a single shard bottleneck. What is the most effective way to reduce latency?

A.Increase the number of shards in the data stream.
B.Wait for automatic scaling to add shards.
C.Use the Kinesis Client Library (KCL) to process records.
D.Switch to Amazon Kinesis Data Firehose.
AnswerA

A single shard caps throughput and processing parallelism, creating the bottleneck. Adding shards spreads records across more consumers, raising aggregate throughput and reducing per-record latency, directly addressing the stem's single-shard constraint. Resharding via UpdateShardCount is the standard remedy.

Why this answer

Increasing the number of shards in the Kinesis data stream distributes the incoming records across more shards, allowing the consumer application to process them in parallel and reducing the per-shard bottleneck that causes high latency. Each shard supports up to 1 MB/s or 1,000 records/s ingress, so adding shards directly increases throughput capacity. This is the most direct and effective way to address a single-shard bottleneck.

Exam trap

The trap is confusing KCL (a consumer-side library for coordination) with a throughput solution — KCL does not add capacity, only shard count does.

How to eliminate wrong answers

Option B is wrong because Kinesis Data Streams does not automatically scale shards; on-demand mode can scale automatically, but the question describes a provisioned stream with a shard bottleneck, and waiting is not an effective action. Option C is wrong because the Kinesis Client Library helps with consumer coordination and checkpointing but does not increase the stream's throughput capacity; it cannot fix a shard-level bottleneck. Option D is wrong because Kinesis Data Firehose is a delivery service for loading data into destinations, not a replacement for a low-latency consumer application, and it does not solve the shard bottleneck.

411
MCQhard

A company is ingesting streaming data from social media feeds using Amazon Kinesis Data Streams. The data volume peaks at 10,000 records per second, and each record is up to 1 KB. The company needs to archive the raw data in Amazon S3 in near real-time and also make it available for real-time analytics using Amazon Kinesis Data Analytics. What is the MOST efficient architecture to meet these requirements?

A.Use Kinesis Data Streams as the ingestion point. Use Kinesis Data Firehose to read from the stream, convert to Parquet, and write to S3. Use a Lambda function to send data to Kinesis Data Analytics.
B.Use Kinesis Data Streams as the ingestion point. Use a Lambda function to read from the stream, write to S3, and send data to Kinesis Data Analytics.
C.Use two Kinesis Data Streams: one for S3 delivery and one for Kinesis Data Analytics.
D.Use Kinesis Data Streams as the ingestion point. Use Kinesis Data Firehose to read from the stream and write to S3. Use Kinesis Data Analytics to read directly from the same stream.
AnswerD

Kinesis Data Firehose consumes directly from the stream and buffers records before delivering them to Amazon S3, satisfying the near real-time archival requirement without custom consumer code. Kinesis Data Analytics reads the same stream independently, so both consumers run in parallel. This decouples archival from analytics, avoiding duplicate ingestion and extra compute.

Why this answer

Kinesis Data Streams can serve as a single ingestion point, with Kinesis Data Firehose reading from the stream to deliver data to S3 (with optional transformation) and Kinesis Data Analytics reading directly from the same stream for real-time analytics. This avoids unnecessary duplication of streams or Lambda-based processing, which would add latency and complexity. The architecture is the most efficient as it leverages native integrations without intermediate compute.

Exam trap

The trap here is that candidates often overcomplicate the architecture by adding unnecessary Lambda functions or duplicate streams, not realizing that Kinesis Data Firehose and Kinesis Data Analytics can both consume from the same Kinesis Data Stream natively.

How to eliminate wrong answers

Option A is wrong because it suggests using a Lambda function to send data to Kinesis Data Analytics, which is unnecessary and introduces additional cost and latency; Kinesis Data Analytics can read directly from the Kinesis Data Stream. Option B is wrong because using a Lambda function to write to S3 and send data to Kinesis Data Analytics adds processing overhead and potential throughput limitations, whereas Kinesis Data Firehose is purpose-built for streaming to S3 with near-real-time delivery. Option C is wrong because using two separate Kinesis Data Streams is redundant and increases cost and management overhead; a single stream can be consumed by both Firehose and Kinesis Data Analytics simultaneously.

412
MCQeasy

A data pipeline ingests streaming data from thousands of IoT devices into Kinesis Data Streams. The data must be transformed using a simple field mapping before being stored in S3. Which service should be used to perform the transformation with minimal operational overhead?

A.AWS Lambda function invoked by the Kinesis stream
B.AWS Glue ETL job
C.Kinesis Data Analytics
D.Kinesis Data Firehose with a Lambda transformation
AnswerD

Firehose performs the field mapping via an attached Lambda function and delivers to S3, so no clusters or application code need managing. This satisfies the minimal operational overhead constraint, unlike Kinesis Data Analytics or a custom consumer on EC2.

Why this answer

Kinesis Data Firehose can invoke a Lambda function to perform simple field mapping transformations before delivering data to S3, minimizing operational overhead. Option A is wrong because AWS Lambda invoked directly by the Kinesis stream requires custom logic for S3 delivery and stream management, increasing overhead. Option B is wrong because AWS Glue ETL jobs are designed for batch processing and are more complex to set up for streaming transformations.

Option C is wrong because Kinesis Data Analytics is used for real-time analytics with SQL or Flink, not simple field mapping transformations.

413
MCQmedium

A company wants to ingest data from SaaS applications (e.g., Salesforce, Marketo) into Amazon S3 for analytics. The data volume is moderate and updates occur frequently. Which AWS service is BEST suited for this task?

A.Amazon Kinesis Data Streams
B.Amazon AppFlow
C.AWS Database Migration Service (DMS)
D.AWS Glue
AnswerB

Amazon AppFlow provides managed, bidirectional connectors to SaaS applications such as Salesforce and Marketo, writing directly into Amazon S3 without custom code. It supports both scheduled and event-driven flows, satisfying the stem's requirement for frequent updates at moderate volume, unlike AWS DataSync or Storage Gateway, which target file or on-premises transfer rather than SaaS APIs.

Why this answer

Amazon AppFlow is purpose-built for ingesting data from SaaS applications such as Salesforce, Marketo, Slack, and Google Analytics into AWS services like Amazon S3. It provides native connectors, supports scheduled or event-driven flows, and handles authentication and incremental transfers without custom code. For moderate-volume, frequently updated SaaS data landing in S3, AppFlow is the most direct and operationally simple fit.

Exam trap

The trap here is confusing 'streaming' with 'SaaS ingestion' — candidates see 'frequent updates' and jump to Kinesis, but Kinesis has no native SaaS connectors, so AppFlow is the correct managed integration service.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Streams is a low-level streaming ingestion service for custom producers (SDKs, KPL, Kinesis Agent) and has no native SaaS connectors — you would have to build and maintain the Salesforce/Marketo integration yourself. Option C is wrong because AWS DMS is designed to migrate databases (homogeneous/heterogeneous) into AWS, not to pull object or record data from SaaS CRM/marketing platforms. Option D is wrong because AWS Glue is an ETL/catalog service that transforms and moves data between sources; it does not natively authenticate to and extract from SaaS apps like Salesforce or Marketo without custom connectors or AppFlow as the source.

414
MCQeasy

A data pipeline ingests daily CSV files from an FTP server into an Amazon S3 bucket. The files must be converted to Parquet format and partitioned by date for efficient querying using Amazon Athena. Which AWS service is most suitable for this transformation?

A.Amazon Kinesis Data Firehose
B.Amazon EMR
C.AWS Glue
D.AWS Lambda
AnswerC

Glue provides a serverless Spark environment that can transform CSV to Parquet and partition data efficiently.

Why this answer

AWS Glue is the most suitable service because it provides a fully managed ETL (Extract, Transform, Load) capability that can natively read CSV files from S3, convert them to Parquet format, and write the output partitioned by date. Glue's built-in transform 'ConvertToParquet' and dynamic frame partitioning make this a straightforward, serverless solution without needing to manage infrastructure.

Exam trap

The trap here is that candidates often confuse AWS Glue with Amazon EMR, thinking EMR is always needed for Parquet conversion, but Glue's serverless ETL is more appropriate for scheduled batch jobs without cluster management overhead.

How to eliminate wrong answers

Option A is wrong because Amazon Kinesis Data Firehose is designed for streaming data ingestion, not batch processing of daily CSV files from an FTP server; it lacks native support for reading from S3 as a source and performing complex transformations like CSV-to-Parquet conversion with custom partitioning. Option B is wrong because Amazon EMR is a managed Hadoop cluster that can perform this transformation, but it requires provisioning and managing EC2 instances, which is overkill for a simple daily batch job and not the most suitable service for a serverless, cost-effective solution. Option D is wrong because AWS Lambda has a maximum execution time of 15 minutes and a limited memory capacity (up to 10 GB), which is insufficient for processing large daily CSV files (e.g., gigabytes in size) and performing efficient Parquet conversion with partitioning.

415
Multi-Selectmedium

A data engineer is designing a pipeline to ingest data from an Amazon RDS for PostgreSQL database into Amazon S3 using AWS Database Migration Service (AWS DMS). The source database has a high volume of transactions and the engineer needs to capture ongoing changes with minimal impact on the source. The target S3 bucket must store the data in Parquet format for querying with Amazon Athena. Which two actions should the engineer take to meet these requirements? (Choose two.)

Select 2 answers
A.Enable multi-AZ on the replication instance and use a single large replication instance to handle all tables.
B.Use AWS DMS to replicate directly to Amazon S3 in Parquet format by selecting the Parquet output option in the target endpoint settings.
C.Set up a full load only and schedule it to run every hour to capture changes.
D.Configure AWS DMS to write to Amazon S3 in CSV format and then use AWS Glue to convert the data to Parquet.
E.Configure AWS DMS to use change data capture (CDC) with a replication instance that has sufficient resources and enable logical replication on the source PostgreSQL database.
AnswersB, E

AWS DMS supports Amazon S3 as a target and can write data in Parquet format when configured in the endpoint settings. This eliminates the need for a separate conversion step and allows Athena to query the data directly. The engineer must specify the appropriate serialization format and ensure the S3 bucket has the necessary permissions.

Why this answer

To capture ongoing changes from PostgreSQL with minimal impact, AWS DMS must use CDC, which requires logical replication on the source and a replication instance sized appropriately. To store data in Parquet for Athena, the DMS target endpoint for S3 must be configured to output Parquet. These two actions together satisfy the requirements: near-real-time replication with low source overhead and a columnar format optimized for query performance.

Exam trap

The trap here is assuming that a periodic full load is sufficient for ongoing changes, when it actually causes high source load and misses intermediate changes; and overlooking that DMS can natively output Parquet to S3.

416
Multi-Selecthard

A data engineer is building a pipeline to ingest data from an on-premises Oracle database into Amazon S3. The pipeline must capture change data (CDC) in near real-time and handle schema changes. Which TWO AWS services should the engineer use?

Select 2 answers
A.AWS Glue Schema Registry
B.AWS Snowball Edge
C.Amazon AppFlow
D.Amazon Kinesis Data Streams with Kinesis Agent
E.AWS Database Migration Service (DMS) with CDC
AnswersA, E

Manages schema evolution for streaming data.

Why this answer

AWS Glue Schema Registry (A) is correct because it enables schema discovery, validation, and evolution for streaming data, allowing the pipeline to handle schema changes from the Oracle CDC source. It integrates with Apache Kafka and Amazon Kinesis Data Streams to enforce schema compatibility rules (e.g., backward, forward, full) as data arrives, ensuring downstream consumers can adapt to evolving schemas without breaking.

Exam trap

The DEA-C01 exam often tests the misconception that Amazon Kinesis Data Streams alone can perform CDC from a database, but Kinesis requires a separate agent or connector (like Debezium or DMS) to read database logs, making DMS the correct CDC service.

417
MCQmedium

A data engineer is using AWS Glue to transform data from Amazon S3. The source data is in CSV format with inconsistent date formats across files (e.g., 'MM/DD/YYYY' and 'YYYY-MM-DD'). The engineer needs to standardize all dates to 'YYYY-MM-DD' format in the output. Which AWS Glue transform should the engineer use to achieve this?

A.ApplyMapping
B.Filter
C.ResolveChoice
D.Map
AnswerD

The `Map` transform allows applying a custom function to each record in a DynamicFrame. The engineer can write a Python function that attempts to parse the date string using multiple formats (e.g., using `datetime.strptime` with different format strings) and then outputs the standardized 'YYYY-MM-DD' string. This provides the flexibility needed to handle inconsistent date formats across files.

Why this answer

The `Map` transform in AWS Glue enables custom record-level transformations. By writing a Python function that parses various date formats and outputs a standardized string, the engineer can handle inconsistent date formats. This approach is flexible and can be applied across all records, ensuring uniform output.

Exam trap

The trap here is thinking that `ApplyMapping` can handle date format conversion, when it only handles type casting and renaming.

418
MCQeasy

A company has CSV files in an S3 bucket that need to be converted to Parquet and loaded into a Redshift table daily. The transformation is a simple schema mapping without joins. Which AWS Glue feature is BEST suited for this task?

A.AWS Glue ETL job
B.AWS Glue DataBrew
C.AWS Glue Workflow
D.AWS Glue Crawler
AnswerA

AWS Glue ETL jobs handle schema mapping and format conversion natively, reading CSV from S3 and writing Parquet to Redshift through the Glue Data Catalog and JDBC/Redshift connections. No joins are required, so a straightforward ETL job satisfies the daily conversion and load requirement.

Why this answer

AWS Glue ETL jobs are purpose-built for extracting data from sources like S3, applying transformations (including schema mapping and format conversion), and loading the results into targets like Redshift. For a simple CSV-to-Parquet conversion with schema mapping and no joins, a Glue ETL job using the built-in transforms or a Spark script is the correct tool. It handles the conversion and load in a single managed job.

Exam trap

DEA-C01 often tests Glue service selection by describing a transformation task — candidates confuse Glue Workflow (orchestration) or Glue Crawler (cataloging) with the actual ETL engine, picking a service that schedules or catalogs rather than transforms.

How to eliminate wrong answers

Option B is wrong because AWS Glue DataBrew is a visual data preparation tool aimed at analysts for cleaning and normalizing data interactively, not for scheduled ETL pipelines that load into Redshift. Option C is wrong because AWS Glue Workflow is an orchestration service that chains crawlers and jobs together — it schedules and coordinates but does not itself perform the transformation. Option D is wrong because a Glue Crawler only discovers schema and populates the Data Catalog; it does not transform or load data into Redshift.

419
MCQmedium

A company uses AWS Glue to process data in Amazon S3. The Glue job fails with an error indicating that the partition keys in the catalog do not match the actual S3 partition structure. What is the most likely cause?

A.The IAM role does not have permissions to read the S3 data
B.The data files are encrypted with SSE-KMS
C.The table name in the catalog is different from the one used in the job
D.The Glue Data Catalog partition metadata is outdated after the S3 structure changed
AnswerD

Glue compares the partition keys registered in the Data Catalog against the actual S3 prefix layout. When the S3 structure changes without a corresponding crawler run or catalog update, the stored metadata no longer matches, so the job fails on the mismatch.

Why this answer

The Glue Data Catalog stores partition metadata separately from the actual S3 partition layout. When the S3 partition structure changes (e.g., new partitions are added or existing ones are renamed) without updating the catalog, the Glue job reads stale partition metadata, leading to a mismatch error. The job fails because it expects partitions based on the catalog, not the live S3 structure.

Exam trap

The trap here is that candidates confuse a partition metadata mismatch with other common Glue errors like IAM permissions or encryption issues, but the error message explicitly references partition keys, not access or decryption problems.

How to eliminate wrong answers

Option A is wrong because an IAM permissions issue would typically cause an Access Denied error, not a partition key mismatch error. Option B is wrong because SSE-KMS encryption affects data decryption, not the structure or metadata of partitions in the catalog. Option C is wrong because a table name mismatch would cause a 'table not found' error, not a partition key mismatch; the error specifically points to partition keys, not table identifiers.

420
MCQhard

A company is using AWS DMS to replicate data from an on-premises Oracle database to Amazon RDS for MySQL. The replication is working, but the target table has a different schema. Which DMS feature should be used to transform the source schema to match the target?

A.Use AWS Schema Conversion Tool (SCT)
B.Use AWS Glue ETL jobs
C.Use DMS transformation rules
D.Use AWS Lambda triggers
AnswerC

DMS transformation rules apply schema and table mappings during replication, letting you rename schemas, tables and columns, change data types, and filter rows so the Oracle source matches the RDS for MySQL target schema. This directly satisfies the stem's requirement to transform a differing schema without altering the source database.

Why this answer

AWS DMS transformation rules allow you to modify the schema, table, or column names and data types during the migration process. This feature is specifically designed to handle schema transformations within the DMS task itself, enabling you to map the source Oracle schema to the target MySQL schema without external tools or services.

Exam trap

The trap here is that candidates often confuse AWS Schema Conversion Tool (SCT) with DMS transformation rules, assuming SCT handles runtime schema mapping, whereas SCT is a separate pre-migration assessment and conversion tool, not a DMS feature for ongoing replication transformations.

How to eliminate wrong answers

Option A is wrong because AWS Schema Conversion Tool (SCT) is used for heterogeneous database migrations to convert the entire database schema and code objects, but it is not a feature of DMS for runtime schema transformation during ongoing replication. Option B is wrong because AWS Glue ETL jobs are for batch data processing and transformation in a data lake or warehouse, not for real-time schema mapping within a DMS replication task. Option D is wrong because AWS Lambda triggers can be used for custom post-processing or validation, but they are not a built-in DMS feature for transforming source schemas to match target schemas during replication.

421
MCQeasy

A data engineer must load a 500 MB CSV file from Amazon S3 into an existing Amazon Redshift table once per day. The file has a header row and uses a pipe delimiter. The engineer wants the fastest load and the least operational overhead. Which approach should the engineer use?

A.Use the Redshift Data API to run individual INSERT statements for each row in the CSV file.
B.Use an AWS Glue ETL job to read the CSV from S3 and write to Redshift through a JDBC connection.
C.Use AWS Database Migration Service with a change data capture task to replicate the S3 file into Redshift.
D.Use the Redshift COPY command with the IAM role, CSV, HEADER, and DELIMITER options specified.
AnswerD

COPY is Redshift's native bulk loader and reads directly from S3 in parallel, which is the fastest option for a 500 MB file and needs no extra infrastructure. Specifying CSV, HEADER, and the pipe DELIMITER matches the file layout, so the load succeeds without preprocessing and with minimal ongoing operational effort.

Why this answer

The Redshift COPY command is purpose-built for bulk loading from S3 and parallelizes the read, making it the fastest and simplest choice for a single delimited file. The CSV, HEADER, and DELIMITER parameters align the load with the file format, so no staging or transformation layer is needed.

Exam trap

The trap here is reaching for a managed ETL service by default, when Redshift's own COPY command is both faster and simpler for a straightforward file load.

422
Multi-Selectmedium

A company is using AWS Glue to run ETL jobs that transform data from Amazon S3 to Amazon Redshift. The jobs are failing intermittently with 'Out of Memory' errors. The team wants to resolve this issue without increasing costs significantly. Which TWO actions should the team take?

Select 1 answer
A.Increase the Spark memory overhead parameter in the Glue job configuration.
B.Use DynamicFrame instead of Spark DataFrame for transformations.
C.Increase the number of workers to maximum allowed.
D.Switch from a Spark job to a Python shell job.
E.Change the worker type from 'G.1x' to 'G.2x' to double memory per worker.
AnswersE

G.2x workers provide double the memory and vCPU of G.1x at a higher hourly rate, so each executor handles larger partitions and shuffles without spilling to disk, eliminating the out-of-memory failures. This directly addresses the memory constraint while scaling cost proportionally rather than adding workers.

Why this answer

Option E is correct because switching from G.1x to G.2x workers doubles the memory (and vCPU) per worker while keeping the same worker count, which directly addresses OOM conditions in memory-intensive transformations and is a targeted, cost-controlled change rather than scaling out to maximum workers. Option A is not correct because AWS Glue does not provide a 'Spark memory overhead parameter' in the job configuration; memory overhead is not a user-configurable Glue job setting, so this action cannot be taken as described. Option B is not correct because DynamicFrame vs Spark DataFrame is an API choice for schema handling and does not by itself reduce memory pressure enough to fix OOM errors.

Option C is not correct because increasing workers to the maximum allowed raises cost significantly and does not fix per-executor memory limits. Option D is not correct because a Python shell job cannot run distributed Spark ETL at the scale needed for S3-to-Redshift transformations and would not resolve Spark executor OOM issues.

Exam trap

The trap here is that candidates may assume increasing the number of workers (Option C) is the only way to fix OOM errors, but this ignores the cost-effective tuning of worker type upgrades. Another trap is selecting a nonexistent 'memory overhead' parameter (Option A) that sounds plausible but is not a real Glue job configuration option.

423
MCQhard

A company uses Amazon EMR to process large datasets stored in Amazon S3. The data is in Parquet format and partitioned by date. The EMR cluster uses Spark SQL for transformations. Recently, the job has been slow and some tasks are failing due to 'java.lang.OutOfMemoryError'. The cluster has 10 core nodes of type m5.xlarge. Which configuration change would MOST improve performance and stability?

A.Increase the number of Spark partitions using repartition(), but keep the same nodes.
B.Change the core node instance type to r5.xlarge (memory-optimized).
C.Increase the number of executor cores in the Spark configuration.
D.Enable Kryo serialization in the Spark configuration.
AnswerB

OutOfMemoryError during Spark SQL tasks indicates executor heap exhaustion, not CPU shortage. Switching core nodes to r5.xlarge increases RAM per node, giving executors more memory for shuffles and aggregations, which addresses the memory constraint causing task failures and slowness.

Why this answer

The error 'java.lang.OutOfMemoryError' indicates that the Spark executors are running out of memory during processing. The m5.xlarge instance type provides 16 GiB of memory, but the workload likely requires more memory per task. Switching to r5.xlarge (32 GiB of memory) doubles the available memory per node, reducing memory pressure and preventing task failures, which directly improves stability and performance for memory-intensive transformations.

Exam trap

The trap here is that candidates often focus on tuning Spark configurations (partitions, cores, serialization) to fix OutOfMemoryErrors, but the real issue is insufficient physical memory per node, which requires a change in instance family rather than software settings.

How to eliminate wrong answers

Option A is wrong because increasing the number of partitions with repartition() can actually increase memory overhead due to shuffle operations and does not address the root cause of insufficient memory per executor; it may even worsen the OutOfMemoryError by creating more tasks that compete for the same limited memory. Option C is wrong because increasing the number of executor cores without increasing memory per core will cause more concurrent tasks to share the same fixed heap, exacerbating memory contention and making OutOfMemoryErrors more likely. Option D is wrong because enabling Kryo serialization reduces the size of serialized objects and improves CPU efficiency, but it does not increase the available heap memory; it cannot prevent OutOfMemoryErrors caused by insufficient memory for data processing.

424
MCQmedium

A data engineering team uses AWS Glue ETL jobs to process data daily. They notice that job run times are increasing as data volume grows. Which action will most effectively improve performance without changing the code?

A.Use a smaller instance type for the Glue job.
B.Enable job bookmark to skip previously processed data.
C.Split the data into more files in S3.
D.Increase the number of DPUs for the Glue job.
AnswerD

Increasing DPUs adds more Apache Spark executors, raising parallel task capacity so each stage processes partitions faster. This directly addresses the growing data volume constraint while requiring no code changes, since Glue scales worker allocation automatically. It outperforms alternatives that alter code or partitioning logic.

Why this answer

Increasing the number of DPUs (Data Processing Units) for the Glue job directly scales the compute resources allocated to the job, which can improve performance and reduce run times as data volume grows. This does not require code changes. Job bookmarks help skip already processed data but do not address increasing data volume.

Smaller instance types and more files in S3 may not effectively improve performance.

Exam trap

The trap is thinking that job bookmarks or file splitting will improve performance for growing data; candidates might overlook that scaling DPUs directly addresses compute needs.

How to eliminate wrong answers

Option A is wrong because using a smaller instance type reduces compute resources, which would likely worsen performance. Option B is wrong because enabling job bookmark only skips previously processed data; it does not improve performance for new data and may not be applicable if all data is new. Option C is wrong because splitting data into more files can sometimes improve parallelism, but it is not the most effective action and may not help if the job is already parallel; increasing DPUs directly scales compute.

425
MCQhard

A data engineer is building an AWS Glue ETL job that reads from an Amazon S3 bucket with millions of small JSON files. The job is running slowly and often fails with out-of-memory errors. The engineer needs to improve performance and reliability. What should the engineer do?

A.Enable job bookmarks to process only new files and reduce the workload.
B.Use AWS Glue's groupFiles and groupSize parameters to combine small files into larger groups.
C.Increase the number of DPUs for the Glue job to handle the large number of files.
D.Convert the JSON files to Parquet using an AWS Glue crawler before running the ETL job.
AnswerB

AWS Glue provides groupFiles and groupSize to coalesce multiple small files into a single partition read. This reduces the number of tasks and metadata overhead, improving performance and reducing memory pressure. It is the recommended approach for large numbers of small files in S3 when using Glue ETL.

Why this answer

When an AWS Glue job reads millions of small files, the overhead of listing and opening each file causes performance degradation and memory issues. The groupFiles and groupSize parameters allow Glue to combine multiple small files into larger read groups, reducing task count and improving throughput. This is the most direct and effective solution for this scenario.

Exam trap

The trap here is assuming that simply adding more DPUs will solve performance problems caused by many small files, when the real fix is to reduce file count through grouping.

426
Multi-Selecthard

A data engineer is designing an AWS Glue ETL job that reads data from an Amazon S3 bucket containing many small JSON files and writes the output to Amazon S3 in Parquet format. The engineer wants to improve read performance and reduce the number of output files. Which two actions should the engineer take? (Choose two.)

Select 2 answers
A.Increase the number of DPUs to allow more executors to read the small files in parallel.
B.Call repartition or coalesce on the DynamicFrame before writing to control the number of output files.
C.Enable job bookmarks to skip previously processed files and reduce the number of files read.
D.Convert the source JSON files to Parquet before the Glue job runs using an AWS Lambda function triggered on each S3 object.
E.Use the AWS Glue groupFiles option to group multiple small files into a single partition during the read.
AnswersB, E

Repartitioning or coalescing before the write step controls how many partitions exist and therefore how many output files are produced. Coalesce reduces the number of partitions without a full shuffle, which is efficient when reducing file count. This action directly reduces the number of output files written to S3, complementing the read-side grouping.

Why this answer

Grouping small files during the read reduces file-open overhead and improves read performance, while repartitioning or coalescing before writing controls the number of output files. Job bookmarks only help across runs, adding DPUs does not fix per-file overhead or output count, and pre-converting files with Lambda does not address the small-file or output-file problems within the Glue job.

Exam trap

The trap here is thinking that adding more compute or enabling bookmarks will solve the small-file problem, when the real fixes are read-side file grouping and write-side partition control.

427
MCQeasy

A company uses AWS Glue to run ETL jobs daily. The data is stored in S3 as Parquet files partitioned by date. Recently, jobs have failed with the error 'No such file or directory' for certain partitions. What is the MOST likely cause?

A.The schema has changed and Glue cannot parse the data.
B.A partition folder was deleted or not created by the upstream process.
C.The files are compressed with an unsupported codec.
D.The IAM role does not have s3:GetObject permission.
AnswerB

Glue resolves each partition path from the catalog at run time; if the upstream job failed to write a partition folder or it was deleted, the read fails with 'No such file or directory'. This matches the stem's missing-partition error rather than a schema or permissions issue.

Why this answer

The error 'No such file or directory' indicates that the Glue ETL job is attempting to read a specific S3 partition path that does not exist. Since the data is partitioned by date and the job runs daily, the most likely cause is that the upstream process failed to create or accidentally deleted the partition folder for that date. Glue's dynamic frame or Spark DataFrame will throw this error when it tries to list or read files from a missing prefix.

Exam trap

The trap here is that candidates confuse file-level permission errors (Option D) with missing directory errors, but S3 returns distinct HTTP status codes (403 vs 404) that map to different error messages in Spark/Glue.

How to eliminate wrong answers

Option A is wrong because a schema change would typically cause a parsing or schema mismatch error (e.g., 'Schema mismatch' or 'Cannot convert type'), not a 'No such file or directory' error. Option C is wrong because unsupported compression codecs (e.g., LZO without proper libraries) would cause a 'Codec not found' or 'Compression error', not a missing file error. Option D is wrong because missing s3:GetObject permission would result in an 'Access Denied' (403) error, not a 'No such file or directory' error.

428
Multi-Selectmedium

A data engineer is designing a data ingestion pipeline for real-time clickstream data from a website. The data must be ingested with low latency (seconds) and made available for multiple consumer applications, including a dashboard that refreshes every minute and a machine learning model that processes data in near-real-time. The engineer needs to choose a streaming ingestion service. Which TWO services meet these requirements? (Select TWO.)

Select 2 answers
A.Amazon Kinesis Data Firehose
B.Amazon Managed Streaming for Apache Kafka (Amazon MSK)
C.Amazon Kinesis Data Streams
D.Amazon Simple Queue Service (SQS)
E.Amazon S3
AnswersB, C

MSK is a fully managed Kafka service that provides low-latency streaming and supports multiple consumer groups.

Why this answer

Amazon Kinesis Data Streams (C) provides sub-second ingestion latency and supports multiple consumer applications via its enhanced fan-out feature, enabling a dashboard and ML model to consume data concurrently with low latency. Amazon MSK (B) offers similar real-time capabilities with Apache Kafka's native pub/sub model, allowing multiple consumers to process the same stream independently and with low latency, meeting the near-real-time requirements.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Firehose with Kinesis Data Streams, assuming Firehose provides real-time ingestion, but Firehose buffers data for at least 60 seconds before delivery, making it unsuitable for sub-second latency requirements.

429
MCQmedium

A company uses Amazon Kinesis Data Streams to ingest real-time logs from thousands of applications. The data must be transformed and enriched with reference data from Amazon S3 before being stored in Amazon S3 in Parquet format. The transformation logic is stateful and requires exactly-once processing. Which AWS service should the data engineer use to perform the transformation?

A.Amazon Kinesis Data Firehose with AWS Lambda transformation.
B.AWS Glue streaming ETL job.
C.Amazon Managed Service for Apache Flink (formerly Kinesis Data Analytics) with a Flink application.
D.AWS Lambda with Kinesis Data Streams event source mapping.
AnswerC

Amazon Managed Service for Apache Flink supports stateful stream processing with exactly-once semantics. It can read from Kinesis Data Streams, enrich data by joining with reference data from S3 (using Flink's rich functions or async I/O), and write to S3 in Parquet format using the FileSystem connector. This service is designed for complex, stateful transformations and ensures exactly-once processing.

Why this answer

The need for stateful processing, exactly-once semantics, and enrichment with reference data from S3 points to a stream processing framework. Amazon Managed Service for Apache Flink provides these capabilities with native support for state, exactly-once processing, and connectors to Kinesis and S3. Other services either lack stateful processing, exactly-once guarantees, or both.

Exam trap

The trap here is assuming that AWS Glue streaming ETL or Lambda can handle stateful, exactly-once processing, when they are better suited for simpler transformations.

430
MCQhard

A data engineer is using AWS Database Migration Service (AWS DMS) to replicate ongoing changes from an Amazon RDS for PostgreSQL database to an Amazon S3 bucket. The source table has a primary key and the engineer needs near-real-time change data capture (CDC) with minimal impact on the source. Which DMS task setting should the engineer configure to meet these requirements?

A.Set the migration type to full load plus CDC and enable logical replication on the source
B.Set the migration type to full load only and schedule hourly re-runs
C.Set the migration type to CDC only and configure a change table on the source
D.Set the migration type to full load only and enable binary logging on the source
AnswerA

Full load plus CDC performs an initial load and then continuously reads changes from the PostgreSQL write-ahead log using logical replication. This provides near-real-time replication of inserts, updates, and deletes while reading from the log rather than querying tables, which minimizes impact on the source database. It is the correct DMS configuration for ongoing CDC.

Why this answer

AWS DMS uses PostgreSQL logical replication to read changes from the write-ahead log for CDC. Configuring the task as full load plus CDC performs the initial copy and then continuously applies changes, giving near-real-time replication with minimal source impact because changes are read from the log rather than by polling tables. Enabling logical replication on the source is a prerequisite for this mode.

Exam trap

The trap here is confusing full load only with CDC or assuming that any logging feature (such as MySQL binary logging) applies to PostgreSQL; the correct CDC mechanism for PostgreSQL in DMS is logical replication from the WAL.

431
MCQeasy

A data engineer needs to ingest data from an Amazon S3 bucket into Amazon Redshift for analytics. The data is in CSV format and the Redshift table already exists. Which service can be used to perform this ingestion with minimal configuration?

A.AWS Glue
B.Amazon Kinesis Data Firehose
C.Amazon Redshift COPY command
D.AWS Database Migration Service (DMS)
AnswerC

The COPY command loads CSV files directly from Amazon S3 into an existing Redshift table using parallel processing, requiring only an IAM role for authorisation. It satisfies the minimal-configuration constraint because no external ETL service, cluster provisioning, or intermediate staging is needed.

Why this answer

The Amazon Redshift COPY command is the most direct and minimal-configuration method to load data from an S3 bucket into an existing Redshift table. It is purpose-built for bulk data ingestion from S3, supports CSV format natively, and requires only the table name, S3 path, IAM role, and format options — no additional services or pipelines needed.

Exam trap

The trap here is that candidates often overthink and choose AWS Glue or Kinesis Firehose because they assume a managed service is always required, but the COPY command is the simplest and most efficient native tool for batch loading from S3 into an existing Redshift table.

How to eliminate wrong answers

Option A is wrong because AWS Glue is an ETL service that requires creating crawlers, jobs, and triggers, which adds unnecessary complexity for a simple S3-to-Redshift load; it is overkill when the COPY command suffices. Option B is wrong because Amazon Kinesis Data Firehose is designed for streaming data ingestion into Redshift, not for batch loading from static CSV files in S3, and it requires configuring a delivery stream, buffering, and transformation. Option D is wrong because AWS DMS is used for continuous database migration and replication between databases, not for one-time bulk loading of CSV files from S3 into an existing Redshift table.

432
MCQmedium

A data engineer is building a data pipeline that ingests data from Amazon S3 into Amazon Redshift. The data is in CSV format and includes a timestamp column. The pipeline should load only new data incrementally. Which approach is most efficient?

A.Use the COPY command to load the entire bucket and rely on Redshift to deduplicate
B.Use the COPY command with a manifest file that lists only the new S3 objects
C.Use Amazon Redshift Spectrum to query the S3 data directly without loading
D.Use INSERT statements within a loop to load each new file
AnswerB

A manifest listing only new S3 objects lets COPY load precisely those files, avoiding rescanning or reprocessing previously ingested data. This satisfies the incremental-load constraint efficiently, since Redshift reads just the specified object set rather than filtering the whole bucket.

Why this answer

Using the COPY command with a manifest file allows you to explicitly list only the new S3 objects to be loaded, enabling incremental loading without scanning or loading the entire bucket. This approach is efficient as it avoids the overhead of deduplication or full-bucket scans, and it leverages Redshift's native high-speed parallel ingestion from S3.

Exam trap

The trap here is that candidates may think Redshift Spectrum is a valid alternative for loading data, but Spectrum is designed for external querying, not for persistent loading into Redshift tables, which is the explicit requirement in the question.

How to eliminate wrong answers

Option A is wrong because loading the entire bucket and relying on Redshift to deduplicate is inefficient; Redshift does not have built-in deduplication logic for COPY commands, and loading all data repeatedly would waste storage and compute resources. Option C is wrong because Redshift Spectrum queries data directly in S3 without loading it into Redshift tables, which does not meet the requirement of loading data into Redshift for persistent storage and incremental processing. Option D is wrong because using INSERT statements within a loop to load each new file is far less efficient than the COPY command, as it lacks parallelization and high-throughput optimization, leading to poor performance for large datasets.

433
MCQmedium

A data engineer is configuring an AWS Glue ETL job that reads semi-structured JSON event logs from Amazon S3 and must flatten nested arrays into relational columns before writing to Amazon Redshift. The job must run reliably without writing custom serialization code. Which approach should the data engineer take?

A.Use the AWS Glue DynamicFrame Relationalize transform to convert nested structures into separate relational tables before loading into Redshift.
B.Enable the AWS Glue job bookmark and rely on the crawler's inferred schema to automatically normalize nested JSON during the write to Redshift.
C.Apply the AWS Glue ApplyMapping transform to rename nested fields and then write the DynamicFrame directly to Redshift.
D.Convert each JSON file to CSV using an AWS Lambda function triggered by S3 event notifications before the Glue job runs.
AnswerA

Relationalize is purpose-built for flattening nested JSON into a set of related tables, handling arrays and structs automatically without custom code. It produces a root table plus child tables keyed by generated join identifiers, which can then be written to Redshift. This matches the requirement to avoid custom serialization.

Why this answer

The Relationalize transform is the AWS Glue built-in that flattens nested JSON into a root table and child tables linked by join keys, precisely the normalization needed before loading into a relational target like Redshift. It removes the need to hand-code serialization, keeping the pipeline maintainable and aligned with serverless Glue patterns.

Exam trap

The trap here is assuming that any Glue transform can flatten nested JSON, when only Relationalize is designed to explode arrays and structs into related relational tables.

434
MCQeasy

A company has a nightly batch job that processes 100 GB of data from an Amazon S3 bucket and loads it into an Amazon Redshift table. The job currently runs on an Amazon EMR cluster. Which service would reduce operational overhead while providing similar functionality?

A.AWS Database Migration Service
B.AWS Glue
C.Amazon Redshift Spectrum
D.Amazon Athena
AnswerB

AWS Glue provides serverless Spark-based ETL, eliminating cluster provisioning and tuning that EMR requires. It reads directly from S3 and writes to Redshift, delivering comparable transformation capability for the 100 GB nightly batch while removing the operational burden of managing nodes.

Why this answer

AWS Glue is a serverless ETL service that can process 100 GB of data from S3 and load it into Redshift without managing any infrastructure. It provides built-in job scheduling, automatic retries, and a Spark-based engine that handles large-scale data transformations, directly replacing the EMR cluster's functionality while eliminating operational overhead.

Exam trap

The DEA-C01 exam often tests the distinction between query engines (Athena, Redshift Spectrum) and ETL services (Glue), where candidates mistakenly choose Athena or Spectrum because they can read from S3, but they lack the batch processing and data loading capabilities required for this use case.

How to eliminate wrong answers

Option A is wrong because AWS Database Migration Service (DMS) is designed for continuous database replication or one-time migrations between databases, not for batch processing and transforming large datasets from S3 into Redshift. Option C is wrong because Amazon Redshift Spectrum allows querying data directly in S3 without loading it into Redshift, but it does not perform ETL transformations or replace the batch job's processing logic. Option D is wrong because Amazon Athena is an interactive query service for ad-hoc analysis on S3 data, not a managed ETL service for scheduled batch processing and loading into Redshift.

435
Multi-Selecteasy

Which TWO AWS services can be used to ingest streaming data into Amazon S3? (Choose two.)

Select 2 answers
A.Amazon S3 Transfer Acceleration
B.Amazon Managed Streaming for Apache Kafka (Amazon MSK)
C.Amazon Kinesis Data Firehose
D.Amazon Elastic Block Store (Amazon EBS)
E.AWS Snowball
AnswersB, C

Amazon MSK runs Apache Kafka clusters whose topics hold streaming records; consumers, or Kafka Connect's S3 sink connector, read those topics and write objects into Amazon S3. This satisfies the stem's requirement for a service that ingests streaming data into S3.

Why this answer

Amazon Kinesis Data Firehose is the easiest way to reliably load streaming data into Amazon S3. It can capture, transform, and deliver streaming data to S3 destinations in near real-time with no code required. Amazon MSK (Managed Streaming for Apache Kafka) can also ingest streaming data into S3 by using Kafka Connect with an S3 sink connector, which writes data from Kafka topics directly to S3.

Exam trap

The trap here is that candidates often confuse Amazon S3 Transfer Acceleration (a speed optimization for existing uploads) with a streaming ingestion service, or they mistakenly think EBS or Snowball can handle real-time streaming data when they are designed for persistent block storage and offline bulk transfer, respectively.

436
Multi-Selecteasy

A data engineer is designing a real-time streaming pipeline to ingest clickstream data from a website into Amazon S3. The data must be transformed before storage. Which TWO AWS services can be used together to build this pipeline? (Choose TWO.)

Select 2 answers
A.Amazon Kinesis Data Firehose
B.AWS Glue
C.Amazon Kinesis Data Streams
D.Amazon S3 Transfer Acceleration
E.AWS Database Migration Service (DMS)
AnswersA, C

Delivers streaming data to S3 with transformation capabilities.

Why this answer

The correct combination is Amazon Kinesis Data Streams to ingest the streaming clickstream data, and Amazon Kinesis Data Firehose to read from the stream, perform transformations (e.g., via Lambda), and deliver the transformed data to Amazon S3. Kinesis Data Streams provides the durable ingestion layer, while Kinesis Data Firehose handles the delivery and transformation without managing servers.

Exam trap

The trap here is that candidates often confuse Kinesis Data Streams (which requires custom consumers and does not natively write to S3) with Kinesis Data Firehose (which directly delivers to S3 and supports built-in transformations), leading them to select only Data Streams or miss the need for a transformation service.

437
MCQhard

A data engineer is using AWS Glue to read a large dataset from Amazon S3 and write it to Amazon Redshift. The job intermittently fails with 'Communication link failure' errors during the write phase. The dataset is several hundred gigabytes and the Redshift cluster is under heavy query load. Which change is MOST likely to resolve the failures while preserving data integrity?

A.Switch the write operation to use Redshift Spectrum and query the S3 data directly without loading.
B.Write the transformed data to Amazon S3 in Parquet, then use a Redshift COPY command from S3 instead of a direct JDBC write.
C.Increase the Glue job's number of workers and enable auto-scaling.
D.Configure the Glue connection to use a Redshift JDBC URL with a longer socketTimeout and retry the write.
AnswerB

The Glue Redshift connector can stage data in S3 and issue a COPY command, which is far more resilient and efficient than row-by-row JDBC inserts. COPY is atomic per transaction, handles large volumes well, and reduces the number of open connections to Redshift. This approach preserves data integrity and avoids the communication link failures caused by direct writes under load.

Why this answer

Staging data in S3 and using the Redshift COPY command is the recommended pattern for bulk loads. It minimizes open connections, leverages Redshift's parallel load capability, and provides transactional integrity. Increasing workers, extending timeouts, or switching to Spectrum does not resolve the underlying contention or integrity concerns during direct writes.

Exam trap

The trap here is treating the communication failure as a Glue-side timeout or capacity problem, when the real fix is to change the write path to use S3 staging plus Redshift COPY.

438
MCQmedium

A company uses AWS Glue ETL jobs to transform data from Amazon S3 to Amazon Redshift. The job reads JSON files, applies schema mapping, and writes to a Redshift table. Recently, the job started failing with memory errors. The data volume has increased tenfold. Which approach should a data engineer take to resolve this issue with minimal code changes?

A.Switch from Spark to Python Shell job type.
B.Implement batch processing with smaller file sizes.
C.Increase the number of DPUs allocated to the Glue job.
D.Use Redshift Spectrum to query data directly from S3.
AnswerC

Glue allocates memory per executor, so raising the DPU count adds workers and distributes the tenfold-larger dataset across more memory, resolving the out-of-memory failures. It requires only a job configuration change, meeting the minimal-code-change constraint.

Why this answer

Increasing the number of DPUs (Data Processing Units) allocated to the AWS Glue job directly addresses the memory constraint caused by a tenfold increase in data volume. Glue ETL jobs run on Apache Spark, which distributes data processing across executors; more DPUs provide more memory and compute capacity, allowing the job to handle larger datasets without code changes.

Exam trap

The trap here is that candidates may assume memory errors always require code optimization (e.g., batching or partitioning), but the question explicitly asks for minimal code changes, making resource scaling the correct answer.

How to eliminate wrong answers

Option A is wrong because switching from Spark to Python Shell job type would reduce parallelism and memory capacity, as Python Shell runs on a single node with limited resources, making it unsuitable for large-scale data transformations. Option B is wrong because implementing batch processing with smaller file sizes would require significant code changes to split and manage files, contradicting the 'minimal code changes' requirement, and does not address the root cause of insufficient memory allocation. Option D is wrong because using Redshift Spectrum to query data directly from S3 bypasses the Glue ETL job entirely, which is a different architectural approach that does not resolve the memory error in the existing Glue job and may introduce new costs and complexity.

439
Multi-Selecthard

A data engineer is building a data ingestion pipeline using AWS Glue. The source is an Amazon DynamoDB table, and the target is an Amazon S3 data lake in Parquet format. The pipeline must handle large volumes and ensure exactly-once processing. Which THREE features should the engineer use together to achieve this? (Choose THREE.)

Select 3 answers
A.Use Amazon Kinesis Data Streams to capture DynamoDB Streams changes.
B.Configure the Glue job to convert data to Parquet format.
C.Use Amazon S3 Object Lambda to transform data on the fly.
D.Enable job bookmarks in the Glue job to track processed items.
E.Use DynamoDB's export to S3 feature to get a full snapshot.
AnswersB, D, E

Parquet is columnar and efficient for analytics.

Why this answer

Converting data to Parquet format is a core requirement for an S3 data lake, as Parquet offers columnar storage, compression, and efficient querying via services like Amazon Athena and Amazon Redshift Spectrum. AWS Glue natively supports Parquet as an output format, enabling the engineer to specify it in the job's output schema or transformation logic.

Exam trap

The trap here is that candidates often confuse streaming services like Kinesis Data Streams with batch processing, assuming they are required for exactly-once guarantees, when in fact AWS Glue job bookmarks combined with DynamoDB Streams or export to S3 provide a simpler and more reliable solution.

440
MCQmedium

A data engineer uses AWS Glue to process data from S3. The Glue job frequently fails with 'Out of Memory' errors. The job reads several large compressed files. What is the MOST effective way to resolve this issue without changing the code?

A.Increase the number of G.1X workers or use G.2X workers
B.Convert the compressed files to uncompressed format before processing
C.Repartition the data to fewer partitions
D.Increase the job timeout setting
AnswerA

Glue workers hold the memory used for shuffles and decompression, so adding G.1X workers or switching to G.2X doubles per-worker memory and vCPU. That relieves the out-of-memory condition caused by large compressed files without editing the job script.

Why this answer

Increasing the number of G.1X workers or switching to G.2X workers directly addresses the 'Out of Memory' errors by allocating more memory per Spark executor. G.1X provides 16 GB of memory per worker, while G.2X provides 32 GB, which is critical when processing large compressed files because decompression and transformation require additional heap space. This approach resolves the issue without modifying the job code, as it only changes the resource configuration.

Exam trap

The trap here is that candidates often confuse 'Out of Memory' errors with performance issues and choose to reduce parallelism (Option C) or increase timeout (Option D), not realizing that memory exhaustion requires more memory per executor, not fewer tasks or longer runtime.

How to eliminate wrong answers

Option B is wrong because converting compressed files to uncompressed format would increase the data volume read from S3, potentially worsening memory pressure and increasing I/O costs, and it requires code changes to handle the new format. Option C is wrong because repartitioning to fewer partitions reduces parallelism, causing each Spark task to process more data, which would exacerbate memory issues rather than resolve them. Option D is wrong because increasing the job timeout setting only extends the maximum runtime before the job is killed; it does not address the underlying memory exhaustion, so the job will still fail with 'Out of Memory' errors.

441
MCQeasy

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed in real-time and then stored in Amazon S3. Which AWS service should be used to perform the transformation?

A.AWS Glue
B.Amazon EMR
C.Amazon Kinesis Data Analytics
D.Amazon Athena
AnswerC

Amazon Kinesis Data Analytics runs Apache Flink SQL or applications directly against Kinesis Data Streams, performing stateful real-time transformation before output is delivered onward to Amazon S3. It satisfies the stem's requirement for in-flight transformation of IoT telemetry without managing servers.

Why this answer

Amazon Kinesis Data Analytics (now part of Amazon Managed Service for Apache Flink) is the correct choice because it can consume streaming data from Kinesis Data Streams, apply real-time transformations using SQL or Apache Flink, and then output the transformed data to destinations like Amazon S3. This directly meets the requirement for real-time transformation of streaming IoT data before storage.

Exam trap

The trap here is that candidates often confuse AWS Glue's batch ETL capabilities with real-time streaming, leading them to select Glue despite its lack of native support for live Kinesis Data Streams processing.

How to eliminate wrong answers

Option A is wrong because AWS Glue is a serverless ETL service designed for batch processing and cataloging data in data lakes, not for real-time stream transformations; it cannot directly process live Kinesis Data Streams with sub-second latency. Option B is wrong because Amazon EMR is a big data platform for running frameworks like Apache Spark or Hadoop on clusters, which is overkill and not optimized for lightweight, continuous real-time transformations on a single Kinesis stream; it introduces cluster management overhead and higher latency. Option D is wrong because Amazon Athena is an interactive query service for analyzing data already stored in S3 using SQL, not for transforming streaming data in motion; it cannot ingest or process live Kinesis Data Streams.

442
MCQmedium

A company uses AWS DMS to migrate an on-premises PostgreSQL database to Amazon RDS for PostgreSQL. After initial load, ongoing replication is set up. The replication task shows 'Task status: failed with error: The specified LSN is not available in the source database logs.' What is the most likely cause?

A.DMS does not support PostgreSQL as a source for ongoing replication.
B.The source database's network security group blocks outbound traffic to DMS.
C.The full load was incomplete, preventing CDC from starting.
D.The source database's WAL retention period is too short, and required logs have been purged.
AnswerD

PostgreSQL logical replication requires the source WAL segments containing the task's restart LSN. If wal_keep_size or the replication slot's retention is too small, those segments are recycled, so DMS cannot resume and reports the LSN as unavailable.

Why this answer

The error 'The specified LSN is not available in the source database logs' indicates that AWS DMS is trying to read a Write-Ahead Log (WAL) position that has already been purged. PostgreSQL sources require sufficient WAL retention to allow DMS to catch up during ongoing replication (CDC). If the WAL segments are recycled or removed before DMS reads them, the replication task fails with this specific LSN error.

Exam trap

The trap here is that candidates confuse connectivity issues (Option B) or general CDC support (Option A) with the specific LSN error, which is a WAL retention problem unique to PostgreSQL logical replication.

How to eliminate wrong answers

Option A is wrong because AWS DMS fully supports PostgreSQL as a source for ongoing replication using logical replication slots and WAL-based CDC. Option B is wrong because network security group blocks would cause connection timeout or 'Unable to connect' errors, not an LSN availability error. Option C is wrong because an incomplete full load would prevent the task from starting CDC at all, but the error message specifically refers to missing LSN in logs, which occurs after CDC has begun and the source WAL has been purged.

443
MCQeasy

Refer to the exhibit. A data engineer runs the above CLI command to find files smaller than 1000 bytes in a bucket. The command returns an empty array, but the engineer knows there are small files. What is the issue?

A.The prefix is incorrect; it should be 'logs/2023/01/01/'.
B.The bucket policy does not allow listing objects.
C.The query syntax is invalid; use a filter instead.
D.The Size is compared as a string, not an integer; remove quotes around '1000'.
AnswerD

JMESPath comparison requires numeric types.

Why this answer

In the AWS CLI `list-objects-v2` command with `--query`, the `Size` field is a numeric value, but the query string `Size < '1000'` compares it as a string. This causes a lexicographic comparison, so files with sizes like '900' would be correctly matched, but any size with more digits (e.g., '1000' itself or '999') may fail due to string ordering. Removing the quotes around `1000` treats it as an integer, enabling proper numeric comparison.

Exam trap

The DEA-C01 exam often tests the subtle distinction between string and numeric comparisons in JMESPath queries, where candidates assume quoted values are automatically coerced to numbers, but in reality, quotes force string comparison.

How to eliminate wrong answers

Option A is wrong because the prefix 'logs/2023/01/01/' is not necessarily incorrect; the engineer knows small files exist, and the prefix is just a filter—if files are under that prefix, the issue is not the prefix. Option B is wrong because if the bucket policy did not allow listing objects, the CLI command would return an access denied error, not an empty array. Option C is wrong because the query syntax using `--query` with JMESPath expressions is valid; a filter is not required for this comparison, and the syntax `Size < '1000'` is syntactically correct but semantically wrong due to type coercion.

444
MCQhard

A data engineer is using AWS Glue to process a large dataset stored in Amazon S3. The dataset is partitioned by year, month, and day. The engineer notices that the Glue job is taking a long time and consuming many DPUs. The job reads all partitions, filters the data, and writes the result to another S3 location. The engineer wants to optimize the job to process only the required partitions and reduce cost. Which action should the engineer take?

A.Convert the dataset to Parquet format before processing.
B.Increase the number of DPUs to speed up the job.
C.Enable AWS Glue job bookmarks to track processed partitions.
D.Use predicate pushdown in the Glue job by specifying a push_down_predicate when reading from the AWS Glue Data Catalog.
AnswerD

Predicate pushdown allows the Glue job to filter partitions at the source based on the push_down_predicate parameter. When reading from the Data Catalog, you can specify a condition like 'year=2023 and month=01' to read only those partitions. This reduces the amount of data read and processed, improving performance and reducing DPU consumption. It is the recommended way to optimize partition pruning in Glue.

Why this answer

Predicate pushdown in AWS Glue allows the job to read only the partitions that match a specified condition, reducing data scanned and DPU usage. By using push_down_predicate when reading from the Data Catalog, the job filters at the source. Other options either do not reduce data read (bookmarks, more DPUs) or are not sufficient alone (Parquet conversion).

Exam trap

The trap here is thinking that job bookmarks or more DPUs will reduce the data read; bookmarks only avoid reprocessing, and more DPUs increase cost without reducing data volume.

445
MCQhard

A company uses Amazon Kinesis Data Analytics for real-time anomaly detection on clickstream data. The application uses a sliding window of 1 minute. The data engineer notices that the application is producing incorrect results because late-arriving records are not being handled properly. What should the data engineer do to ensure late records are included in the window calculations?

A.Use a Kinesis Data Firehose to buffer the data and then send to Kinesis Data Analytics.
B.Increase the watermark delay in the Kinesis Data Analytics application to allow more time for late records.
C.Increase the window size from 1 minute to 2 minutes.
D.Increase the retention period of the Kinesis stream to 7 days.
AnswerB

Increasing the watermark delay lets the application tolerate out-of-order events by extending how long it waits before closing a window, so late-arriving clickstream records still fall inside the 1-minute sliding window and are included in the anomaly calculations. This directly satisfies the requirement to handle late data correctly.

Why this answer

Kinesis Data Analytics uses watermarks to track event time progress and determine when to finalize window calculations. Increasing the watermark delay allows the application to wait longer for late-arriving records before closing the window, ensuring they are included in the aggregation.

Exam trap

The trap here is that candidates confuse stream retention (how long data is stored) with watermark delay (how long the application waits for late events), leading them to incorrectly choose option D.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Firehose is a delivery stream that buffers data for loading into destinations like S3 or Redshift; it does not provide late-record handling logic for Kinesis Data Analytics windows. Option C is wrong because increasing the window size from 1 minute to 2 minutes does not address late arrivals—it simply aggregates over a longer period, which can still miss records that arrive after the window's end time. Option D is wrong because increasing the retention period of the Kinesis stream to 7 days only affects how long data is stored in the stream, not how the analytics application handles late-arriving records within its window computations.

446
MCQhard

A data engineer is using AWS Glue to read data from an Amazon Kinesis Data Stream. The Glue job is configured to process the stream in micro-batches. The engineer notices that the job is not processing all records and sometimes skips data. The Kinesis stream has multiple shards, and the Glue job is using the 'kinesis' connection type. What is the most likely cause of the missing records?

A.The Glue job is reading from the stream using the 'GetRecords' API without specifying a shard iterator, causing it to miss records.
B.The Glue job is not checkpointing its progress correctly, so it reprocesses or skips records when the job restarts or fails.
C.The Glue job is not configured with the correct starting position, so it starts reading from the latest record instead of the earliest.
D.The Glue job is using the 'kinesis' connection but has not enabled the '--enable-auto-scaling' option, causing it to under-provision shards.
AnswerB

AWS Glue uses checkpoints to track the last processed record in a Kinesis stream. If checkpointing is not configured or fails, the job may start from an incorrect position after a restart, leading to skipped or reprocessed records. This is a common cause of missing data in stream processing jobs. Ensuring proper checkpointing is essential for exactly-once or at-least-once processing.

Why this answer

Checkpointing is critical for AWS Glue jobs reading from Kinesis Data Streams. Without proper checkpointing, the job cannot track which records have been processed, leading to data loss or duplication upon restart. The most likely cause of missing records is incorrect or disabled checkpointing, which causes the job to resume from an outdated position.

Exam trap

The trap here is focusing on starting position or scaling issues, when the core problem is often checkpointing, which ensures continuity across job runs.

447
MCQhard

A data engineer is orchestrating a multi-step ingestion workflow where CSV files land in Amazon S3, an AWS Glue job transforms them, and the output is loaded into Amazon Redshift. The engineer wants conditional branching, retry logic, and the ability to pass parameters between steps. Which AWS service should be used to orchestrate this workflow?

A.AWS Glue workflows with triggers that chain crawlers and jobs together.
B.Amazon EventBridge rules that react to S3 events and start the Glue job.
C.AWS Step Functions with a state machine that invokes Glue and Redshift tasks.
D.AWS Lambda functions chained by direct invocations to run each pipeline step.
AnswerC

Step Functions supports conditional branching through choice states, configurable retry and catch logic, and passing data between states. It can invoke AWS Glue jobs and Amazon Redshift operations, making it suitable for orchestrating a multi-step ingestion workflow with the exact control-flow features described. This directly satisfies branching, retries, and parameter passing across the pipeline steps.

Why this answer

Step Functions is a state-machine-based orchestrator that natively supports choice states for conditional branching, retry and catch blocks for fault tolerance, and input and output passing between states. It integrates with AWS Glue and Amazon Redshift, so the entire CSV-to-transformed-to-Redshift workflow can be modeled, monitored, and retried as a single managed execution with clear visibility into each step.

Exam trap

The trap here is assuming that Glue triggers or EventBridge rules provide full workflow orchestration, when they lack rich branching and retry semantics.

← PreviousPage 6 of 6 · 447 questions total

Ready to test yourself?

Try a timed practice session using only Data Ingestion Transformation questions.