Knowledge + Practice

CCNA Data Ingestion and Transformation Questions

75 of 591 questions · Page 7/8 · Data Ingestion and Transformation · Answers revealed

Practice these questions Domain overview All questions

451

MCQeasy

A data engineering team needs to transform CSV files stored in Amazon S3 into Parquet format using AWS Glue. The files are partitioned by date and are updated hourly. Which AWS Glue feature should be used to automatically detect the schema and partition structure?

A.AWS Glue Crawler

B.AWS Glue DataBrew

C.AWS Lake Formation

D.Amazon Athena

AnswerA

Discovers schema and partitions automatically.

Why this answer

AWS Glue Crawler is the correct choice because it automatically scans data in S3, infers the schema (including data types), and detects the partition structure (e.g., date-based partitions like year/month/day) by examining the folder hierarchy. It then populates the AWS Glue Data Catalog with metadata, enabling ETL jobs to read the data without manual schema definition.

Exam trap

AWS often tests the distinction between tools that discover metadata (Crawler) versus tools that consume or transform data (Athena, DataBrew), leading candidates to pick Athena because it can query partitioned data, but it cannot automatically detect the partition structure without a pre-existing catalog.

How to eliminate wrong answers

Option B (AWS Glue DataBrew) is wrong because it is a visual data preparation tool for cleaning and normalizing data, not for automatic schema or partition detection. Option C (AWS Lake Formation) is wrong because it provides centralized security and governance for data lakes, but it does not perform schema discovery or partition detection itself. Option D (Amazon Athena) is wrong because it is a query engine that can read data from the Glue Data Catalog, but it does not automatically detect schemas or partitions; it relies on existing catalog metadata.

Practice this question →

452

Multi-Selecthard

A company is ingesting IoT sensor data into Amazon Kinesis Data Streams. Each sensor sends a JSON payload every second. The data must be transformed and aggregated in real-time before being stored in Amazon DynamoDB. Which THREE services should be used together in the pipeline? (Choose THREE.)

Select 3 answers

A.AWS Lambda

B.Amazon Kinesis Data Analytics

C.Amazon S3

D.Amazon Kinesis Data Streams

E.Amazon Kinesis Data Firehose

AnswersA, B, D

AWS Lambda can be used as a consumer of Kinesis Data Streams to perform lightweight, per-record transformations on the JSON payloads in real-time, but it is not sufficient alone for aggregation.

Why this answer

Amazon Kinesis Data Streams (D) is the ingestion layer that captures the JSON payloads from IoT sensors in real-time, ensuring data is available for processing. AWS Lambda (A) can be used as a consumer of the stream to perform lightweight, per-record transformations, such as filtering or enriching the JSON payloads. Amazon Kinesis Data Analytics (B) is required for real-time aggregation and complex transformations using SQL or Apache Flink, enabling calculations like averages or counts per second before storing in DynamoDB.

Together, these three services form a complete real-time pipeline: ingest with Kinesis Data Streams, transform/aggregate with Kinesis Data Analytics, and optionally further transform with Lambda before writing to DynamoDB.

Exam trap

The trap is that candidates often confuse Amazon Kinesis Data Firehose with a real-time processing service, but Firehose is a delivery service with near-real-time latency (minimum 60 seconds) and cannot perform per-second aggregations. Additionally, some might think only Lambda is needed for transformation, but Kinesis Data Analytics is better suited for real-time aggregations like sliding windows.

Practice this question →

453

Multi-Selectmedium

A company is using AWS Glue ETL to process data from Amazon RDS for MySQL to Amazon S3. The job runs daily and takes 2 hours to complete. The engineer wants to improve performance without increasing cost significantly. Which TWO actions should the engineer take? (Choose TWO.)

Select 2 answers

A.Switch to a smaller worker type (e.g., G.1X instead of G.2X).

B.Use Spark DataFrames instead of DynamicFrames.

C.Enable 'Auto Scaling' in the Glue job configuration.

D.Add a partition column to the source table based on a date column.

E.Increase the number of Glue DPUs.

AnswersD, E

Partitioning allows Glue to read data in parallel.

Why this answer

Adding a partition column (e.g., based on a date column) to the source table enables AWS Glue to use partition pruning during the read phase. This reduces the amount of data scanned and processed by the ETL job, directly improving performance without increasing cost. Partitioning is a common optimization for large datasets in RDS or S3-based sources.

Exam trap

The trap here is that candidates often confuse 'Auto Scaling' with a performance improvement feature, but Auto Scaling only adjusts resources to match workload, not reduce runtime; the real performance gain comes from reducing data volume via partitioning.

Practice this question →

454

MCQhard

A data pipeline ingests JSON data from an S3 bucket using AWS Glue. The JSON files contain nested structures, and the team wants to flatten them for analysis in Amazon Athena. Which Glue transformation is most appropriate?

A.Filter

B.Join

C.Map

D.Relationalize

AnswerD

Flattens nested JSON into separate tables.

Why this answer

Relationalize is specifically designed to flatten nested JSON into relational tables. Option A (Map) applies a function to each record. Option B (Filter) removes records.

Option C (Join) combines datasets.

Practice this question →

455

MCQmedium

A company uses Kinesis Data Streams to ingest IoT data. The data volume varies, and occasionally the shard write throughput is exceeded, causing ProvisionedThroughputExceeded exceptions. The data engineer needs to handle these spikes without losing data. Which approach is most cost-effective and requires minimal code changes?

A.Implement custom retry logic using the Kinesis Client Library with exponential backoff

B.Increase the number of shards to handle peak throughput

C.Use Kinesis Data Firehose as a consumer with retries and buffer settings

D.Send data to an SQS queue first, then have a Lambda function write to Kinesis

AnswerC

Firehose can buffer data and retry, handling spikes with minimal code changes.

Why this answer

Kinesis Data Firehose is the most cost-effective solution because it can be configured as a consumer of the Kinesis Data Stream with built-in retry logic and buffer settings (e.g., buffer size up to 128 MB or buffer interval up to 900 seconds). This handles throughput spikes by buffering data and retrying failed writes without requiring custom code, and it scales automatically without the need to manage shard counts.

Exam trap

The trap here is that candidates often assume increasing shards (Option B) is the only way to handle throughput spikes, but the question emphasizes cost-effectiveness and minimal code changes, making Firehose's buffering and retry mechanism the optimal choice without over-provisioning.

How to eliminate wrong answers

Option A is wrong because implementing custom retry logic with the Kinesis Client Library (KCL) requires significant code changes and ongoing maintenance, and it does not address the root cause of shard throughput limits—it only retries failed writes, which can still lead to data loss if retries are exhausted. Option B is wrong because increasing the number of shards to handle peak throughput is not cost-effective; it over-provisions resources for rare spikes, leading to higher costs during normal operation. Option D is wrong because sending data to an SQS queue first adds latency, complexity, and cost (SQS charges per request), and requires a Lambda function to bridge the two services, which introduces additional code changes and potential points of failure.

Practice this question →

456

MCQeasy

A company is streaming data from an application to Amazon Kinesis Data Streams. The data must be transformed in real time and then stored in Amazon S3 in Parquet format. Which AWS service should be used for the transformation step?

A.Amazon Kinesis Data Firehose with a Lambda transformation.

B.Amazon EMR running Apache Spark Streaming.

C.Amazon Kinesis Data Analytics for Apache Flink.

D.AWS Lambda with a Kinesis trigger.

AnswerC

Amazon Kinesis Data Analytics for Apache Flink is a serverless service that can run Apache Flink applications to perform real-time transformations on streaming data and can output to Kinesis Data Firehose for delivery to S3 in Parquet format.

Why this answer

Amazon Kinesis Data Analytics for Apache Flink is a serverless service that can run Apache Flink applications to perform real-time transformations on streaming data. Option A (Amazon Kinesis Data Firehose with a Lambda transformation) is primarily for loading data and has limitations for complex transformations. Option B (Amazon EMR running Apache Spark Streaming) can work but adds management overhead and is not the simplest managed service.

Option D (AWS Lambda with a Kinesis trigger) is suitable for lightweight transformations but may hit execution time limits for complex or long-running transformations.

Practice this question →

457

MCQmedium

A data engineer needs to ingest streaming data from thousands of devices sending JSON messages via HTTP POST. The data should be stored in Amazon S3 with minimal latency and also be available for real-time analytics. Which combination of services is MOST appropriate?

A.Amazon DynamoDB with DynamoDB Streams and Lambda.

B.Amazon SQS and AWS Lambda to write to S3.

C.AWS Lambda directly writing to S3 via API Gateway.

D.Amazon API Gateway, Amazon Kinesis Data Streams, and Kinesis Data Firehose.

AnswerD

API Gateway receives POST, sends to Kinesis for real-time analytics, and Firehose batches to S3.

Why this answer

It combines Amazon API Gateway to ingest HTTP POST messages, Amazon Kinesis Data Streams for real-time analytics, and Kinesis Data Firehose to deliver the data to Amazon S3 with minimal latency. Option A (DynamoDB with Streams) is meant for database change tracking, not direct HTTP streaming. Option B (SQS + Lambda) introduces latency and lacks native streaming analytics.

Option C (Lambda directly via API Gateway) does not provide the buffering and streaming analytics capabilities needed.

Practice this question →

458

Multi-Selectmedium

A company ingests IoT sensor data into Kinesis Data Streams. The data is then processed by a Lambda function that aggregates readings and writes to DynamoDB. The Lambda function is experiencing high error rates due to throttling. Which TWO actions would reduce throttling?

Select 2 answers

A.Increase the number of shards in the Kinesis stream.

B.Increase the batch size in the Lambda event source mapping.

C.Decrease the batch window in the Lambda event source mapping.

D.Configure DynamoDB to use on-demand capacity mode.

E.Increase the Lambda reserved concurrency to 1000.

AnswersB, D

Larger batches mean fewer invocations, reducing throttling.

Why this answer

Increasing the batch size in the Lambda event source mapping allows each invocation to process more records from the Kinesis stream, reducing the number of concurrent Lambda invocations and thus lowering the risk of throttling. Option D is correct because switching DynamoDB to on-demand capacity mode eliminates write capacity limits, preventing throttling on the DynamoDB side that can cause Lambda retries and backpressure.

Exam trap

The trap here is that candidates often assume increasing shards (Option A) always improves throughput, but in a Lambda-integrated Kinesis stream, more shards mean more concurrent invocations, which can actually increase throttling risk.

Practice this question →

459

Multi-Selecthard

A company uses AWS Glue to transform data stored in S3. The Glue job runs daily and processes data in the range of hundreds of GB. The data engineer wants to optimize the job for cost and performance. Which THREE actions should be taken? (Choose THREE.)

Select 3 answers

A.Store intermediate data in HDFS on Amazon EMR

B.Increase the number of DPUs for the job

C.Reduce the number of DPUs to save cost

D.Use columnar data formats such as Parquet

E.Partition the data by date or other high-cardinality columns

AnswersB, D, E

More DPUs can reduce runtime, improving cost if job runs shorter.

Why this answer

Increasing the number of DPUs (Data Processing Units) for an AWS Glue job can improve performance by enabling parallel processing of large datasets (hundreds of GB). AWS Glue allocates resources in increments of DPUs, where each DPU provides 4 vCPU and 16 GB of memory; scaling out DPUs reduces execution time, which can lower overall cost if the job runs fewer minutes, balancing cost and performance.

Exam trap

The trap here is that candidates mistakenly think reducing DPUs always saves cost, but AWS Glue bills by DPU-hour, so longer runtimes from fewer DPUs can actually increase cost, and the question explicitly asks for both cost and performance optimization.

Practice this question →

460

Multi-Selectmedium

Which TWO options are valid methods to ingest on-premises relational database data into Amazon S3 for analytics? (Choose 2.)

Select 2 answers

A.AWS Snowball Edge

B.AWS Glue ETL job with JDBC connection to source

C.Amazon Kinesis Data Streams with Direct Put

D.AWS Database Migration Service (DMS) with S3 target

E.Amazon AppFlow

AnswersB, D

Glue can read from JDBC and write to S3.

Why this answer

AWS Glue ETL jobs can connect to on-premises relational databases via JDBC, extract data, and write it directly to Amazon S3 in formats like Parquet or ORC. This is a fully managed, serverless approach suitable for batch ingestion and transformation of structured data for analytics.

Exam trap

The trap here is that candidates confuse AWS Glue ETL (which uses JDBC for batch extraction) with Amazon Kinesis (which is for streaming), or they overlook that AWS DMS is a dedicated service for database migration and replication to S3, while Snowball Edge is for offline bulk transfer, not live ingestion.

Practice this question →

461

MCQmedium

The exhibit shows an AWS CLI command and its output. A data engineer wants to copy only objects larger than 10 MB from the S3 bucket to another bucket for processing. Which approach should be used to automate this task?

A.Use S3 replication rules to replicate objects above 10 MB

B.Use AWS CLI with a script to filter and copy objects

C.Use S3 Inventory to generate a list and then copy

D.Use AWS Lambda with S3 event notifications

AnswerB

The CLI can filter by size and copy objects using a script.

Why this answer

The command lists objects larger than 10 MB. To automate copying, a script using AWS CLI with the --query parameter can filter and copy. Using S3 Batch Operations allows performing actions on a list of objects.

The correct approach is to use AWS CLI with a script that iterates over the filtered list and uses aws s3 cp. S3 replication is for continuous sync, not one-time copy. Lambda with S3 events triggers only on new objects, not existing ones.

S3 Inventory provides metadata but not direct copy.

Practice this question →

462

MCQeasy

A data engineer needs to ingest streaming data from thousands of IoT devices into AWS for real-time processing. The data volume peaks at 5 GB/min. Which AWS service should be used as the ingestion endpoint?

A.Amazon Kinesis Data Streams

B.AWS Glue

C.Amazon S3

D.AWS Lambda

AnswerA

Kinesis Data Streams is built for real-time streaming data ingestion.

Why this answer

Amazon Kinesis Data Streams is designed for real-time data ingestion at scale, supporting throughput of up to 1 MB/s or 1,000 records/s per shard. With a peak of 5 GB/min (~83 MB/s), you can horizontally scale by adding shards to meet the required throughput, making it the ideal ingestion endpoint for high-volume streaming IoT data.

Exam trap

The trap here is that candidates often confuse AWS Glue's streaming ETL capability (which reads from a stream but does not ingest) with a direct ingestion endpoint, or they assume S3's high durability makes it suitable for real-time ingestion, ignoring its lack of streaming semantics and low-latency write guarantees.

How to eliminate wrong answers

Option B (AWS Glue) is wrong because it is a serverless ETL service for batch data processing and cataloging, not a real-time streaming ingestion endpoint; it cannot handle continuous, high-velocity data streams. Option C (Amazon S3) is wrong because it is an object storage service that does not provide real-time ingestion or streaming capabilities; data must be written via API calls or batch uploads, and it lacks the low-latency, ordered replay features needed for streaming. Option D (AWS Lambda) is wrong because it is a compute service for running code in response to events, not a dedicated ingestion endpoint; it has a maximum invocation payload limit of 256 KB and is not designed to buffer or scale for sustained 5 GB/min throughput.

Practice this question →

463

Multi-Selectmedium

A company is building a data lake on Amazon S3 and needs to ingest data from multiple sources. Which of the following AWS services can be used to ingest and transform data in near real-time? (Select TWO.)

Select 2 answers

A.AWS Glue

B.Amazon Kinesis Data Firehose

C.Amazon Athena

D.AWS Step Functions

E.Amazon Simple Queue Service (SQS)

AnswersA, B

Can be used for ETL jobs triggered by S3 events.

Why this answer

AWS Glue is correct because it provides a serverless ETL (Extract, Transform, Load) service that can ingest data from various sources and transform it in near real-time using its streaming ETL capabilities. Glue can consume data from Amazon Kinesis Data Streams or Apache Kafka, apply transformations using Apache Spark, and write the results to Amazon S3 or other destinations, making it suitable for near real-time data ingestion and transformation.

Exam trap

The trap here is that candidates often confuse Amazon Athena (a query engine) with an ingestion service, or assume SQS alone can perform transformations, when neither service is designed for near real-time data ingestion and transformation into a data lake.

Practice this question →

464

Multi-Selecthard

A company is migrating on-premises Apache Kafka clusters to Amazon MSK. The migration must be seamless with no data loss. The team is using MirrorMaker 2 to replicate data from on-premises Kafka to MSK. Which THREE configurations are necessary to ensure exactly-once semantics and minimal downtime? (Choose three.)

Select 3 answers

A.Set auto.create.topics.enable to false to prevent automatic topic creation.

B.Set offsets.topic.replication.factor to 3 for the consumer offsets topic.

C.Set replication.factor to 3 on the MSK cluster.

D.Enable TLS encryption between on-premises and MSK.

E.Configure MirrorMaker 2 to use exactly-once semantics.

AnswersB, C, E

Ensures offset data is durable and replicated.

Why this answer

To ensure exactly-once semantics and minimal downtime when migrating on-premises Kafka to Amazon MSK using MirrorMaker 2, three configurations are essential. First, setting `offsets.topic.replication.factor` to 3 (option B) ensures that consumer offsets are replicated across multiple brokers, preventing loss of offset information during broker failures, which supports exactly-once semantics by maintaining accurate consumer state. Second, setting `replication.factor` to 3 on the MSK cluster (option C) increases data durability and availability, reducing the risk of data loss during the migration.

Third, configuring MirrorMaker 2 to use exactly-once semantics (option E) prevents duplicate message delivery by enabling idempotent producers and transactional semantics in the replication flow. Options A and D are not required: `auto.create.topics.enable` should typically be left as default (true) to allow dynamic topic creation, and TLS encryption (option D) is a security measure but not necessary for exactly-once semantics or minimal downtime.

Practice this question →

465

MCQhard

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The data is then processed by a Kinesis Data Analytics application running SQL queries. The analytics application is falling behind and processing records with increasing latency. The stream has 4 shards, and the average record size is 5 KB. What is the MOST effective way to improve processing latency?

A.Increase the number of shards in the Kinesis stream to 8.

B.Enable enhanced fan-out on the Kinesis stream for the analytics application.

C.Increase the parallelism of the Kinesis Data Analytics application.

D.Increase the retention period of the Kinesis stream to 7 days.

AnswerC

More parallelism allows the application to process more records per unit time.

Why this answer

Increasing the parallelism of the Kinesis Data Analytics application (e.g., by increasing the number of in-application streams or ParallelismPerKPU) allows it to consume from the stream faster, reducing latency. Option A is wrong because 5 KB is well below the 1 MB/s shard limit, so increasing shards is unnecessary. Option B is wrong because enhanced fan-out is for consumers that need low latency, but does not help if the application is CPU-bound.

Option D is wrong because increasing the retention period does not affect processing speed; it only keeps data longer.

Practice this question →

466

MCQeasy

A company wants to move data from an Amazon RDS for MySQL database to Amazon Redshift for analytics. The data needs to be refreshed daily. Which AWS service is best suited for this?

A.AWS Database Migration Service (DMS)

B.AWS Glue

C.Amazon EMR

D.Amazon Athena

AnswerB

Can extract from RDS and load to Redshift with scheduling.

Why this answer

(AWS Glue) is correct because it is a fully managed ETL service that can connect to RDS for MySQL as a source and Amazon Redshift as a target, and it supports scheduling for daily refreshes. Option A (AWS DMS) is designed for database migration and continuous replication, not for scheduled batch loads. Option C (Amazon EMR) is for big data processing using Hadoop/Spark, which is overkill for this simple transfer.

Option D (Amazon Athena) is an interactive query service for data in Amazon S3, not for moving data between databases.

Practice this question →

467

Multi-Selecteasy

A company needs to ingest data from an on-premises Oracle database into Amazon S3 for analytics. The data volume is about 1 TB initially, with daily incremental updates of about 10 GB. Which TWO services can be combined to achieve this with minimal custom code?

Select 2 answers

A.AWS Glue

B.Amazon Kinesis Data Streams

C.Amazon Athena

D.Amazon S3

E.AWS Database Migration Service (DMS)

AnswersD, E

Target for the ingested data.

Why this answer

The correct combination is AWS DMS (Database Migration Service) for migration and Amazon S3 as the target. DMS can perform full load and ongoing replication with minimal custom code, making it suitable for the initial 1 TB load and daily 10 GB increments. AWS Glue (option A) could be used but often requires more custom code for change data capture (CDC) compared to DMS.

Amazon Kinesis Data Streams (option B) is designed for real-time streaming data, not database migration. Amazon Athena (option C) is a query service, not a data movement service. Therefore, the correct answers are D (Amazon S3) and E (AWS DMS).

Practice this question →

468

Multi-Selectmedium

A company is ingesting real-time clickstream data into Amazon S3 using Amazon Kinesis Data Firehose. The data is semi-structured and the company wants to transform the data into Parquet format and partition it by year, month, day, and hour. Which TWO steps should be taken to achieve this? (Choose TWO.)

Select 2 answers

A.Set up an Amazon S3 event notification to trigger an AWS Lambda function that partitions the data after delivery.

B.Enable dynamic partitioning in Kinesis Data Firehose and specify the partition keys as year, month, day, hour extracted from the data.

C.Use an AWS Glue Crawler to infer the schema and automatically partition the data in S3.

D.Create an AWS Lambda function that transforms incoming records to Parquet and attach it to the Firehose delivery stream as a data transformation.

E.Configure Kinesis Data Firehose to convert the data to Parquet format using a schema from the AWS Glue Data Catalog.

AnswersB, D

Correct. Dynamic partitioning extracts partition keys from the data and creates S3 prefixes accordingly.

Why this answer

Kinesis Data Firehose's dynamic partitioning feature allows you to specify partition keys (year, month, day, hour) extracted from the incoming data, and Firehose will automatically create the corresponding S3 prefix structure (e.g., year=2024/month=01/day=15/hour=10/) during delivery. Option D is correct because to convert semi-structured data to Parquet format, you can attach an AWS Lambda function as a data transformation to Firehose, which converts each record to Parquet before delivery to S3. Option E is incorrect because while Kinesis Data Firehose does support converting data to Parquet format using a schema from the AWS Glue Data Catalog, this approach requires a pre-defined Glue schema and is less flexible for semi-structured data.

Moreover, the question does not mention any existing Glue Data Catalog, and the Lambda transformation in option D is a more direct and customizable method for the transformation needed.

Exam trap

AWS often tests the misconception that dynamic partitioning alone handles format conversion, but in reality, dynamic partitioning only manages the S3 prefix structure, while Parquet conversion requires a separate Lambda transformation or the use of Firehose's built-in Parquet conversion with a compatible input format.

Practice this question →

469

Multi-Selecteasy

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3 for analytics. The data changes frequently and the engineer wants to capture both initial load and incremental changes with minimal latency. Which TWO AWS services should be used together? (Choose TWO.)

Select 2 answers

A.AWS Database Migration Service (DMS)

B.AWS Lambda

C.AWS Glue

D.AWS Transfer Family

E.Amazon Kinesis Data Streams

AnswersA, E

DMS can perform ongoing replication from Oracle to S3.

Why this answer

AWS DMS (Option A) is correct because it can perform an initial load of data from an on-premises Oracle database to Amazon S3 and then continuously replicate incremental changes using change data capture (CDC) with minimal latency. Amazon Kinesis Data Streams (Option E) complements DMS by enabling near-real-time streaming of data changes, which can be further processed or stored in S3. Option B (Lambda) is not a direct replacement for database replication; it can process events but does not natively capture database changes.

Option C (Glue) is an ETL service, not designed for real-time replication. Option D (Transfer Family) is for file transfers, not database replication.

Practice this question →

470

Multi-Selecthard

Which THREE factors should a data engineer consider when choosing between AWS Glue and Amazon EMR for a data transformation job? (Choose three.)

Select 3 answers

A.The ability to output results to Amazon S3

B.The support for Apache Spark

C.The level of control over the execution environment and dependencies

D.The need for a serverless vs. cluster-based environment

E.The cost model: pay per DPU for Glue vs. per instance for EMR

AnswersC, D, E

EMR offers more control; Glue is less customizable.

Why this answer

When choosing between AWS Glue and Amazon EMR for a data transformation job, key considerations include: the level of control over the execution environment (EMR offers more customization, while Glue is managed), the deployment model (Glue is serverless, EMR is cluster-based), and the cost structure (Glue charges per DPU, EMR charges per EC2 instance). Options A and B are not differentiating factors because both services support Apache Spark and can output to S3.

Practice this question →

471

MCQhard

A company uses AWS DMS to migrate a 2 TB Oracle database to Amazon RDS for PostgreSQL. The migration is taking longer than expected. The task status shows 'Full load in progress' with a low 'Table throughput (rows/s)'. Which action would MOST improve throughput?

A.Enable Multi-AZ on the DMS replication instance

B.Change the target table preparation mode to 'Do nothing'

C.Increase the number of parallel tasks in the DMS task settings

D.Increase the number of shards in the source database

AnswerC

Parallel tasks allow concurrent loading of tables, increasing throughput.

Why this answer

The low 'Table throughput (rows/s)' during the full load phase indicates that the DMS task is not processing tables with enough parallelism. Increasing the number of parallel tasks in the DMS task settings allows the replication instance to load multiple tables concurrently, which directly improves throughput by utilizing available CPU and memory resources more efficiently.

Exam trap

The trap here is that candidates confuse 'parallel tasks' with 'Multi-AZ' or 'target table preparation mode', assuming that high availability or skipping table preparation will speed up data transfer, when in fact only increasing parallelism directly addresses low row throughput during full load.

How to eliminate wrong answers

Option A is wrong because enabling Multi-AZ on the DMS replication instance provides high availability and failover support, but does not increase throughput during full load; it may even reduce performance due to synchronous replication overhead. Option B is wrong because changing the target table preparation mode to 'Do nothing' only affects how DMS handles existing tables (e.g., truncate or drop), not the speed of data transfer; it does not address low row throughput. Option D is wrong because increasing the number of shards in the source database is a source-side change that does not directly affect DMS's ability to read and load data faster; DMS's throughput is limited by its own parallelism settings, not the source shard count.

Practice this question →

472

MCQmedium

A company uses AWS Glue ETL to transform data from Amazon RDS for MySQL to Amazon S3. The Glue job reads from a JDBC connection. The job runs once daily and processes all records, but the data volume is growing. Which change would improve performance and reduce costs?

A.Increase the number of DPUs for the Glue job

B.Switch to a Glue Python shell job

C.Use a higher JDBC fetch size

D.Enable Glue job bookmarking and set the job to process only new data

AnswerD

Bookmarking enables incremental loads.

Why this answer

Enabling Glue job bookmarking allows the job to process only new or changed data since the last run, rather than reprocessing the entire dataset. This reduces both the data volume read from the JDBC source and the transformation time, directly improving performance and lowering costs by minimizing DPU usage.

Exam trap

The trap here is that candidates often assume increasing parallelism (Option A) is the universal fix for performance, overlooking the fact that reducing the data volume processed (Option D) is a more fundamental and cost-effective optimization.

How to eliminate wrong answers

Option A is wrong because increasing the number of DPUs for the Glue job would increase parallelism and potentially speed up execution, but it does not address the root cause of reprocessing all records daily; it would only scale the cost linearly without reducing the data volume processed. Option B is wrong because a Glue Python shell job is designed for lightweight, single-node Python scripts and cannot handle JDBC connections or large-scale data transformations; it lacks the distributed processing capabilities of a full Glue ETL job. Option C is wrong because using a higher JDBC fetch size can improve the efficiency of reading rows from MySQL by reducing round trips, but it still processes all records every run and does not eliminate the overhead of scanning the entire table daily.

Practice this question →

473

MCQeasy

A company needs to ingest real-time clickstream data from a web application into Amazon S3 for analytics. The data must be available within minutes of generation. Which AWS service should be used to capture and deliver this streaming data?

A.Amazon RDS

B.Amazon Kinesis Data Firehose

C.AWS Glue

D.Amazon Simple Queue Service (SQS)

AnswerB

Correct: Kinesis Data Firehose captures streaming data and delivers it to S3 with low latency.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to capture, transform, and load streaming data into Amazon S3, Redshift, Elasticsearch, or Splunk in near real-time (typically within 60 seconds). It directly addresses the requirement for ingesting real-time clickstream data and delivering it to S3 within minutes, without requiring custom code or manual scaling.

Exam trap

The trap here is confusing Amazon Kinesis Data Streams (which requires custom consumers and is not directly integrated with S3) with Amazon Kinesis Data Firehose (which is purpose-built for automated delivery to destinations like S3), leading candidates to overlook the 'within minutes' requirement and choose a service that needs additional components.

How to eliminate wrong answers

Option A (Amazon RDS) is wrong because it is a relational database service for transactional workloads, not designed for streaming data ingestion or direct delivery to S3; it would require additional ETL processes to move data to S3. Option C (AWS Glue) is wrong because it is a serverless ETL service for batch data processing and cataloging, not for real-time streaming capture; it can process data from S3 but does not ingest streaming data directly. Option D (Amazon Simple Queue Service (SQS)) is wrong because it is a message queue service for decoupling application components, not a streaming data delivery service; it does not automatically write data to S3 and requires custom consumers to do so.

Practice this question →

474

MCQhard

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3 using AWS DMS. The change data capture (CDC) must be enabled to capture ongoing changes. Which additional AWS service is required to store the transaction logs for CDC?

A.Amazon RDS

B.Amazon S3

C.Amazon EBS

D.Amazon CloudWatch Logs

AnswerB

Amazon S3 is not required; DMS reads redo logs directly from the source database.

Why this answer

AWS DMS change data capture (CDC) for Oracle databases does not require an additional AWS service to store transaction logs. DMS reads the redo logs directly from the source Oracle database, either using LogMiner or binary reader. Therefore, none of the listed services are needed for storing transaction logs.

As none of the provided options are correct, this question contains no valid answer.

Practice this question →

475

MCQhard

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams. The stream has 10 shards, and the data volume is expected to grow by 50% over the next month. The engineer needs to ensure that the pipeline can scale without manual intervention. Which approach should be used?

A.Set up a CloudWatch Alarm to trigger a Lambda function to add shards

B.Use an Auto Scaling group to add more shards

C.Switch the Kinesis stream to on-demand capacity mode

D.Configure the stream to use a Lambda function that scales shards

AnswerC

On-demand mode automatically scales shards based on ingestion throughput.

Why this answer

Kinesis Data Streams on-demand capacity mode automatically scales the number of shards based on the incoming traffic pattern, eliminating the need for manual intervention. Option A is incorrect because CloudWatch Alarms can trigger a Lambda function to add shards via the UpdateShardCount API, but this approach requires custom code and does not provide automatic scaling without manual setup. Option B is incorrect because Auto Scaling groups are used for EC2 instances, not for Kinesis stream shards.

Option D is incorrect because Lambda functions can be used to scale shards programmatically, but this still requires custom implementation and is not a native automatic scaling feature.

Practice this question →

476

Multi-Selecteasy

A data engineer needs to ingest data from multiple on-premises relational databases into Amazon S3 for analytics. The data must be transformed and loaded daily. Which THREE AWS services should the engineer use together to build this pipeline? (Choose THREE.)

Select 3 answers

A.AWS Glue

B.AWS Glue Data Catalog

C.Amazon Athena

D.AWS Database Migration Service (DMS)

E.Amazon Kinesis Data Streams

AnswersA, B, D

Performs ETL transformations on the data.

Why this answer

AWS Glue is correct because it provides a serverless ETL (Extract, Transform, Load) service that can read data from Amazon S3, apply transformations (e.g., using PySpark or Scala), and write the transformed data back to S3. In this pipeline, AWS Glue jobs can be scheduled to run daily to perform the required transformations on the ingested data.

Exam trap

The DEA-C01 exam often tests the distinction between batch and streaming services; the trap here is that candidates might confuse Amazon Kinesis Data Streams (real-time) with a batch ingestion tool, or think Amazon Athena can perform ETL transformations when it is only a query engine.

Practice this question →

477

MCQeasy

A company wants to migrate on-premises data to Amazon S3 using AWS DataSync. The data is stored on an NFS file server and the total volume is 50 TB. The network bandwidth between the on-premises data center and AWS is 1 Gbps (gigabit per second). What is the primary factor that will determine the total time required for the initial data transfer?

A.The available network bandwidth between on-premises and AWS

B.The number of S3 buckets used as the destination

C.The average file size in the dataset

D.The IOPS (I/O operations per second) of the on-premises NFS server

AnswerA

With 50 TB and 1 Gbps, the theoretical minimum time is ~4.7 days; network bandwidth is the key constraint.

Why this answer

AWS DataSync transfers data over the network, so the primary constraint is the available bandwidth between the on-premises NFS server and AWS. With 50 TB of data and a 1 Gbps link, the theoretical minimum transfer time is approximately 50 TB * 8 / 1 Gbps = 400,000 seconds (~111 hours), but real-world throughput is lower due to protocol overhead, latency, and competing traffic. The network bandwidth directly dictates the maximum data transfer rate, making it the dominant factor for the initial transfer duration.

Exam trap

The trap here is that candidates may focus on the NFS server's IOPS or file size, assuming storage performance is the bottleneck, but the question explicitly provides a 1 Gbps bandwidth figure, signaling that network throughput is the key limiting factor for the initial transfer.

How to eliminate wrong answers

Option B is wrong because the number of S3 buckets does not affect transfer speed; DataSync can write to multiple buckets, but the throughput is still limited by the network pipe. Option C is wrong because while average file size can impact the number of file operations, DataSync uses parallel streams and can handle small files efficiently; the total data volume and bandwidth are the primary drivers, not file size. Option D is wrong because the NFS server's IOPS is rarely the bottleneck for a bulk transfer over a 1 Gbps link; DataSync reads files sequentially and the network bandwidth is typically the limiting factor, not the storage I/O performance.

Practice this question →

478

MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver streaming data to an Amazon S3 bucket. The delivery occasionally fails due to 'ThrottlingException' from S3. What should the team do to resolve this issue without losing data?

A.Enable S3 Transfer Acceleration on the destination bucket.

B.Disable error logging in Firehose to reduce API calls.

C.Configure Firehose to deliver data to Amazon DynamoDB instead.

D.Increase the Firehose buffer size and buffer interval to reduce the number of S3 PUT requests.

AnswerD

Larger buffers mean fewer writes, reducing throttling risk.

Why this answer

The ThrottlingException from S3 indicates that Kinesis Data Firehose is sending too many PUT requests to S3. Increasing the buffer size or buffer interval causes Firehose to accumulate more records before writing, reducing the number of PUT requests and preventing throttling. Option A (S3 Transfer Acceleration) improves transfer speed, not request rate limits.

Option B (disabling error logging) does not reduce API calls and hides issues. Option C (deliver to DynamoDB) is not supported by Kinesis Data Firehose. Therefore, option D is correct.

Practice this question →

479

MCQhard

A company uses AWS Glue ETL jobs to transform data in Amazon S3. The data is partitioned by date and hour. The job reads the latest hour's data, performs aggregations, and writes results to a separate S3 bucket. The job runs every hour and processes approximately 500 MB of input data. The team notices that the job takes longer than expected, often exceeding the 1-hour window. Which action would most effectively reduce the job's runtime?

A.Use a Python shell job instead of a Spark job.

B.Switch from using DynamicFrame to using Spark SQL for transformations.

C.Repartition the input data into more partitions before reading.

D.Increase the number of workers (DPUs) for the Glue job.

AnswerD

More workers increase parallelism, reducing runtime for the given data size.

Why this answer

Increasing the number of workers (DPUs) for the Glue job directly addresses the root cause: the job is CPU- or memory-bound due to insufficient parallelism for the 500 MB hourly workload. By allocating more DPUs, AWS Glue can distribute the aggregation and write operations across more executors, reducing wall-clock time and keeping the job within the 1-hour window. This is the most effective action because the job's bottleneck is compute capacity, not data format or processing framework.

Exam trap

The trap here is that candidates confuse 'repartitioning' (Option C) with 'increasing parallelism' — but without more workers, more partitions simply create scheduling overhead and do not reduce runtime.

How to eliminate wrong answers

Option A is wrong because a Python shell job runs on a single node with limited memory and no distributed processing, which would likely increase runtime for a 500 MB aggregation workload. Option B is wrong because switching from DynamicFrame to Spark SQL does not inherently improve performance; both use the same underlying Spark engine, and the bottleneck is parallelism, not the API abstraction. Option C is wrong because repartitioning the input data into more partitions before reading does not reduce the total data volume or computation; it only changes how data is distributed, and if the job already has insufficient workers, more partitions can actually increase overhead without improving runtime.

Practice this question →

480

MCQeasy

A data engineer needs to ingest log files from multiple EC2 instances into Amazon S3. The logs are written to local disk on each instance. The engineer wants a simple agent-based solution that can collect, compress, and upload logs to S3 with minimal configuration. The solution must support incremental uploads (only new log lines) and handle log rotation. What should the engineer use?

A.Install and configure Amazon CloudWatch Agent to collect logs and send them to Amazon CloudWatch Logs, then use a subscription filter to export logs to S3.

B.Use AWS CLI cp command with --recursive in a cron job to copy logs to S3 every minute.

C.Install AWS DataSync agent on each EC2 instance to sync logs to S3 daily.

D.Use an S3 sync command from the AWS CLI scheduled every hour.

AnswerA

Kinesis Agent tails log files, compresses, and sends to CloudWatch Logs; export to S3 can be automated.

Why this answer

Amazon CloudWatch Agent is a lightweight agent that can tail log files, compress them on the fly, and send them to CloudWatch Logs. From CloudWatch Logs, a subscription filter can export the logs to Amazon S3, supporting incremental uploads and log rotation. Option B (AWS CLI cp) is manual and does not handle incremental uploads efficiently.

Option C (AWS DataSync) is designed for bulk data transfers, not real-time log ingestion. Option D (S3 sync) is also not real-time and lacks agent-based tailing and compression.

Practice this question →

481

MCQhard

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The Firehose delivery stream has a buffer size of 64 MB and a buffer interval of 300 seconds. The data volume is 1 GB per minute, and the average record size is 1 KB. The data must be delivered to S3 within 5 minutes of ingestion. The engineer notices that some files are being delivered after 10 minutes. What is the most likely cause?

A.The buffer size of 64 MB is too small for the data volume

B.The data is not compressed, causing larger file sizes

C.The buffer interval of 300 seconds is too long

D.The S3 bucket is throttling PUT requests due to high throughput

AnswerD

High PUT request rates can cause throttling, leading to retries and increased delivery time.

Why this answer

Amazon S3 buckets have a default limit of 3,500 PUT requests per second per prefix. With a data volume of 1 GB per minute and an average record size of 1 KB, Firehose generates approximately 1,000,000 records per minute, resulting in roughly 16,667 PUT requests per second (since each 64 MB buffer yields about 65,536 records, and 1 GB/min ÷ 64 MB = ~15.6 buffers per minute, each requiring a PUT). This far exceeds the S3 PUT request limit, causing throttling (HTTP 503 Slow Down errors) and delivery delays beyond the 5-minute target.

Exam trap

The trap here is that candidates often focus on buffer size or interval settings, overlooking the S3 PUT request rate limit, which is a common cause of delivery delays in high-throughput Firehose-to-S3 pipelines.

How to eliminate wrong answers

Option A is wrong because a 64 MB buffer size is actually appropriate for this data volume; increasing it would reduce the number of PUT requests but the core issue is S3 throttling, not buffer size. Option B is wrong because compression reduces file size and thus the number of PUT requests, but the problem is caused by excessive PUT request rate, not file size; compression would help but its absence is not the root cause. Option C is wrong because a 300-second buffer interval is within the 5-minute delivery requirement; the delay occurs due to S3 throttling, not because the interval is too long.

Practice this question →

482

MCQeasy

A company needs to ingest data from multiple SaaS applications (Salesforce, Marketo) and load it into Amazon Redshift. The data must be transformed before loading. Which AWS service should be used to build the ingestion pipelines?

A.AWS Database Migration Service (DMS)

B.AWS Data Pipeline

C.Amazon AppFlow

D.AWS Glue (crawlers and ETL jobs)

AnswerD

AWS Glue can connect to SaaS sources via JDBC and perform complex transformations.

Why this answer

AWS Glue is the correct choice because it provides a fully managed ETL service that can connect to various data sources (including SaaS applications via JDBC or custom connectors), transform the data using Apache Spark or Python scripts, and load it into Amazon Redshift. Glue crawlers can catalog the source schemas, and Glue ETL jobs handle the transformation logic required before loading into Redshift, making it ideal for building ingestion pipelines from multiple SaaS sources.

Exam trap

The trap here is that candidates often confuse Amazon AppFlow's simplicity for ETL capability, but AppFlow lacks the advanced transformation and orchestration features needed for complex data pipelines, making Glue the correct choice despite AppFlow's direct Redshift integration.

How to eliminate wrong answers

Option A is wrong because AWS DMS is designed for database migration and continuous replication between databases, not for ingesting data from SaaS applications like Salesforce or Marketo, and it lacks built-in transformation capabilities beyond basic data type conversions. Option B is wrong because AWS Data Pipeline is a legacy orchestration service that requires managing EC2 instances and has limited native support for SaaS sources; it is less flexible and more complex than Glue for ETL workloads. Option C is wrong because Amazon AppFlow is optimized for simple, no-code data transfers between SaaS applications and AWS services (like S3 or Redshift) but does not support complex transformations or custom ETL logic, which is required in this scenario.

Practice this question →

483

MCQeasy

A data engineer needs to ingest streaming data from an IoT fleet into Amazon S3 for near-real-time analytics. The data volume is approximately 5 GB per hour, and each event is less than 1 KB. Which AWS service should be used as the ingestion endpoint?

A.AWS IoT Core

B.AWS DataSync

C.Amazon AppFlow

D.Amazon Kinesis Data Streams

AnswerA

Designed for IoT device ingestion.

Why this answer

AWS IoT Core is purpose-built for ingesting data from IoT devices, supporting MQTT, HTTP, and WebSocket protocols. It can handle millions of devices and high-throughput, small-message payloads (each event <1 KB) and integrates directly with Amazon S3 via IoT Core rules, making it the ideal ingestion endpoint for near-real-time analytics on streaming IoT data.

Exam trap

The trap here is that candidates often default to Amazon Kinesis Data Streams for any streaming workload, overlooking that AWS IoT Core is the specialized, fully managed service designed specifically for IoT device ingestion, with native MQTT support and direct S3 integration via rules.

How to eliminate wrong answers

Option B (AWS DataSync) is wrong because it is designed for one-time or scheduled bulk data transfers between on-premises storage and AWS, not for continuous, near-real-time streaming ingestion from IoT devices. Option C (Amazon AppFlow) is wrong because it is a fully managed integration service for transferring data between SaaS applications (e.g., Salesforce, Slack) and AWS, not for ingesting IoT device telemetry streams. Option D (Amazon Kinesis Data Streams) is wrong because while it can ingest streaming data, it is a generic stream processing service that requires additional configuration (e.g., Kinesis Data Firehose) to write to S3, and it is not the dedicated IoT ingestion endpoint; AWS IoT Core is the recommended first-hop for IoT data.

Practice this question →

484

Multi-Selectmedium

A data engineer is designing a data ingestion pipeline for clickstream data. The data arrives in batches of 10-50 MB every 5 seconds. The engineer needs to buffer the data, perform simple transformations (e.g., add timestamp, remove PII), and land it in S3 within 10 minutes. Which TWO services should be combined? (Choose TWO.)

Select 2 answers

A.Amazon Simple Queue Service (SQS)

B.Amazon Kinesis Data Firehose

C.AWS Lambda

D.AWS Glue ETL

E.Amazon Kinesis Data Streams

AnswersB, C

Firehose can buffer and invoke Lambda for transformation, then deliver to S3.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to ingest streaming data, buffer it, perform simple transformations (such as adding timestamps or removing PII) via built-in Lambda functions, and automatically deliver the data to S3. It can handle the 10-50 MB batches every 5 seconds and meet the 10-minute delivery window without requiring custom code for buffering or delivery.

Exam trap

The DEA-C01 exam often tests the distinction between Kinesis Data Streams (raw streaming, custom consumers) and Kinesis Data Firehose (managed ingestion with built-in transformation and delivery), leading candidates to mistakenly choose Data Streams when Firehose is the simpler, correct choice for this pipeline.

Practice this question →

485

MCQmedium

A data engineer runs the command shown. The consumer application is unable to read data older than 24 hours. What is the most likely cause?

A.The shard has reached its maximum sequence number.

B.The stream is encrypted with KMS, preventing access.

C.The retention period is set to 24 hours, so data older than 24 hours is deleted.

D.The stream is in ACTIVE status but not processing data.

AnswerC

Data retention is 24 hours; data beyond that is expired.

Why this answer

The stream's retention period is 24 hours, meaning data is automatically deleted after 24 hours. The consumer tries to read data older than 24 hours, which is no longer available.

Practice this question →

486

MCQmedium

Refer to the exhibit. A data engineer is troubleshooting a Glue job that reads objects from this S3 bucket. The job runs successfully but produces no output. The Glue catalog table points to the same S3 path. What is the most likely cause?

A.The S3 key does not follow Hive-style partitioning (e.g., year=2024/month=01).

B.The object metadata is too large.

C.The StorageClass is not supported by Glue.

D.The ContentType is not supported by Glue.

AnswerA

AWS Glue Data Catalog relies on Hive-style partitioning (e.g., year=2024/month=01) to automatically discover and register partitions. Partition projection is a separate Athena feature, not a Glue feature.

Why this answer

AWS Glue Data Catalog relies on Hive-style partitioning (e.g., year=2024/month=01) to automatically discover and register partitions. The S3 key in the exhibit does not follow this pattern, so Glue cannot identify the partitions, resulting in no data being read even though the job runs. Option B is incorrect because object metadata size does not affect Glue's ability to read data.

Option C is incorrect because the STANDARD storage class is fully supported by Glue. Option D is incorrect because ContentType is not a factor in Glue catalog table definitions.

Practice this question →

487

MCQhard

A company is ingesting data from multiple on-premises databases into AWS using AWS Database Migration Service (DMS). The data must be continuously replicated with minimal downtime. However, the source databases do not support native CDC. What should the data engineer do to enable continuous replication?

A.Use Amazon Kinesis Data Streams with a custom producer to capture database changes.

B.Use Amazon Redshift Spectrum to directly query the on-premises databases.

C.Use AWS DMS with log-based CDC if the source databases support it; otherwise, use DMS with batch replication and schedule frequent refreshes.

D.Set up AWS Glue jobs to run every minute to extract and load the data.

AnswerC

DMS supports CDC via source database logs, and if not available, batch replication can approximate continuous sync.

Why this answer

When source databases do not support native CDC, AWS DMS can still achieve continuous replication by using log-based CDC if the database engine supports it (e.g., Oracle, SQL Server, MySQL, PostgreSQL). If log-based CDC is not supported, DMS batch replication with frequent scheduled refreshes provides near-continuous replication by periodically extracting full or incremental changes, minimizing downtime. This approach leverages DMS's built-in task settings for change data capture and resumption without requiring external streaming services.

Exam trap

The trap here is that candidates assume AWS DMS always requires native CDC or that Kinesis is the only way to achieve streaming replication, but DMS provides fallback mechanisms like batch replication with frequent refreshes to handle sources without native CDC support.

How to eliminate wrong answers

Option A is wrong because Amazon Kinesis Data Streams with a custom producer requires building and maintaining custom code to capture database changes, which adds complexity and does not leverage AWS DMS's native replication capabilities; it also does not address the lack of native CDC in the source databases. Option B is wrong because Amazon Redshift Spectrum is a query engine for data in Amazon S3 and cannot directly query on-premises databases; it requires data to already be in S3 and does not provide continuous replication. Option D is wrong because AWS Glue jobs running every minute are batch-oriented and not designed for continuous, low-latency replication; they introduce significant overhead and cannot achieve minimal downtime compared to DMS's CDC or scheduled batch replication.

Practice this question →

488

MCQeasy

A company uses AWS Database Migration Service (DMS) to continuously replicate data from an on-premises Oracle database to Amazon S3. The data is stored as CSV files. The downstream team requires the data to be in Apache Parquet format. Which change should the data engineer make to the DMS task?

A.Modify the DMS task to use Apache Parquet as the target table preparation mode.

B.Add an S3 lifecycle rule to convert CSV to Parquet.

C.Change the DMS task to use full load instead of continuous replication.

D.Configure a Lambda function to transform data after DMS writes to S3.

AnswerA

DMS can write directly in Parquet format.

Why this answer

AWS DMS supports specifying Apache Parquet as the target data format for Amazon S3 targets directly within the task configuration. By setting the 'Data format' to 'Parquet' in the S3 target endpoint or task settings, DMS automatically converts the replicated data into Parquet files, eliminating the need for post-processing. This is the most efficient and native approach to meet the downstream team's requirement.

Exam trap

The trap here is that candidates may assume DMS only supports CSV for S3 targets, overlooking the built-in Parquet option, and instead choose a complex workaround like Lambda or lifecycle rules.

How to eliminate wrong answers

Option B is wrong because S3 lifecycle rules can transition objects between storage classes or expire them, but they cannot convert file formats (e.g., CSV to Parquet). Option C is wrong because changing from continuous replication to full load would stop ongoing data synchronization and does not address the format conversion requirement. Option D is wrong because while a Lambda function could transform CSV to Parquet, it introduces unnecessary complexity, latency, and cost compared to the native DMS capability; DMS can directly write Parquet without additional services.

Practice this question →

489

MCQhard

A company uses Kinesis Data Firehose with a Lambda function for data transformation. The transformation is failing intermittently due to Lambda timeouts. The maximum record size is 1 MB. What is the most cost-effective way to reduce failures without losing data?

A.Use Kinesis Data Analytics to pre-process data before Firehose

B.Decrease the Firehose batch size to reduce the number of records per invocation

C.Configure the Firehose delivery stream to send failed records to an S3 dead-letter bucket

D.Increase the Lambda function timeout and memory allocation

AnswerD

Increasing timeout and memory reduces timeouts without losing data.

Why this answer

The most cost-effective way to reduce failures from Lambda timeouts without losing data is to increase the Lambda function's timeout and memory allocation (Option D). Lambda timeouts occur when the transformation takes longer than the allocated timeout. Increasing memory also increases CPU, which can speed up processing and reduce timeouts.

This approach is cost-effective because you only pay for the increased resources when the function runs, and it avoids data loss since all records are still transformed. Option A (Kinesis Data Analytics) adds unnecessary complexity and cost. Option B (decreasing batch size) reduces the number of records per invocation, which can help but may increase costs because more invocations are needed; also, it doesn't directly address timeouts.

Option C (sending failed records to an S3 dead-letter bucket) would result in data loss for those records, as they are not transformed, and the question says 'without losing data'. Thus, D is the best choice.

Practice this question →

490

MCQeasy

A data engineer needs to ingest data from an Amazon S3 bucket into an Amazon Redshift table on a daily schedule. The data is in CSV format and the schema matches. Which service is simplest for this batch ingestion?

A.Amazon Redshift COPY command

B.AWS Glue ETL job with JDBC connection

C.AWS Data Pipeline

D.Amazon Athena CREATE TABLE AS SELECT

AnswerA

Direct and optimized for loading from S3.

Why this answer

The Amazon Redshift COPY command is the simplest and most efficient method for batch loading data from Amazon S3 into Redshift when the schema matches and the data is in CSV format. It leverages Redshift's massively parallel processing (MPP) architecture to read data directly from S3, automatically handling compression, encryption, and error logging without requiring any intermediate services or custom code.

Exam trap

The trap here is that candidates may overcomplicate the solution by choosing AWS Glue or Data Pipeline, forgetting that Redshift's native COPY command is purpose-built for high-speed, parallel batch ingestion from S3 with minimal configuration.

How to eliminate wrong answers

Option B is wrong because AWS Glue ETL with JDBC connection introduces unnecessary complexity and overhead for a simple schema-matching CSV load; Glue is better suited for complex transformations or semi-structured data, not for a direct COPY operation. Option C is wrong because AWS Data Pipeline is a legacy orchestration service that requires defining pipelines, schedules, and activities, adding operational overhead compared to the single COPY command. Option D is wrong because Amazon Athena CREATE TABLE AS SELECT (CTAS) writes query results to a new table in S3, not into Redshift; it cannot directly ingest data into a Redshift table.

Practice this question →

491

MCQmedium

A data engineer is using Amazon EMR to transform large datasets stored in S3. The cluster runs once a day and takes 3 hours. The engineer notices that the cluster is idle for 30 minutes at the start while waiting for resources. What is the most cost-effective way to reduce the idle time?

A.Increase the instance type size

B.Use Spot Instances for all nodes

C.Configure a larger initial core instance count and enable managed scaling

D.Purchase Reserved Instances for the cluster

AnswerC

More core nodes reduce the time to allocate resources, and managed scaling adjusts during the job.

Why this answer

Enabling managed scaling allows the EMR cluster to automatically adjust its core instance count based on workload demands, reducing the initial idle time caused by waiting for resource allocation. By configuring a larger initial core instance count, the cluster can start processing immediately with sufficient capacity, and managed scaling then optimizes resource usage throughout the job, making it the most cost-effective solution for a daily 3-hour transformation job.

Exam trap

The trap here is that candidates often confuse cost optimization strategies (like Spot Instances or Reserved Instances) with performance improvements, failing to recognize that idle time at startup is a resource allocation issue best solved by managed scaling and initial capacity configuration.

How to eliminate wrong answers

Option A is wrong because increasing the instance type size (e.g., using larger EC2 instances) does not address the root cause of idle time—resource allocation delays—and would increase costs without guaranteeing faster startup. Option B is wrong because using Spot Instances for all nodes can reduce cost but introduces the risk of interruptions and does not reduce the initial idle time; in fact, Spot Instances may have longer provisioning delays or be preempted, worsening the problem. Option D is wrong because purchasing Reserved Instances is a commitment-based discount model that reduces per-hour costs for steady-state workloads, but it does not reduce the idle time at cluster startup; it would be cost-ineffective for a cluster that runs only 3 hours per day.

Practice this question →

492

MCQmedium

A data engineer needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The on-premises network has a 1 Gbps link to AWS. The transfer must complete within 5 days. Which solution is MOST cost-effective and meets the requirements?

A.Use Amazon S3 Transfer Acceleration to speed up the transfer over the internet.

B.Use AWS DataSync to transfer the data over the existing network link.

C.Use AWS Snowball Edge to physically transfer the data.

D.Use AWS Direct Connect to establish a dedicated network connection.

AnswerC

AWS Snowball Edge is a physical device used for offline data transfer. It can handle large volumes faster than network transfer and is cost-effective for this scenario.

Why this answer

AWS Snowball Edge is a physical device that can transfer large amounts of data faster than a network link, especially with a 1 Gbps link that would take about 4.6 days for 50 TB (theoretical max, but actual throughput will be lower due to overhead). Snowball Edge can transfer 50 TB in a few days and is cost-effective for large data volumes. Option A (Amazon S3 Transfer Acceleration) speeds up transfers but still limited by network bandwidth.

Option B (AWS DataSync) is efficient for online transfers but may not meet the 5-day deadline over 1 Gbps. Option D (AWS Direct Connect) would require additional setup and cost, and still limited by the 1 Gbps link.

Practice this question →

493

MCQmedium

A data engineer needs to ingest JSON files from an on-premises SFTP server into Amazon S3. The files are uploaded daily and each file is up to 500 MB. The solution must be serverless and minimize cost. Which service should the engineer use?

A.Amazon Kinesis Data Firehose.

B.AWS DataSync with an on-premises agent.

C.Amazon S3 Transfer Acceleration.

D.AWS Transfer Family (SFTP) endpoint.

AnswerD

Fully managed SFTP service that writes directly to S3.

Why this answer

AWS Transfer Family provides a fully managed SFTP endpoint that directly integrates with Amazon S3, enabling file transfers without managing servers. Option A (Kinesis Data Firehose) is designed for streaming data, not file-based SFTP ingestion. Option B (AWS DataSync with an on-premises agent) requires installing and managing an agent, which adds operational overhead and is not fully serverless.

Option C (S3 Transfer Acceleration) accelerates transfers to S3 but does not support ingesting from SFTP sources.

Practice this question →

494

MCQhard

A data engineering team is troubleshooting a slow AWS Glue ETL job that reads from an Amazon DynamoDB table and writes to Amazon S3 in Parquet format. The job processes 50 GB of data. Which action would most effectively improve job performance?

A.Use S3 Select to push down filters

B.Reduce the batch size in the DynamoDB connector

C.Increase the number of DPUs

D.Change output to JSON format to reduce overhead

AnswerC

More DPUs increase parallelism and can speed up the job.

Why this answer

Increasing the number of DPUs (Data Processing Units) for the AWS Glue job directly allocates more distributed computing resources (CPU, memory, and network bandwidth) to parallelize the read from DynamoDB and the write to S3. Since the job processes 50 GB of data, the bottleneck is likely the throughput of the Glue Spark cluster, and adding DPUs increases parallelism, reducing overall execution time.

Exam trap

The trap here is that candidates often confuse S3 Select as a universal filter mechanism or assume that reducing batch size always improves performance, when in fact it increases API overhead and latency in distributed systems like Glue.

How to eliminate wrong answers

Option A is wrong because S3 Select is used to filter data within S3 objects (e.g., CSV or JSON files) and cannot be applied to a DynamoDB source; the filter pushdown must happen at the DynamoDB API level using expressions, not S3 Select. Option B is wrong because reducing the batch size in the DynamoDB connector would decrease the number of items read per request, increasing the number of API calls and likely worsening performance due to higher latency and throttling risk. Option D is wrong because changing output to JSON format would increase file size and write overhead compared to Parquet (which is columnar and compressed), thus degrading performance, not improving it.

Practice this question →

495

MCQhard

A data engineer is designing a data ingestion pipeline for IoT sensor data. The data arrives as JSON via AWS IoT Core, and must be stored in Amazon S3 in partitioned Parquet format. The pipeline must handle late-arriving data (up to 1 hour) and ensure exactly-once processing. Which combination of services should the engineer use?

A.Amazon Kinesis Data Streams with AWS Lambda for transformation and Amazon S3.

B.Amazon Simple Queue Service (SQS) with AWS Lambda for transformation and Amazon S3.

C.AWS Glue streaming jobs consuming from Amazon Kinesis Data Streams and writing to Amazon S3.

D.Amazon Kinesis Data Firehose with data transformation via AWS Lambda, delivering to Amazon S3.

AnswerD

Firehose supports Parquet conversion and partitioning; Lambda handles transformation.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it can directly ingest streaming data from AWS IoT Core, use a built-in AWS Lambda function to transform JSON to Parquet, and deliver the data to Amazon S3 with automatic partitioning. It also supports buffering and retry logic to handle late-arriving data (up to 1 hour) and provides exactly-once delivery to S3 when configured with the appropriate error handling and idempotent transformations.

Exam trap

The trap here is that candidates often choose Kinesis Data Streams with Lambda (Option A) because they think it offers more control, but they overlook that Firehose provides a managed, exactly-once, partitioned Parquet delivery pipeline with built-in late-arriving data handling, which is the exact requirement in the question.

How to eliminate wrong answers

Option A is wrong because Amazon Kinesis Data Streams with AWS Lambda requires custom code to manage checkpointing, partitioning, and exactly-once semantics, and does not natively support Parquet conversion or S3 delivery without additional complexity. Option B is wrong because Amazon SQS does not guarantee exactly-once processing (standard queues offer at-least-once, FIFO queues offer exactly-once but lack native streaming integration with IoT Core and Parquet transformation). Option C is wrong because AWS Glue streaming jobs consume from Kinesis Data Streams, not directly from IoT Core, and they do not provide built-in exactly-once delivery to S3; they rely on checkpointing that can lead to duplicates or data loss in failure scenarios.

Practice this question →

496

MCQeasy

A company wants to ingest streaming data from IoT devices into Amazon S3 using Amazon Kinesis Data Firehose. The data must be transformed from JSON to Parquet format before landing in S3. What is the SIMPLEST way to achieve this?

A.Configure Kinesis Data Firehose with a built-in Parquet converter.

B.Use an AWS Lambda function as a data transformation in Kinesis Data Firehose to convert JSON to Parquet.

C.Use Kinesis Data Firehose to deliver data directly to S3 in JSON format and run a nightly Glue job to convert to Parquet.

D.Use Kinesis Data Analytics to convert the data to Parquet before sending to Firehose.

AnswerA

Correct. Firehose has a built-in Parquet converter that uses an AWS Glue schema. This is the simplest method as it requires no custom code or additional services.

Why this answer

Amazon Kinesis Data Firehose has a built-in Parquet conversion feature that uses an AWS Glue schema to convert incoming JSON data to Parquet format. This is the simplest approach because it requires no custom code or additional services; you only need to provide a schema and enable the conversion in the Firehose delivery stream configuration. Option B (using Lambda) is more complex, as it requires writing and maintaining a custom transformation function.

Exam trap

Candidates often overlook the native Parquet conversion capability in Kinesis Data Firehose and assume a Lambda function is required. The built-in conversion using an AWS Glue schema is actually simpler and supported directly.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Firehose does not have a built-in Parquet converter; it can deliver data in Parquet format only if the input data is already in a format that can be converted (e.g., via a Lambda transformation or by using a schema from AWS Glue), but there is no native 'Parquet converter' toggle. Option C is wrong because it introduces unnecessary complexity and latency by storing JSON in S3 first and then running a nightly Glue job, which is not the simplest approach and does not meet the requirement for real-time transformation before landing. Option D is wrong because Kinesis Data Analytics is designed for real-time analytics and stream processing, not for format conversion; it would add unnecessary overhead and complexity compared to using Firehose's built-in Lambda transformation.

Practice this question →

497

MCQhard

A data engineer is troubleshooting a slow AWS Glue ETL job that reads from Amazon S3 and writes to Amazon Redshift. The job processes 10 GB of CSV data. The engineer notices that the job runs with a single DPU and takes longer than expected. Which change would MOST likely improve performance?

A.Replace Redshift with Amazon Redshift Spectrum.

B.Change the input format to Parquet and enable predicate pushdown.

C.Use a JDBC connection to read data directly from S3.

D.Increase the number of DPUs and configure the job to use the S3 list implementation for parallel reads.

AnswerD

More DPUs allow parallel processing, and S3 list implementation improves file discovery.

Why this answer

The job runs with a single DPU, which limits parallelism. Increasing the number of DPUs allows the job to process data in parallel across multiple workers. Additionally, configuring the S3 list implementation enables the job to list objects in parallel, reducing the overhead of discovering input files.

This combination directly addresses the bottleneck of a single-threaded read from S3.

Exam trap

The trap here is that candidates often focus on data format optimization (Parquet) or query engine changes (Redshift Spectrum) without realizing that the primary bottleneck is the single-DPU configuration limiting parallelism, which is a common oversight in Glue job tuning questions.

How to eliminate wrong answers

Option A is wrong because replacing Redshift with Redshift Spectrum does not address the root cause of low parallelism in the Glue job; Spectrum is a query engine for data in S3, not a performance fix for a Glue ETL writing to Redshift. Option B is wrong because changing to Parquet and enabling predicate pushdown improves read efficiency and reduces data scanned, but the job is bottlenecked by a single DPU, not by I/O format or filtering. Option C is wrong because using a JDBC connection to read data directly from S3 is not a valid approach; JDBC is for relational databases, not for reading files from S3, and this would introduce unnecessary overhead.

Practice this question →

498

MCQeasy

A startup is building a data pipeline to ingest user activity logs from a mobile app. The logs are sent in real-time via HTTP POST requests. The data volume is low (a few hundred requests per second) but can spike to a few thousand during promotions. The team wants to store the logs in Amazon S3 for analysis. They also need to be able to query the data using Amazon Athena with minimal latency. The data must be transformed from JSON to Parquet and partitioned by date. The team is considering using Amazon API Gateway with AWS Lambda to receive the logs and write to S3. However, they are concerned about Lambda cold starts and the complexity of handling spikes. Which alternative solution should they choose?

A.Use Amazon API Gateway with AWS Lambda that sends logs to Amazon SQS, then a separate Lambda reads from SQS and writes to S3

B.Use Amazon Kinesis Data Firehose with a HTTP endpoint as source, enable Parquet conversion, and deliver to S3 with dynamic partitioning

C.Use Amazon Kinesis Data Streams with AWS Lambda to process and write to S3

D.Use Amazon EMR with Spark Streaming to ingest logs from a custom endpoint

AnswerB

Firehose handles ingestion, transformation, and partitioning with automatic scaling.

Why this answer

Amazon Kinesis Data Firehose is the best choice because it can directly receive HTTP POST requests (via its HTTP endpoint or integrated with API Gateway), automatically buffer incoming data, convert JSON to Parquet, and deliver to S3 with dynamic partitioning by date. This handles traffic spikes without custom code or Lambda cold starts, meeting all requirements with minimal operational overhead. Option A (API Gateway + Lambda → SQS → Lambda) adds complexity and still involves Lambda cold starts.

Option C (Kinesis Data Streams + Lambda) also requires Lambda and cold start management. Option D (EMR Spark Streaming) is overkill for this low-volume use case.

Practice this question →

499

MCQmedium

A company needs to ingest data from multiple SaaS applications (e.g., Salesforce, Marketo) into Amazon S3 for analytics. The data sources have different schemas and update frequencies. Which AWS service should be used to build this ingestion pipeline with minimal code?

A.AWS Data Pipeline

B.AWS Glue

C.Amazon Kinesis Data Firehose

D.Amazon AppFlow

AnswerD

AppFlow is designed to ingest data from SaaS applications to S3 with minimal code.

Why this answer

Amazon AppFlow (Option D) is the correct answer because it is purpose-built for ingesting data from SaaS applications like Salesforce and Marketo into Amazon S3 with minimal code. It supports various source connectors, handles schema variations, and allows scheduling based on update frequencies. AWS Data Pipeline (A) requires more manual configuration and code for connectors.

AWS Glue (B) has connectors but is more complex for this use case, often requiring additional ETL scripting. Amazon Kinesis Data Firehose (C) is designed for streaming data, not batch extraction from SaaS APIs.

Practice this question →

500

MCQeasy

A company wants to ingest data from multiple SaaS applications into Amazon S3 using a fully managed service that supports schema discovery and transformation. Which AWS service should they use?

A.Amazon Kinesis Data Firehose

B.Amazon AppFlow

C.AWS Glue

D.AWS Data Pipeline

AnswerB

Amazon AppFlow is fully managed, supports multiple SaaS sources, schema discovery, and transformation.

Why this answer

Amazon AppFlow is a fully managed integration service that supports SaaS sources, schema discovery, and data transformation. Option A (Amazon Kinesis Data Firehose) is for streaming data but not for SaaS ingestion directly. Option C (AWS Glue) is an ETL service but not for SaaS ingestion directly.

Option D (AWS Data Pipeline) is not fully managed for SaaS.

Practice this question →

501

MCQhard

A company uses Amazon Kinesis Data Firehose to ingest JSON logs from multiple sources into an S3 data lake. The data is then consumed by Amazon Athena for analysis. Recently, some queries have been failing with the error 'HIVE_BAD_DATA: Field xyz's type is an unsupported type'. The firehose delivery stream transforms the data using a Lambda function that converts timestamps to Unix epoch. What is the MOST likely cause of the query failure?

A.Some records contain timestamps that were not converted to epoch, so Athena infers the column as a string.

B.The data is in JSON format instead of Parquet.

C.The S3 partitions are not registered in the Glue Data Catalog.

D.The IAM role for Firehose does not have permission to write to S3.

AnswerA

Inconsistent data types in a column cause Athena to default to string, leading to type mismatch when queried.

Why this answer

The error 'HIVE_BAD_DATA: Field xyz's type is an unsupported type' occurs when Athena's schema inference encounters inconsistent data types for the same column. Since the Lambda function converts timestamps to Unix epoch, but some records may have failed conversion (e.g., due to malformed input or Lambda errors), those records retain the original string timestamp. Athena then sees a mix of numeric epoch values and string timestamps, causing it to infer the column as a string type, which is unsupported for the expected numeric operations in the query.

Exam trap

The DEA-C01 exam often tests the misconception that Athena errors are always due to file format or permissions, when in reality schema-on-read type inference from inconsistent data is a common pitfall.

How to eliminate wrong answers

Option B is wrong because Athena can query JSON data directly; the error is about type mismatch, not file format. Option C is wrong because unregistered partitions would cause a 'Table not found' or 'Partition not found' error, not a type inference error. Option D is wrong because a missing S3 write permission would cause Firehose delivery failures or missing data, not a schema/type error during query execution.

Practice this question →

502

Multi-Selectmedium

A data engineer is designing a data ingestion pipeline for real-time user activity logs. The logs are generated by a web application and must be ingested into Amazon S3 with minimal latency (under 1 minute). The logs also need to be queried in Amazon Athena. The engineer considers using Amazon Kinesis Data Firehose. Which TWO configurations are required to achieve near-real-time delivery to S3? (Choose TWO.)

Select 2 answers

A.Set the BufferIntervalInSeconds to 60 seconds.

B.Enable S3 compression (e.g., GZIP) on the delivery stream.

C.Enable Amazon CloudWatch error logging for the delivery stream.

D.Enable data format conversion to Parquet using AWS Glue.

E.Set the BufferSizeInMBs to 1 MB.

AnswersA, E

Controls how often data is delivered.

Why this answer

Setting BufferIntervalInSeconds to 60 seconds forces Kinesis Data Firehose to deliver data to S3 every 60 seconds, meeting the sub-1-minute latency requirement. Option E is correct because setting BufferSizeInMBs to 1 MB ensures that the buffer fills quickly and triggers delivery when the buffer reaches 1 MB, which, combined with the time-based trigger, minimizes latency.

Exam trap

The trap here is that candidates often confuse features that improve query performance or monitoring (compression, Parquet conversion, CloudWatch logging) with features that directly control delivery latency, leading them to select those options instead of the buffer configuration parameters.

Practice this question →

503

MCQeasy

A company uses AWS Glue ETL jobs to transform data stored in Amazon S3. The job reads data in Parquet format, applies transformations, and writes the output back to S3 in Parquet format. The team wants to improve the job's performance and reduce costs. Which action is MOST effective?

A.Change the input format from Parquet to CSV to simplify parsing.

B.Coalesce the input data into a single large file before processing.

C.Use column pruning and predicate pushdown to read only necessary columns and filter data early.

D.Increase the number of workers to maximum allowed.

AnswerC

Reduces the amount of data processed, improving performance and reducing costs.

Why this answer

Column pruning and predicate pushdown reduce the amount of data read from S3 by Spark-based AWS Glue ETL jobs. By reading only the necessary columns and filtering rows early in the scan, I/O and memory usage decrease, directly improving performance and reducing costs.

Exam trap

The trap here is that candidates often confuse 'coalesce' (reducing partitions) with 'repartition' (increasing parallelism) and assume fewer files always improve performance, ignoring that Glue ETL benefits from parallel reads across many small files when using columnar formats.

How to eliminate wrong answers

Option A is wrong because changing from Parquet to CSV would increase data size and parsing overhead, degrading performance and increasing costs. Option B is wrong because coalescing input into a single file eliminates parallelism, causing a single executor to process all data, which increases runtime and resource contention. Option D is wrong because increasing workers to the maximum allowed without addressing data skew or I/O bottlenecks can lead to excessive cost with diminishing returns, and may hit service limits or shuffle overhead.

Practice this question →

504

MCQhard

Refer to the exhibit. A data engineer runs this AWS Glue job but it fails with an error that the table 'orders' does not exist in the 'sales_db' database. The engineer has verified that the table exists in the AWS Glue Data Catalog. What is the most likely cause of the error?

A.The IAM role used by the Glue job does not have permission to read the Data Catalog

B.The Glue job has job bookmark enabled and is skipping the table

C.The script uses 'create_dynamic_frame.from_catalog' incorrectly

D.The S3 path 's3://data-lake/raw/' does not exist

AnswerA

The job needs glue:GetTable permission to access the table metadata.

Why this answer

The error 'table orders does not exist in sales_db' occurs because the IAM role associated with the Glue job does not have the necessary permissions to read the AWS Glue Data Catalog. Even though the table exists, the job requires 'glue:GetTable' permission on the 'sales_db.orders' table to discover it. Option B is incorrect because job bookmark settings control data processing state, not table discovery.

Option C is incorrect because the script using 'create_dynamic_frame.from_catalog' is syntactically correct. Option D is incorrect because the error specifically indicates the table is missing, not the S3 path.

Practice this question →

505

MCQhard

Refer to the exhibit. A data engineer runs an AWS Glue ETL job that writes output to an S3 bucket. The job fails with the error shown. What is the most likely cause?

A.The IAM role used by the Glue job lacks the s3:PutObject permission for the output bucket

B.The Glue job attempted to write data in an unsupported format

C.The S3 bucket does not exist

D.The output file name contains invalid characters

AnswerA

The error explicitly states the role is not authorized to perform s3:PutObject.

Why this answer

The error shown in the exhibit indicates an access denied or permission failure when the AWS Glue ETL job attempts to write its output to the S3 bucket. The most likely cause is that the IAM role assigned to the Glue job does not include the s3:PutObject permission for the target bucket, which is required to upload objects. Without this permission, the job cannot complete the write operation, resulting in the failure.

Exam trap

The trap here is that candidates may focus on the data format or bucket existence, but the error message explicitly points to an access permission issue, which is a common misconfiguration in IAM roles for Glue jobs.

How to eliminate wrong answers

Option B is wrong because AWS Glue supports writing data in multiple formats (e.g., Parquet, ORC, JSON, CSV) and the error message does not indicate an unsupported format issue; such a problem would typically produce a different error related to format conversion. Option C is wrong because if the S3 bucket did not exist, the error would be a 'NoSuchBucket' or '404 Not Found' error, not an access denied error. Option D is wrong because while invalid characters in file names can cause errors, the error message shown is specifically about access permissions, not about invalid object key syntax.

Practice this question →

506

MCQeasy

A company wants to ingest streaming data from thousands of IoT devices into AWS for real-time analytics. Which AWS service is best suited for this purpose?

A.Amazon S3

B.AWS Lambda

C.Amazon RDS

D.Amazon Kinesis Data Streams

AnswerD

It is designed for real-time streaming data ingestion.

Why this answer

Amazon Kinesis Data Streams is purpose-built for ingesting and processing streaming data at scale from thousands of sources. It can capture and store terabytes of data per hour from IoT devices, enabling real-time analytics with millisecond latencies. The service provides durable, ordered data streams that can be consumed by multiple applications simultaneously.

Exam trap

The trap here is that candidates often confuse batch-oriented services like S3 or compute services like Lambda with the dedicated streaming ingestion layer required for real-time data, overlooking that Kinesis Data Streams provides the necessary buffering, ordering, and replay capabilities.

How to eliminate wrong answers

Option A is wrong because Amazon S3 is an object storage service designed for static data, not for real-time streaming ingestion; it lacks the low-latency, ordered delivery and concurrent consumer support required for streaming IoT data. Option B is wrong because AWS Lambda is a serverless compute service that can process events but is not designed as a primary ingestion buffer for high-throughput streaming data; it has a maximum invocation duration of 15 minutes and cannot natively store or replay streaming data. Option C is wrong because Amazon RDS is a relational database service for transactional workloads and structured queries, not for high-velocity, unbounded streaming data ingestion; it would create bottlenecks and cannot handle the throughput and ordering requirements of thousands of IoT devices.

Practice this question →

507

MCQhard

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The application reads from a Kinesis stream with 10 shards and writes to an S3 bucket. The application is experiencing high latency. Analysis shows that the application is not keeping up with the incoming data rate. Which action would MOST effectively reduce latency?

A.Increase the number of shards in the Kinesis stream

B.Increase the Parallelism of the Flink application

C.Enable exactly-once delivery to S3

D.Use a larger Kinesis Data Analytics application (increase KPU)

AnswerB

Higher parallelism allows more concurrent processing.

Why this answer

The high latency is caused by the Flink application not keeping up with the incoming data rate, which indicates a processing bottleneck within the application itself. Increasing the Parallelism of the Flink application (Option B) directly increases the number of parallel subtasks that can process data concurrently, improving throughput and reducing latency. This is the most effective action because it addresses the root cause—insufficient compute resources for stream processing—without changing the source or sink configuration.

Exam trap

The trap here is that candidates often confuse scaling the infrastructure (KPU or shards) with scaling the application logic (parallelism), assuming that more shards or larger instances automatically resolve processing bottlenecks without explicitly tuning the Flink application's parallelism.

How to eliminate wrong answers

Option A is wrong because increasing the number of shards in the Kinesis stream would increase the incoming data rate and parallelism at the source, but the application is already unable to keep up with the current rate; adding more shards would worsen the bottleneck unless the Flink application's parallelism is also increased. Option C is wrong because enabling exactly-once delivery to S3 increases processing overhead and checkpointing frequency, which would further degrade performance and increase latency, not reduce it. Option D is wrong because increasing the Kinesis Data Analytics application size (KPU) increases the underlying compute resources (vCPU and memory), but without increasing Flink parallelism, the additional resources may not be fully utilized; parallelism must be explicitly configured to match the increased KPU for effective scaling.

Practice this question →

508

MCQeasy

A data engineer needs to ingest data from a relational database (MySQL) into Amazon S3 for analytics. The database is 500 GB and the job must run daily with incremental updates. Which AWS service is BEST suited for this task?

A.Amazon EMR with Apache Sqoop.

B.Amazon Kinesis Data Firehose with a database source.

C.AWS Database Migration Service (DMS) with a replication task.

D.AWS Glue ETL job with a JDBC connection.

AnswerC

DMS supports continuous replication and can write to S3.

Why this answer

AWS DMS with a replication task is the best choice because it is specifically designed for continuous, incremental data replication from relational databases like MySQL to Amazon S3. DMS supports ongoing replication (change data capture) to capture incremental changes without custom scripting, and it can handle the initial 500 GB load efficiently. Other services either lack native incremental support or require additional configuration for this use case.

Exam trap

The trap here is that candidates often choose AWS Glue ETL (Option D) because it is familiar for data transformation, but they overlook that Glue lacks native incremental replication from databases, whereas DMS is purpose-built for this exact scenario with minimal overhead.

How to eliminate wrong answers

Option A is wrong because Amazon EMR with Apache Sqoop is a batch-oriented tool that does not natively support continuous incremental updates; it requires manual scripting for change data capture and is more complex to manage for daily incremental runs. Option B is wrong because Amazon Kinesis Data Firehose does not natively connect to a relational database as a source; it ingests streaming data from producers like Kinesis Data Streams or SDK, not directly from MySQL. Option D is wrong because AWS Glue ETL with a JDBC connection is designed for batch ETL jobs and does not have built-in change data capture for incremental updates; it would require custom logic to track changes, making it less suitable for daily incremental ingestion.

Practice this question →

509

Multi-Selectmedium

A data engineer is designing a data ingestion pipeline for real-time clickstream data. The data must be available for both real-time analytics and batch processing. The engineer wants to use Amazon Kinesis Data Streams. Which THREE components should be included in the architecture?

Select 3 answers

A.Amazon Kinesis Data Analytics

B.Amazon Kinesis Data Streams

C.Amazon Kinesis Client Library (KCL) application

D.Amazon Kinesis Data Firehose to deliver data to Amazon S3

E.Amazon SQS as a buffer

AnswersB, C, D

Primary ingestion service.

Why this answer

Amazon Kinesis Data Streams is the core ingestion service for real-time clickstream data, providing low-latency, durable storage of data records that can be consumed by multiple applications simultaneously. It enables both real-time analytics (via Kinesis Data Analytics or KCL applications) and batch processing (by integrating with Kinesis Data Firehose to deliver data to Amazon S3).

Exam trap

The trap here is that candidates often assume Kinesis Data Analytics is a required component for real-time analytics, but the question only requires the data to be available for real-time analytics—not that analytics must be performed within the pipeline—so Kinesis Data Analytics is optional, not mandatory.

Practice this question →

510

Multi-Selecteasy

A data engineer needs to transform CSV files in S3 to Parquet format using a serverless solution. The files are large (up to 5 GB each) and arrive irregularly. Which TWO services can accomplish this with minimal operational overhead? (Choose TWO.)

Select 2 answers

A.AWS Glue ETL job

B.AWS Step Functions with Athena CTAS queries

C.Amazon EC2 with a script

D.Amazon EMR cluster

E.Amazon Redshift Spectrum

AnswersA, B

Glue is serverless and can convert large CSV to Parquet efficiently.

Why this answer

AWS Glue ETL job is correct because it is a fully managed, serverless service that can automatically convert CSV to Parquet without provisioning infrastructure. It handles large files (up to 5 GB) by scaling Spark executors dynamically, and can be triggered by S3 events for irregular arrivals, minimizing operational overhead.

Exam trap

The trap here is that candidates often confuse query engines (like Athena or Redshift Spectrum) with transformation services, or assume that any AWS service with 'serverless' in its name can perform ETL, when in fact Athena CTAS requires Step Functions orchestration and is limited to SQL-based transformations, not direct file format conversion.

Practice this question →

511

MCQmedium

A media company ingests large video files from partners via AWS Transfer Family (SFTP) into an S3 bucket. Each file is typically 2-5 GB. Once uploaded, an AWS Lambda function is triggered to transcode the video using Amazon Elastic Transcoder. The Lambda function reads the file from S3, submits a transcoding job to Elastic Transcoder, and writes the output back to a different S3 bucket. Recently, the Lambda function has been failing intermittently with timeouts, and the company reports that some files are not being transcoded. The CloudWatch logs show that the Lambda function is timing out after 15 minutes. The average transcoding job takes about 10 minutes to complete. The data engineer needs to fix the issue without changing the architecture drastically. What should the data engineer do?

A.Increase the Lambda function's reserved concurrency to allow multiple invocations in parallel.

B.Increase the Lambda function timeout to 20 minutes to accommodate longer transcoding jobs.

C.Modify the Lambda function to submit the transcoding job asynchronously and exit, using an SNS topic to trigger a second Lambda function when the job completes.

D.Replace AWS Transfer Family with AWS Database Migration Service to handle file transfers more efficiently.

AnswerC

Decoupling submission from completion avoids timeout.

Why this answer

The Lambda function is timing out because it waits synchronously for the Elastic Transcoder job to complete, which can exceed the 15-minute Lambda timeout. The correct fix is to decouple the transcoding submission from the waiting: modify the Lambda function to submit the job asynchronously and exit immediately, then use an SNS topic or CloudWatch Events to trigger a second Lambda function when the job completes. This avoids the timeout.

Option A (increasing reserved concurrency) does not address the root cause; timeouts occur per invocation, not due to lack of concurrency. Option B (increasing timeout to 20 minutes) might work but is not ideal because jobs can vary and still exceed the new timeout; the better pattern is asynchronous processing. Option D (replacing Transfer Family with DMS) is incorrect because DMS is for database migration, not file transfer and transcoding.

Practice this question →

512

MCQmedium

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed in real-time using custom Python code before being stored in Amazon S3. Which AWS service should be used to perform this transformation?

A.Amazon EMR with Spark Streaming

B.AWS Lambda function triggered by Kinesis Data Streams

C.Kinesis Data Analytics for Apache Flink

D.Kinesis Data Firehose with custom data transformation

AnswerC

Kinesis Data Analytics for Apache Flink is the correct choice because it allows running custom Apache Flink applications that support custom Python code via the Apache Flink Python API, enabling real-time data transformation.

Why this answer

Amazon EMR with Spark Streaming, which is optimized for large-scale batch and stream processing but is not the simplest or most direct service for this specific use case. Option B is AWS Lambda, which can be used for simple transformations but has limitations on execution time and complexity. Option D is Kinesis Data Firehose with custom data transformation, which supports only built-in transformations or Lambda functions, not arbitrary custom Python code directly.

Option C, Kinesis Data Analytics for Apache Flink, is correct because it allows running custom Apache Flink applications, which support custom Python code via the Apache Flink Python API, for real-time data transformation.

Practice this question →

513

Multi-Selecthard

A company needs to ingest data from multiple SaaS applications (Salesforce, Marketo) into Amazon S3 for analytics. The data volume is moderate (~100 GB per day). The pipeline must handle schema changes, deduplicate records, and provide low latency (under 1 hour). Which THREE services should be used? (Choose THREE.)

Select 3 answers

A.Amazon AppFlow

B.Amazon EventBridge

C.Amazon Kinesis Data Streams

D.AWS Glue DataBrew

E.AWS Database Migration Service (DMS)

AnswersA, B, D

AppFlow can ingest data from SaaS applications like Salesforce and Marketo.

Why this answer

Amazon AppFlow is the correct choice because it is a fully managed integration service specifically designed to transfer data from SaaS applications like Salesforce and Marketo to AWS services such as Amazon S3. It supports incremental transfers, handles schema changes automatically via its schema evolution feature, and can achieve sub-hour latency for moderate data volumes (~100 GB/day) without custom coding.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Streams as a universal ingestion service, but it lacks native SaaS connectors and schema evolution handling, making it unsuitable for this specific use case compared to AppFlow.

Practice this question →

514

Multi-Selecthard

A company is migrating a legacy on-premises ETL pipeline to AWS. The pipeline processes daily batch files from an FTP server. The data must be transformed using complex business logic before being loaded into Amazon Redshift. Which THREE AWS services should be used for this migration?

Select 3 answers

A.Amazon Athena

B.Amazon Redshift

C.AWS Glue

D.Amazon Kinesis Data Streams

E.AWS Transfer Family

AnswersB, C, E

Redshift is the target data warehouse.

Why this answer

(Amazon Redshift) is correct as the target data warehouse for loading transformed data. Option C (AWS Glue) is correct because it can handle complex business transformations using PySpark or Python. Option E (AWS Transfer Family) is correct for replacing the FTP server and ingesting daily batch files securely into Amazon S3, which can then be processed by Glue.

Option A (Amazon Athena) is incorrect because Athena is a query service, not an ETL tool for complex transformations. Option D (Amazon Kinesis Data Streams) is incorrect because it is designed for real-time streaming data, not batch file processing from FTP.

Practice this question →

515

MCQeasy

A data engineer needs to ingest data from multiple SaaS applications (Salesforce, Marketo) into Amazon S3 for a data lake. The data volumes are moderate and the sync needs to be scheduled daily. Which AWS service is most appropriate for this task?

A.AWS Glue

B.Amazon AppFlow

C.AWS Database Migration Service (DMS)

D.Amazon Kinesis Data Firehose

AnswerB

Designed for SaaS data ingestion.

Why this answer

Amazon AppFlow is purpose-built for securely transferring data between SaaS applications (like Salesforce and Marketo) and AWS services (like S3). It supports scheduled, incremental data syncs with built-in connectors, making it the most appropriate choice for moderate-volume daily ingestion into a data lake.

Exam trap

The trap here is that candidates often confuse AWS Glue's ETL capabilities with direct SaaS ingestion, overlooking that Glue requires a custom connector or script to pull from APIs, whereas AppFlow provides native, managed connectors.

How to eliminate wrong answers

Option A is wrong because AWS Glue is an ETL service designed for batch data transformation and cataloging, not for direct ingestion from SaaS applications; it lacks native connectors for Salesforce or Marketo. Option C is wrong because AWS DMS is intended for migrating databases (e.g., Oracle, MySQL) to AWS, not for pulling data from SaaS APIs. Option D is wrong because Amazon Kinesis Data Firehose is optimized for streaming data ingestion (e.g., from IoT or logs) and does not provide native SaaS connectors or scheduled sync capabilities.

Practice this question →

516

Matchingmedium

Match each AWS data migration tool to its primary function.

Drag a concept onto its matching description — or click a concept then click the description.

Concepts

Matches

Migrate databases with minimal downtime

Physical device for large data transfer

Online data transfer between on-prem and AWS

Fast uploads over long distances

Combine data across sources into views

Why these pairings

AWS DMS migrates databases with minimal downtime; AWS Snowball transfers large datasets physically; AWS DataSync automates online data transfer. Common confusions include swapping tool functions or attributing schema conversion to DataSync instead of SCT.

Practice this question →

517

MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver streaming data to an S3 bucket. The data must be transformed from JSON to Parquet format before delivery. Which approach should be used?

A.Configure Kinesis Data Firehose to invoke a Lambda function for data transformation.

B.Use an AWS Glue ETL job to read from S3 and write Parquet back to S3.

C.Use Amazon EMR to process the data and output Parquet.

D.Use Kinesis Data Analytics to convert the data to Parquet.

AnswerA

Firehose can call a Lambda function to transform records, including converting JSON to Parquet.

Why this answer

Kinesis Data Firehose can invoke a Lambda function as a transformation step before data is delivered to S3. This allows you to convert JSON records to Parquet format inline, without needing an intermediate storage or separate processing pipeline. The Lambda function receives batches of records, transforms them (e.g., using PyArrow or similar libraries), and returns them to Firehose for delivery.

Exam trap

The trap here is that candidates may think they need a separate ETL service like Glue or EMR for format conversion, but Firehose's built-in Lambda integration is the simplest and most cost-effective way to transform data in-flight before delivery.

How to eliminate wrong answers

Option B is wrong because using an AWS Glue ETL job to read from S3 and write Parquet back to S3 introduces unnecessary latency and cost, and does not meet the requirement for transformation before delivery — it processes data after it is already stored. Option C is wrong because Amazon EMR is a heavy, cluster-based solution designed for large-scale batch or stream processing, not for lightweight, real-time transformation within a Firehose delivery stream. Option D is wrong because Kinesis Data Analytics is used for real-time analytics and SQL-based processing, not for converting data formats like JSON to Parquet; it cannot output Parquet directly to S3.

Practice this question →

518

MCQmedium

A company is streaming IoT sensor data to Amazon Kinesis Data Streams. The data is JSON with a schema that changes occasionally. They want to load the data into Amazon S3 in Parquet format partitioned by date and sensor_id. Which approach is MOST cost-effective and operationally efficient?

A.Use Amazon EMR to read from Kinesis Data Streams and write to S3 in Parquet format.

B.Use a Lambda function to transform records to Parquet and write to S3.

C.Use a custom Kinesis Client Library application on EC2 to buffer and write Parquet files to S3.

D.Use Amazon Kinesis Data Firehose with a schema from AWS Glue Data Catalog to convert to Parquet and enable dynamic partitioning by date and sensor_id.

AnswerD

Amazon Kinesis Data Firehose can directly convert incoming JSON data to Parquet using a schema from AWS Glue Data Catalog, and supports dynamic partitioning by date and sensor_id without custom code. It is fully managed, making it the most cost-effective and operationally efficient.

Why this answer

Amazon Kinesis Data Firehose can directly convert incoming JSON data to Parquet using a schema from AWS Glue Data Catalog, and it supports dynamic partitioning by date and sensor_id without requiring custom code. This is the most cost-effective and operationally efficient approach as it is a fully managed service that handles buffering, conversion, and partitioning automatically. Option A (EMR) is overkill for this use case and adds operational complexity.

Option B (Lambda) would require additional transformation logic and is less efficient for high-throughput streaming. Option C (custom KCL application on EC2) requires ongoing management and is not as simple as using Firehose.

Practice this question →

519

MCQhard

A data engineer is designing a data ingestion pipeline for JSON files landing in an Amazon S3 bucket. The pipeline must transform the data (e.g., flatten nested structures) and load it into Amazon Redshift. The transformation logic is complex and may evolve frequently. Which approach provides the MOST flexibility and ease of maintenance?

A.Use AWS Lambda functions to transform each file and load into Redshift.

B.Use the Amazon Redshift COPY command to load raw JSON directly.

C.Use AWS Glue ETL jobs to transform the data and load into Redshift.

D.Use Amazon Athena to query the raw data and insert into Redshift.

AnswerC

Glue ETL supports complex transformations and is easy to maintain.

Why this answer

AWS Glue ETL jobs provide a serverless, code-based environment using Apache Spark, which offers flexibility for complex transformations like flattening nested JSON structures. Glue can be configured to support exactly-once semantics through Spark checkpointing and transactional writes to Redshift, making it reliable for critical data pipelines. It handles varying file sizes and can be easily updated as transformation logic evolves.

Option A is incorrect because Lambda has execution time and memory limits, making it unsuitable for large JSON files or complex transformations, and achieving exactly-once requires careful idempotency design. Option B is incorrect because the Redshift COPY command loads raw JSON without transformation. Option D is incorrect because Athena is primarily for querying data in S3, not for performing ETL transformations and loading into Redshift.

Practice this question →

520

MCQmedium

Refer to the exhibit. A data engineer is troubleshooting an AWS Lambda function that reads from an S3 bucket and writes to a Kinesis Data Stream. The Lambda function fails with an AccessDeniedException when calling the kinesis:PutRecords API. Which change is needed to the IAM policy?

A.Add s3:PutObject permission to the policy

B.Change the resource ARN for Kinesis to a wildcard

C.Change the resource ARN for Kinesis to include the correct stream name

D.Add kinesis:PutRecords permission to the policy

AnswerC

Correct. The resource ARN must match the stream name. Using the correct stream name fixes the error and follows security best practices.

Why this answer

The Lambda function's IAM policy grants kinesis:PutRecords permission but the resource ARN does not match the target Kinesis Data Stream. The AccessDeniedException occurs because IAM evaluates the resource ARN against the stream's ARN and denies access on mismatch. The correct fix is to change the resource ARN to include the correct stream name (option C).

While using a wildcard (option B) would also resolve the error, it is not the recommended approach because it violates the principle of least privilege. Option A is incorrect because the error is from Kinesis, not S3. Option D is incorrect because the kinesis:PutRecords action is already present in the policy.

Exam trap

The DEA-C01 exam often tests the misconception that adding the missing API action (kinesis:PutRecords) is the fix, but here the action is already present and the error stems from an incorrect resource ARN. The exam expects the specific stream name to be used, aligning with AWS best practices for least privilege.

How to eliminate wrong answers

Option A is wrong because the error is from Kinesis, not S3; adding s3:PutObject does not address the kinesis:PutRecords AccessDeniedException. Option C is wrong because the policy already includes a specific stream name, but it is incorrect; changing to a wildcard is the fix, not specifying a different name. Option D is wrong because the policy already includes kinesis:PutRecords permission; the issue is the resource ARN restriction, not a missing action.

Practice this question →

521

MCQhard

A company ingests clickstream data into Amazon S3 via Kinesis Data Firehose. The data arrives in 20 MB files every 2 minutes. The data engineering team needs to transform nested JSON into a flat structure before loading into Amazon Redshift. Which approach is most cost-effective and scalable?

A.Create an AWS Glue ETL job that runs on a schedule, using dynamic frames to flatten the data and write to S3 in Parquet

B.Run an Amazon EMR cluster with Spark to flatten the data and write back to S3

C.Use AWS Lambda to transform each file as it arrives in S3

D.Use Amazon Redshift Spectrum to query the nested JSON directly and create a view

AnswerA

Glue's dynamic frames natively handle nested JSON and can run cost-effectively on a schedule.

Why this answer

AWS Glue ETL jobs are designed for serverless, scalable data transformation, and using dynamic frames to flatten nested JSON and write to Parquet is both cost-effective (pay per DPU-hour) and scalable (automatically handles data volume). The 20 MB files arriving every 2 minutes are well-suited for Glue's batch processing, and Parquet output optimizes Redshift loading via COPY commands.

Exam trap

The trap here is that candidates overestimate Lambda's suitability for file transformations, overlooking its payload and timeout constraints, while underestimating Glue's efficiency for small, frequent batch jobs compared to the overhead of a full EMR cluster.

How to eliminate wrong answers

Option B is wrong because running an Amazon EMR cluster with Spark for this small, frequent workload (20 MB every 2 minutes) incurs significant overhead from cluster provisioning and management, making it less cost-effective than serverless Glue. Option C is wrong because AWS Lambda has a 15-minute timeout and 6 MB invocation payload limit, making it unsuitable for transforming 20 MB files (even with streaming, it would require chunking and complex orchestration). Option D is wrong because Redshift Spectrum queries nested JSON directly without flattening, and creating a view does not transform the data into a flat structure required for loading into Redshift; it only provides a query-time abstraction.

Practice this question →

522

Multi-Selectmedium

A company wants to use AWS Glue to transform data stored in Amazon S3. The data is partitioned by date and includes both CSV and Parquet files. The transformation should be optimized for cost and performance. Which THREE actions should the data engineer take? (Choose THREE.)

Select 3 answers

A.Run a crawler to update the schema before each job run.

B.Use partition pruning by filtering on the date column in the ETL script.

C.Use job bookmarks to process only new data.

D.Increase the number of DPUs to the maximum allowed.

E.Convert all files to Parquet format before processing.

AnswersB, C, E

Reduces data scanned.

Why this answer

Options B, C, and E are correct. Partition pruning (B) reduces the amount of data scanned by filtering on the date column, lowering cost and improving performance. Job bookmarks (C) track processed data, preventing reprocessing and saving time and cost.

Converting files to Parquet (E) reduces data size and enhances query performance due to its columnar format. Option A is incorrect because running a crawler before each job creates unnecessary overhead; the schema can be defined once or crawled periodically. Option D is incorrect because increasing DPUs to the maximum is costly and does not optimize performance; proper data partitioning and format are more effective.

Practice this question →

523

MCQmedium

Refer to the exhibit. A data engineer creates an AWS Glue job using this CloudFormation template. The job processes new data files in S3 and uses job bookmarks to track processed files. After initial success, the job runs again but processes all files again instead of only new ones. What is the most likely cause?

A.The job bookmark option is set to 'job-bookmark-disable'

B.The enable-metrics parameter is set to true

C.The MaxRetries parameter is set to 0

D.The S3 input path does not have a partitioning scheme or timestamp to identify new files

AnswerD

Job bookmarks rely on partition structure or file timestamps to track progress.

Why this answer

AWS Glue job bookmarks rely on the structure of the input data to identify new files. Without a partitioning scheme or a timestamp-based naming convention in the S3 path, Glue cannot determine which files are new; it falls back to reprocessing all files. The job bookmark feature tracks processed files by examining the S3 path and file metadata, so a flat or non-partitioned structure prevents it from distinguishing new files from old ones.

Exam trap

The trap here is that candidates often assume job bookmarks automatically track any new files in S3, but they fail to realize that without a partitioning scheme or timestamp in the path, Glue cannot differentiate new files from existing ones, leading to full reprocessing.

How to eliminate wrong answers

Option A is wrong because if the job bookmark option were set to 'job-bookmark-disable', the job would never use bookmarks and would always process all files, but the question states the job initially succeeded and then reprocessed all files, implying bookmarks were enabled initially but failed to track new files. Option B is wrong because setting 'enable-metrics' to true enables CloudWatch metrics for monitoring, which has no effect on job bookmark behavior or file reprocessing. Option C is wrong because 'MaxRetries' set to 0 controls the number of retry attempts for the job run, not the bookmark tracking or file selection logic.

Practice this question →

524

MCQeasy

A media company is building a data pipeline to ingest user activity logs from multiple sources into Amazon S3. The logs are JSON files generated every minute. The company wants to use Amazon Athena to query the logs with minimal latency and cost. The current approach is to use AWS Kinesis Data Firehose to deliver the logs to S3 with a prefix like 'logs/2024/01/01/00/file.json'. However, when running Athena queries, the team notices high query costs because Athena scans all files in the 'logs/' prefix even when querying for a specific date. What should the team do to reduce the amount of data scanned by Athena?

A.Create an Athena view that filters by date.

B.Increase the number of partitions by using a more granular prefix like 'logs/2024/01/01/00/00/'.

C.Convert the JSON files to Apache Parquet format using AWS Glue ETL jobs.

D.Create a Hive-style partition structure in S3 with keys like 'year=2024/month=01/day=01/hour=00/' and update the Glue Data Catalog accordingly.

AnswerD

Partition pruning allows Athena to scan only relevant directories, reducing costs.

Why this answer

Creating a Hive-style partition structure (e.g., year=2024/month=01/day=01/) enables Athena to perform partition pruning. When querying for a specific date, Athena scans only the relevant partition, reducing data scanned and cost. Option A is incorrect because Athena views do not reduce data scanning; they just store query logic.

Option B is incorrect because more granular prefixes without a partition structure (like Hive-style) do not enable partition pruning in Athena; Athena treats the prefix as a folder and still scans all files. Option C is incorrect because while converting to Parquet reduces scan size due to columnar storage and compression, it does not address the lack of partitioning; the main issue is full scan of all files regardless of format.

Practice this question →

525

MCQmedium

A company runs a nightly ETL job using AWS Glue. The job reads data from a JDBC connection to an on-premises MySQL database. The job fails with an error indicating that the connection pool is exhausted. What is the most likely cause and solution?

A.The database is not reachable due to network issues. Check VPC and security groups.

B.The Glue job is hitting the AWS Glue connection pool limit. Increase the Glue connection pool size.

C.The database credentials are expired. Rotate the password in AWS Secrets Manager.

D.The Glue job is using too many executors, exhausting the database connections. Reduce the number of DPUs or increase the database max connections.

AnswerD

Glue can open multiple connections; reducing parallelism or scaling database helps.

Why this answer

AWS Glue jobs distribute work across multiple executors, each of which opens its own JDBC connection to the source database. When the number of executors (controlled by DPUs) exceeds the database's configured maximum connections, the database connection pool is exhausted, causing the error. Reducing the number of DPUs or increasing the database's max_connections setting resolves the issue.

Exam trap

The trap here is that candidates confuse a database-side connection pool exhaustion with an AWS Glue service limit or network issue, leading them to incorrectly choose options about Glue connection pools or VPC configurations.

How to eliminate wrong answers

Option A is wrong because a network connectivity issue (e.g., VPC or security group misconfiguration) would typically result in a timeout or 'cannot connect' error, not a 'connection pool exhausted' error. Option B is wrong because AWS Glue does not have a configurable 'connection pool size' for JDBC connections; the pool exhaustion is on the database side, not Glue's internal pool. Option C is wrong because expired credentials would cause an authentication failure (e.g., 'Access denied for user'), not a connection pool exhaustion error.

Practice this question →

← PreviousPage 7 of 8 · 591 questions totalNext →

Ready to test yourself?

Try a timed practice session using only Data Ingestion and Transformation questions.

Start 20-question session