CCNA Data Ingestion and Transformation Questions — Page 5 of 8

301

MCQeasy

A data engineer needs to ingest data from an external HTTP API into Amazon S3. The API returns JSON data for a list of users, updated hourly. The engineer wants to use a serverless solution with minimal operational overhead. Which AWS service should the engineer use?

A.Amazon Kinesis Data Firehose with a custom HTTP endpoint.

B.AWS Lambda function triggered by CloudWatch Events.

C.Amazon AppFlow with an HTTP connector on a scheduled flow.

D.AWS Glue ETL job triggered by EventBridge.

AnswerC

AppFlow is serverless and designed for API ingestion.

Why this answer

Amazon AppFlow with an HTTP connector on a scheduled flow is the correct choice because it provides a fully managed, serverless integration that directly connects to external HTTP APIs, retrieves JSON data, and writes it to Amazon S3 on a scheduled basis (e.g., hourly) without requiring any custom code or infrastructure management. This minimizes operational overhead while meeting the ingestion requirements.

Exam trap

The trap here is that candidates often assume AWS Lambda is the default serverless choice for any custom integration, overlooking that AppFlow provides a purpose-built, no-code solution for SaaS and HTTP API ingestion with lower operational overhead.

How to eliminate wrong answers

Option A is wrong because Amazon Kinesis Data Firehose does not support a custom HTTP endpoint as a source; it can only ingest data from Kinesis Data Streams, Amazon CloudWatch, AWS IoT, or custom sources via the Kinesis Agent, not directly from an external HTTP API. Option B is wrong because while an AWS Lambda function triggered by CloudWatch Events can poll an HTTP API and write to S3, it requires custom code for HTTP requests, error handling, and data transformation, increasing operational overhead compared to a managed service like AppFlow. Option D is wrong because AWS Glue ETL jobs are designed for batch data transformation and processing, not for direct ingestion from external HTTP APIs; they would require a custom script to fetch the API data, adding complexity and overhead.

Practice this question →

302

MCQeasy

A data engineer needs to ingest on-premises CSV files into Amazon S3 every hour. The files are less than 1 GB each. Which service is the most cost-effective and requires the least operational overhead?

A.AWS DataSync

B.Amazon Kinesis Data Firehose

C.AWS Snowball Edge

D.AWS Database Migration Service (DMS)

AnswerA

DataSync automates scheduled transfers from on-premises to S3.

Why this answer

AWS DataSync is the most cost-effective and least overhead option for scheduled, recurring transfers of on-premises CSV files to S3. It provides a simple agent-based setup, supports hourly scheduling, and handles files under 1GB efficiently without complex configuration. In contrast, Amazon Kinesis Data Firehose is designed for streaming data ingestion, not batch file transfers; AWS Snowball Edge is intended for large-scale offline data migrations, not hourly incremental transfers; and AWS Database Migration Service (DMS) is specialized for migrating databases, not file transfers.

Practice this question →

303

MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format. The delivery stream is configured with a buffer size of 5 MB and a buffer interval of 60 seconds. However, the data engineer notices that S3 objects are being created with sizes much smaller than 5 MB. What is a likely cause?

A.The data is being compressed before delivery, reducing object size.

B.The incoming data rate is too low, causing the buffer interval to trigger before reaching the buffer size.

C.The data transformation lambda is splitting records into smaller ones.

D.The S3 bucket is configured with a lifecycle policy that splits objects.

AnswerB

Buffer interval triggers first.

Why this answer

Kinesis Data Firehose delivers data to S3 when either the buffer size (5 MB) or the buffer interval (60 seconds) is reached, whichever occurs first. If the incoming data rate is low, the buffer interval will expire before accumulating 5 MB of data, resulting in smaller S3 objects.

Exam trap

The trap here is that candidates may assume the buffer size is a hard minimum that must be reached before delivery, but Firehose uses an 'or' condition between buffer size and buffer interval, so low data rate causes interval-based delivery of small objects.

How to eliminate wrong answers

Option A is wrong because compression reduces the size of data after buffering, but the buffer size limit is based on the uncompressed data; compression does not cause smaller objects to be created before the buffer interval triggers. Option C is wrong because a data transformation Lambda can modify records but does not inherently split records into smaller ones; it processes records as a batch and returns them, and any splitting would be a custom logic not default behavior. Option D is wrong because S3 lifecycle policies manage object transitions or deletions after objects are created; they do not split objects during delivery.

Practice this question →

304

MCQmedium

Refer to the exhibit. An IAM policy for an AWS Lambda function. The Lambda function is triggered by an S3 event (object created) and needs to read from a Kinesis stream. However, the function fails with access denied when trying to read from Kinesis. What is the most likely cause?

A.The Lambda function is not in the same region as the Kinesis stream

B.The Lambda function does not have permission to list S3 buckets

C.The Kinesis stream is encrypted with a customer managed KMS key, and the Lambda function lacks kms:Decrypt permission

D.The S3 bucket policy denies access to the Lambda function

AnswerC

If the stream uses SSE-KMS, Lambda needs kms:Decrypt on the key.

Why this answer

When a Kinesis stream is encrypted with a customer managed KMS key, the Lambda function must have the `kms:Decrypt` permission on that key to read data from the stream. Without this permission, the Lambda function will receive an access denied error even if it has the necessary Kinesis actions (e.g., `kinesis:GetRecords`) allowed in its IAM policy. The S3 event trigger only invokes the function; it does not grant Kinesis access.

Exam trap

The DEA-C01 exam often tests the interaction between Kinesis SSE-KMS and Lambda IAM permissions, trapping candidates who assume that Kinesis read permissions alone are sufficient without considering the KMS key policy.

How to eliminate wrong answers

Option A is wrong because Lambda functions can access Kinesis streams across regions as long as the IAM permissions and network connectivity (e.g., VPC endpoints) are correctly configured; region mismatch does not inherently cause access denied. Option B is wrong because the Lambda function is triggered by an S3 event and only needs permission to read from Kinesis; listing S3 buckets is irrelevant to the Kinesis read failure. Option D is wrong because the S3 bucket policy controls access to the S3 bucket itself, not to Kinesis; the error occurs when reading from Kinesis, not when the S3 event triggers the function.

Practice this question →

305

MCQhard

A company uses AWS Glue to transform data in an S3 data lake. The transformation logic requires joining two large datasets that are each hundreds of gigabytes. The Glue job runs out of memory. Which configuration change will most likely resolve this issue?

A.Repartition the data before the join.

B.Increase the number of DPUs for the Glue job.

C.Use a different file format like Parquet with compression.

D.Use the 'spark.sql.autoBroadcastJoinThreshold' setting to broadcast the smaller table.

AnswerB

More DPUs provide more memory and parallelism, helping the join fit in memory.

Why this answer

Increasing the number of DPUs provides more memory for the join operation. Glue automatically distributes data across workers, so more workers mean more total memory.

Practice this question →

306

MCQeasy

A data engineer is setting up an Amazon Kinesis Data Firehose delivery stream to load data into Amazon Redshift. The data is coming from an application that produces JSON records. The engineer needs to transform the data to match the Redshift table schema. Which approach is the MOST cost-effective and requires the least operational overhead?

A.Use AWS Glue as a transformation step between Firehose and Redshift, with a trigger on S3.

B.Use Kinesis Data Firehose with direct PUT to Redshift and rely on Redshift's COPY command to transform.

C.Configure a Lambda function in the Firehose delivery stream to transform records before delivery.

D.Use the Kinesis Client Library (KCL) to consume the stream, transform in an EC2 instance, and then load to Redshift.

AnswerC

Firehose supports Lambda for data transformation with minimal overhead.

Why this answer

Kinesis Data Firehose natively supports invoking a Lambda function as a transformation step within the delivery stream. This allows the engineer to write a simple Lambda function that parses the incoming JSON records and transforms them to match the Redshift table schema, all without provisioning or managing any additional infrastructure. This approach is the most cost-effective (pay per invocation) and requires the least operational overhead since Firehose handles the orchestration, retries, and delivery to Redshift automatically.

Exam trap

The trap here is that candidates often overestimate the transformation capabilities of Redshift's COPY command, mistakenly believing it can perform complex record-level transformations, when in fact it only supports basic data mapping and format parsing, not arbitrary JSON restructuring.

How to eliminate wrong answers

Option A is wrong because inserting AWS Glue as an intermediate step between Firehose and Redshift introduces unnecessary complexity, cost (Glue jobs run on a per-DPU-hour basis), and latency, as Glue is designed for batch ETL, not real-time streaming transformations. Option B is wrong because Redshift's COPY command does not perform record-level transformations; it only maps source fields to target columns and can apply basic data format conversions (e.g., JSON parsing via 'jsonpaths'), but it cannot restructure or compute new fields from the JSON payload. Option D is wrong because using the Kinesis Client Library (KCL) on an EC2 instance requires manual provisioning, scaling, and management of the EC2 fleet, which incurs significant operational overhead and cost compared to the serverless Lambda integration within Firehose.

Practice this question →

307

MCQmedium

A company is building a data lake on Amazon S3 and wants to ingest data from multiple AWS services (CloudTrail, VPC Flow Logs, and ALB logs). The data should be stored in a central S3 bucket with a common partitioning scheme. Which service can be used to collect and centralize this data with minimal configuration?

A.Use AWS Data Pipeline to copy logs from each source S3 bucket to the central bucket.

B.Use AWS Glue to crawl the logs from each source and write to a central S3 bucket.

C.Set up Amazon Kinesis Data Firehose to ingest logs from each service and write to S3.

D.Configure each source service to deliver logs directly to the central S3 bucket.

AnswerD

CloudTrail, VPC Flow Logs, and ALB can all deliver to S3 directly.

Why this answer

CloudTrail, VPC Flow Logs, and ALB logs can each be configured to deliver logs directly to a specified S3 bucket, including a central bucket, with no intermediary service required. This approach minimizes configuration overhead and avoids data movement costs, as each service writes natively to S3 using its own built-in delivery mechanism.

Exam trap

The trap here is that candidates often overcomplicate the solution by choosing a data pipeline or ETL service (like Data Pipeline or Glue) when the simplest and most efficient method is to configure each source service to write directly to the central S3 bucket, leveraging native AWS integrations.

How to eliminate wrong answers

Option A is wrong because AWS Data Pipeline is designed for scheduled data movement and transformation between data stores, not for real-time log ingestion from multiple AWS services; it would require custom pipeline definitions and adds unnecessary complexity. Option B is wrong because AWS Glue is an ETL service for crawling, cataloging, and transforming data, not a log collection or delivery service; it cannot natively ingest logs from CloudTrail, VPC Flow Logs, or ALB logs without first having the data in S3. Option C is wrong because Amazon Kinesis Data Firehose can ingest streaming data but does not natively subscribe to CloudTrail, VPC Flow Logs, or ALB logs; these services do not send data to Firehose directly, requiring additional setup like CloudWatch Logs subscriptions or custom agents.

Practice this question →

308

MCQeasy

A data engineer needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The network bandwidth is limited to 100 Mbps. The transfer must be completed within one week. Which service should be used?

A.AWS DataSync

B.AWS Snowball

C.Amazon CloudFront

D.AWS Database Migration Service (DMS)

AnswerB

Physical device for large data transfers.

Why this answer

AWS Snowball is the correct choice because transferring 50 TB over a 100 Mbps network would take approximately 46 days (50 TB * 8 bits/byte / 100 Mbps / 86400 seconds/day), far exceeding the one-week deadline. Snowball provides a physical storage device that can be shipped to the on-premises location, allowing data to be loaded locally and shipped to AWS, bypassing network bandwidth constraints entirely.

Exam trap

The trap here is that candidates may underestimate the time required for online transfer and choose AWS DataSync, failing to calculate that 50 TB at 100 Mbps takes over 46 days, not one week, and overlooking Snowball's physical shipping approach for offline data transfer.

How to eliminate wrong answers

Option A is wrong because AWS DataSync is designed for online data transfer over the network, and at 100 Mbps, it would take over 46 days to transfer 50 TB, which does not meet the one-week requirement. Option C is wrong because Amazon CloudFront is a content delivery network (CDN) for caching and distributing content to edge locations, not a data transfer service for ingesting large volumes of historical data into S3. Option D is wrong because AWS Database Migration Service (DMS) is specialized for migrating databases (e.g., relational, NoSQL) and does not support transferring HDFS files or large-scale file-based data to S3.

Practice this question →

309

MCQeasy

A data engineer needs to ingest data from an Amazon RDS for PostgreSQL database into Amazon S3 on a daily basis. The data volume is approximately 500 GB per day. Which service is most appropriate for this task?

A.AWS Database Migration Service (DMS) with continuous replication

B.Amazon Athena with federated query to RDS

C.Amazon EMR with Spark job

D.AWS Glue with a scheduled ETL job

AnswerD

AWS Glue can run scheduled ETL jobs to extract from RDS and load to S3.

Why this answer

AWS Glue with a scheduled ETL job is the most appropriate choice because it provides a fully managed, serverless ETL service that can efficiently extract 500 GB of data daily from Amazon RDS for PostgreSQL and write it to Amazon S3. Glue can handle large volumes via its distributed Spark-based execution, and scheduling ensures the daily cadence without manual intervention.

Exam trap

The trap here is that candidates often overcomplicate by choosing EMR (Option C) due to familiarity with Spark, overlooking that AWS Glue provides the same Spark-based ETL capability in a fully managed, serverless form that is more cost-effective and simpler for scheduled batch ingestion.

How to eliminate wrong answers

Option A is wrong because AWS DMS with continuous replication is designed for ongoing, near-real-time data synchronization or migration, not for a daily batch ingestion of 500 GB; continuous replication would incur unnecessary overhead and cost for a scheduled daily load. Option B is wrong because Amazon Athena with federated query to RDS is an interactive query engine that can read data directly from RDS, but it is not designed for ingesting or moving large volumes of data into S3; it would require additional steps to write results and is inefficient for 500 GB daily transfers. Option C is wrong because Amazon EMR with a Spark job is a valid but overkill and more complex option; it requires provisioning and managing clusters, whereas AWS Glue offers a simpler, serverless alternative that is better suited for this scheduled batch ETL workload.

Practice this question →

310

MCQmedium

A company uses AWS DMS to migrate a 2 TB PostgreSQL database to Amazon Aurora PostgreSQL. The migration is taking longer than expected due to the initial load. Which AWS service can be used to accelerate the initial load by transferring the database files directly?

A.AWS Snowball

B.Amazon S3 Transfer Acceleration

C.AWS Direct Connect

D.Amazon Kinesis Data Firehose

AnswerA

Snowball allows physical transfer of data, which can be faster than network transfer for very large datasets.

Why this answer

AWS Snowball is a petabyte-scale data transport solution that uses physical storage devices to transfer large amounts of data into and out of AWS. For a 2 TB PostgreSQL database, the initial load via DMS over the network can be slow due to bandwidth constraints, especially for large datasets. By using Snowball, you can export the database files (e.g., using pg_dump or physical file copy) to the device, ship it to AWS, and have the data loaded directly into Amazon Aurora PostgreSQL, bypassing the network bottleneck and significantly accelerating the initial load.

Exam trap

The trap here is that candidates may assume network-based acceleration services (like S3 Transfer Acceleration or Direct Connect) are sufficient for large migrations, overlooking the fact that physical data transport (Snowball) is the only option that completely avoids network transfer for the initial load.

How to eliminate wrong answers

Option B (Amazon S3 Transfer Acceleration) is wrong because it only speeds up uploads to S3 over the internet using optimized network paths and edge locations, but it does not transfer database files directly to Aurora PostgreSQL; DMS would still need to read from S3 and apply the data, which does not bypass the network transfer for the initial load. Option C (AWS Direct Connect) is wrong because it provides a dedicated network connection from on-premises to AWS, which can improve bandwidth and latency but still requires the full 2 TB to traverse the network; it does not eliminate the network transfer bottleneck for large initial loads. Option D (Amazon Kinesis Data Firehose) is wrong because it is a real-time streaming data ingestion service designed for streaming data (e.g., logs, events) into S3, Redshift, or Elasticsearch, not for bulk transferring database files for a migration; it cannot handle the initial load of a 2 TB database directly to Aurora PostgreSQL.

Practice this question →

311

MCQmedium

A company uses AWS DMS to migrate data from an on-premises Oracle database to Amazon RDS for PostgreSQL. The migration is ongoing with continuous replication. The data engineer notices that some changes are not being captured in the target database. What is the MOST likely cause?

A.The VPC peering connection between on-premises and AWS is down.

B.The DMS task's table mapping is incorrectly configured.

C.DMS is not publishing task logs to CloudWatch Logs.

D.The source Oracle database is not configured to retain archived redo logs for a sufficient period.

AnswerD

DMS requires archived logs to capture changes; if logs are purged, changes are lost.

Why this answer

In AWS DMS continuous replication (CDC), the service reads changes from the source Oracle database's archived redo logs. If the logs are rotated or deleted before DMS has a chance to process them, changes are lost. Option D is correct because insufficient retention of archived redo logs is the most likely cause of missing changes in the target.

Exam trap

The trap here is that candidates often assume missing changes are due to network or configuration errors (like VPC or table mapping), but the real issue is the source database's log retention policy, which is a subtle but critical requirement for CDC with DMS.

How to eliminate wrong answers

Option A is wrong because a down VPC peering connection would cause a complete connectivity failure, not selective missing changes; DMS would report a connection error. Option B is wrong because incorrect table mapping would cause specific tables or columns to be missing entirely, not intermittent missing changes during CDC. Option C is wrong because DMS not publishing logs to CloudWatch Logs affects monitoring and troubleshooting, not the actual data capture or replication process.

Practice this question →

312

MCQeasy

A company is using Amazon Kinesis Data Firehose to ingest clickstream data from a website into an S3 bucket. The data is then analyzed using Amazon Athena. Recently, the company noticed that Athena queries are returning incomplete results for the last 30 minutes of data. The Firehose delivery stream is configured to buffer data for 60 seconds or 5 MB before delivering to S3. The S3 bucket has a lifecycle policy that transitions objects to Amazon S3 Glacier after 30 days. The IAM role for Firehose has permissions to write to S3 and access a CloudWatch Logs group. The engineer checks the Firehose monitoring and sees that the delivery rate is healthy, but the 'S3.Bytes' metric shows a spike in the last hour. The 'BackupToS3.Bytes' metric is zero. What is the MOST likely cause of the missing data?

A.The lifecycle policy is transitioning data before Athena can query it.

B.The backup is enabled and data is being sent to the backup bucket instead.

C.The data is still being buffered in Firehose and has not yet been delivered to S3.

D.The IAM role for Firehose does not have permissions to write to the S3 bucket.

AnswerC

The data is still being buffered in Firehose. With buffer settings of 60 seconds or 5 MB, data can be held for up to 60 seconds before delivery. Athena queries only see data that has been delivered to S3, so data still in the buffer is missing from query results.

Why this answer

The data is still being buffered in Firehose and has not yet been delivered to S3. The buffer settings (60 seconds or 5 MB) mean data can be held in the buffer for up to 60 seconds before being written to S3. For the last 30 minutes, some data may still be in the buffer and not yet delivered.

Athena queries only see data that has been delivered to S3. Option A (lifecycle policy) would not affect recent data. Option B (backup) is unrelated and the 'BackupToS3.Bytes' metric is zero, indicating backup is not active.

Option D (IAM permissions) would cause errors, not missing data.

Practice this question →

313

Multi-Selecteasy

A company is designing a data lake on Amazon S3. The data ingestion pipeline must handle both structured and unstructured data. The data must be cataloged for easy discovery. Which THREE services should be included in the solution? (Choose THREE.)

Select 3 answers

A.Amazon S3

B.AWS Glue Data Catalog

C.Amazon Athena

D.Amazon RDS

E.Amazon Redshift

AnswersA, B, C

S3 is the core storage for data lakes.

Why this answer

Amazon S3 is the core storage layer for the data lake, providing scalable, durable, and cost-effective object storage for both structured and unstructured data. AWS Glue Data Catalog acts as the central metadata repository, enabling data discovery and schema management across the data lake. Amazon Athena allows serverless querying of data directly from S3 using standard SQL, leveraging the Glue Data Catalog for schema-on-read, which is essential for easy discovery and analysis.

Exam trap

The trap here is that candidates often confuse Amazon Redshift as a data lake solution because it can query data in S3 via Redshift Spectrum, but it is not a cataloging service and does not handle unstructured data ingestion natively, making it incorrect for this specific requirement.

Practice this question →

314

MCQeasy

A data engineer needs to transform JSON data from a Kinesis Data Stream into Parquet format and store it in an S3 data lake. The transformation includes simple field mapping and data type conversions. Which AWS service is the most cost-effective for performing this transformation in near-real-time?

A.Amazon Athena with CTAS (CREATE TABLE AS SELECT)

B.AWS Lambda function triggered by Kinesis Data Firehose

C.AWS Glue ETL job

D.Amazon EMR with Spark Streaming

AnswerB

Lambda can be invoked by Firehose for record transformation and can output Parquet; it is serverless and cost-effective for near-real-time.

Why this answer

AWS Lambda functions triggered by Kinesis Data Firehose are the most cost-effective solution for near-real-time transformations because Lambda allows you to perform lightweight field mapping and data type conversions on streaming data with a pay-per-invocation model, while Firehose handles buffering, compression, and direct delivery to S3 in Parquet format. This serverless approach avoids the overhead and cost of provisioning clusters or running continuous jobs, making it ideal for simple transformations on high-frequency, low-latency streams.

Exam trap

The trap here is that candidates often choose AWS Glue ETL jobs or Amazon EMR because they associate them with data transformation, but the question specifies 'near-real-time' and 'most cost-effective' for simple transformations, which points to the serverless, pay-per-use Lambda integration with Firehose rather than provisioned cluster-based solutions.

How to eliminate wrong answers

Option A is wrong because Amazon Athena with CTAS is a batch query engine that reads data from S3, not a streaming ingestion service; it cannot process data from a Kinesis Data Stream in near-real-time and would require intermediate storage, adding latency and cost. Option C is wrong because AWS Glue ETL jobs are designed for batch processing and incur costs for DPU hours even when idle; they are not optimized for continuous, low-latency streaming transformations and would be overkill for simple field mapping. Option D is wrong because Amazon EMR with Spark Streaming requires provisioning and managing a cluster of EC2 instances, which incurs significant cost even for small workloads, and is more complex than necessary for simple transformations on a single Kinesis stream.

Practice this question →

315

MCQmedium

A company uses AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job fails intermittently with a 'MemoryError'. What is the MOST likely cause?

A.The Glue job worker type is too small for the data volume

B.The Glue job uses too many DynamicFrames

C.The S3 output bucket is in a different region

D.The Kinesis stream has insufficient shards

AnswerA

Small worker type leads to out-of-memory errors when data volume exceeds capacity.

Why this answer

The 'MemoryError' in AWS Glue indicates that the worker type allocated to the job does not have sufficient memory to process the data volume. Glue workers (Standard, G.1X, G.2X) have fixed memory allocations (e.g., 16 GB for Standard), and if the streaming data from Kinesis exceeds this, the job fails. Increasing the worker type or the number of workers resolves this.

Exam trap

The trap here is that candidates confuse memory errors with throttling or connectivity issues, leading them to pick insufficient shards (Option D) or cross-region problems (Option C), when the root cause is almost always an undersized worker type for the data volume.

How to eliminate wrong answers

Option B is wrong because using too many DynamicFrames does not directly cause a MemoryError; DynamicFrames are lazy transformations and memory issues arise from data volume or worker size, not the number of frames. Option C is wrong because an S3 output bucket in a different region would cause a cross-region access error (e.g., AccessDenied or timeout), not a MemoryError. Option D is wrong because insufficient Kinesis shards cause throttling (ProvisionedThroughputExceededException) or data latency, not a memory exhaustion in the Glue job.

Practice this question →

316

MCQmedium

A company uses AWS Glue DataBrew to clean and transform data for analytics. The source data is in Parquet format in Amazon S3. The transformation includes filtering rows and adding calculated columns. What is the MOST cost-effective way to run these transformations on a schedule?

A.Use Amazon EMR with Spark

B.Create a Glue DataBrew recipe and schedule the job using a cron expression

C.Create an AWS Lambda function triggered by S3 events

D.Use AWS Glue ETL with PySpark

AnswerB

DataBrew supports scheduling directly.

Why this answer

AWS Glue DataBrew is purpose-built for visual data preparation, and scheduling a DataBrew recipe job with a cron expression directly meets the requirement to run filtering and column calculations on Parquet data in S3 without writing code. This is the most cost-effective approach as it avoids provisioning or managing compute resources beyond the serverless DataBrew job runs.

Exam trap

The trap here is that candidates may over-engineer the solution by choosing Glue ETL or EMR, assuming that Parquet processing requires custom Spark code, when DataBrew's visual recipes can handle filtering and calculated columns without any code and at lower cost.

How to eliminate wrong answers

Option A is wrong because Amazon EMR with Spark requires provisioning and managing a cluster, which incurs higher costs and operational overhead for simple transformations that DataBrew can handle natively. Option C is wrong because AWS Lambda has a maximum execution timeout of 15 minutes and is not designed for data transformation workloads that may process large Parquet files, making it unsuitable for scheduled batch jobs. Option D is wrong because AWS Glue ETL with PySpark requires writing and maintaining custom Spark code, which is more complex and costly than using DataBrew's visual recipe approach for the described transformations.

Practice this question →

317

MCQhard

A data engineer is designing a data pipeline that ingests CSV files from an FTP server to Amazon S3. The files arrive hourly and each file is about 500 MB. The engineer wants to minimize operational overhead and cost. Which approach is best?

A.Write a Python script in AWS Lambda using boto3 to download from FTP and upload to S3

B.Use AWS Snowball Edge to transfer files weekly

C.Use AWS Transfer for SFTP and point the endpoint to an S3 bucket

D.Deploy an Amazon EC2 instance with a cron job to run wget and aws s3 cp

AnswerC

Fully managed, no servers to manage, direct to S3.

Why this answer

AWS Transfer for SFTP provides a fully managed FTP service that writes directly to S3, eliminating the need to manage servers. Lambda with boto3 is code-heavy; EC2 requires management; Snowball is for large offline transfers.

Practice this question →

318

MCQhard

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is transformed using an AWS Lambda function. Some records fail transformation and are lost because the Lambda function throws an exception. The data engineer needs to capture the failed records for analysis without affecting the pipeline. What should the engineer do?

A.Configure the Firehose delivery stream to send failed records to a backup S3 bucket

B.Increase the buffer size of the Firehose stream

C.Disable the Lambda transformation and process all records in batch later

D.Modify the Lambda function to write failed records to Amazon DynamoDB

AnswerA

Firehose can be configured to send failed records to a backup S3 bucket.

Why this answer

Amazon Kinesis Data Firehose natively supports a 'backup' or 'error output' configuration: when a Lambda transformation fails (e.g., throws an exception), Firehose can automatically route the failed records to a designated Amazon S3 bucket for failed data. This allows the engineer to capture and analyze the failed records without blocking or altering the main delivery pipeline, as the stream continues processing successfully transformed records.

Exam trap

The trap here is that candidates often confuse Firehose's 'backup' mode (which copies all records) with the 'error output' configuration (which only captures failed records), or they assume that modifying the Lambda function to handle failures is the only option, missing the native Firehose feature that requires no code changes.

How to eliminate wrong answers

Option B is wrong because increasing the buffer size (e.g., MBs or interval) only affects how long Firehose waits before delivering a batch; it does not capture or handle Lambda transformation failures. Option C is wrong because disabling the Lambda transformation would stop all data transformation, which is a core requirement of the pipeline, and processing records in batch later would not recover records already lost during the stream. Option D is wrong because while writing failed records to DynamoDB from within the Lambda function is possible, it requires modifying the Lambda code and does not leverage Firehose's built-in error handling; moreover, DynamoDB has a 400 KB item size limit and is not designed for high-volume streaming failure capture, making it less suitable than the native S3 backup bucket.

Practice this question →

319

MCQeasy

A company needs to transfer 20 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The network bandwidth is limited and the transfer must complete within one week. Which service should the company use?

A.Amazon S3 Transfer Acceleration

B.AWS Snowball Edge

C.AWS Direct Connect with DataSync

D.AWS DataSync over a VPN connection

AnswerB

Snowball is a physical device that can transfer large data quickly.

Why this answer

AWS Snowball Edge is designed for large-scale data transfers (up to 80 TB per device) over limited bandwidth. It physically ships the data, bypassing network constraints entirely. The requirement of 20 TB within one week with limited bandwidth makes Snowball the ideal choice.

Option A is wrong because S3 Transfer Acceleration still uses the internet and cannot guarantee completion within a week with limited bandwidth. Option C is wrong because Direct Connect provides a dedicated network connection but requires sufficient bandwidth; if bandwidth is limited, it cannot complete the transfer in time. Option D is wrong because DataSync over VPN still relies on the internet bandwidth and would not meet the deadline.

Practice this question →

320

MCQeasy

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed into Parquet format and stored in Amazon S3. Which AWS service can perform the transformation in near real-time with minimal operational overhead?

A.Amazon EMR cluster running Spark Streaming

B.AWS Glue ETL job triggered by Kinesis stream

C.Amazon Kinesis Data Firehose with a transformation Lambda function

D.Amazon Kinesis Data Analytics for Apache Flink

AnswerC

Kinesis Data Firehose can invoke a Lambda function to convert data to Parquet and deliver to S3.

Why this answer

Amazon Kinesis Data Firehose is the fully managed service designed to load streaming data into S3 with built-in data format conversion. By attaching a Lambda transformation function, you can convert incoming records to Parquet format in near real-time without managing any infrastructure, making it the lowest-operational-overhead choice for this task.

Exam trap

The trap here is that candidates often confuse AWS Glue's batch ETL capabilities with near real-time streaming, or assume that Kinesis Data Analytics is the only option for transformations, overlooking Firehose's simpler managed conversion feature.

How to eliminate wrong answers

Option A is wrong because Amazon EMR running Spark Streaming requires you to provision and manage a cluster, incurring significant operational overhead for a simple transformation task. Option B is wrong because AWS Glue ETL jobs are batch-oriented and not designed for near real-time streaming; they cannot be directly triggered by a Kinesis stream in a low-latency manner. Option D is wrong because Amazon Kinesis Data Analytics for Apache Flink is intended for complex stream processing and analytics (e.g., windowed aggregations), not for simple data format conversion to Parquet, and it requires more operational effort than Firehose.

Practice this question →

321

MCQmedium

A company uses AWS Glue to transform data in S3. The transformation job reads Parquet files, filters rows, and writes to another S3 bucket. The job takes longer than expected. Which change would MOST likely reduce the job execution time?

A.Use a single large file instead of multiple small files.

B.Reduce the number of partitions in the output data.

C.Convert the input files from Parquet to CSV format.

D.Increase the number of DPUs allocated to the Glue job.

AnswerD

More DPUs allow more parallel processing, reducing runtime.

Why this answer

Increasing the number of DPUs (Data Processing Units) allocated to the Glue job directly increases the parallelism of the Apache Spark-based execution environment. Since the job reads Parquet files, filters rows, and writes output, a bottleneck in compute capacity is the most likely cause of prolonged execution time. More DPUs allow Spark to distribute the workload across more executors, reducing overall runtime.

Exam trap

The trap here is that candidates often assume optimizing file format or output partitioning will always improve performance, but for a compute-bound transformation job, increasing parallelism via DPUs is the most direct solution.

How to eliminate wrong answers

Option A is wrong because using a single large file instead of multiple small files would reduce parallelism in Spark's file scanning, potentially increasing execution time due to less efficient task distribution. Option B is wrong because reducing the number of partitions in the output data affects the write phase but does not address the root cause of slow processing during the read and filter stages. Option C is wrong because converting from Parquet to CSV would increase I/O and CPU overhead due to CSV's lack of compression, schema enforcement, and columnar storage, making the job slower, not faster.

Practice this question →

322

MCQhard

A company uses AWS Glue to process data from Amazon RDS MySQL into Amazon S3. The Glue job uses a JDBC connection and runs on a schedule. Recently, the job has been failing with a 'Communications link failure' error. The RDS instance is in a private subnet. Which troubleshooting step should the data engineer take FIRST?

A.Check the Glue job's DPU allocation; increase if too low.

B.Review the Glue job script for data type mismatches.

C.Verify that the Glue job's VPC subnet and security group allow outbound traffic to RDS.

D.Increase the RDS instance's max_connections parameter.

AnswerC

Network connectivity is the first thing to check for link failures.

Why this answer

The 'Communications link failure' error typically indicates a network connectivity issue between the Glue job and the RDS instance. Since the RDS instance is in a private subnet, the Glue job must be configured with a VPC subnet and security group that allows outbound traffic to the RDS instance's security group on port 3306 (MySQL). Without this network path, the JDBC connection cannot be established, making verifying the VPC and security group configuration the first logical troubleshooting step.

Exam trap

The trap here is that candidates often assume 'Communications link failure' is a database-side issue (like connection limits or timeouts) and jump to tuning RDS parameters, when in fact it is most commonly a network connectivity problem in a VPC environment.

How to eliminate wrong answers

Option A is wrong because DPU allocation affects job parallelism and memory, not network connectivity; increasing DPUs would not resolve a 'Communications link failure' caused by a missing network path. Option B is wrong because data type mismatches typically cause runtime errors during data conversion, not connection-level failures like 'Communications link failure'. Option D is wrong because increasing max_connections addresses connection limits on the RDS side, but the error indicates the Glue job cannot even reach the database, not that connections are being refused due to exhaustion.

Practice this question →

323

Multi-Selecthard

A company is running a 10-node Amazon EMR cluster to process data from Amazon S3. The cluster is using Apache Spark for transformations. The data processing is taking longer than expected. Which THREE actions can improve the performance of the Spark jobs on EMR? (Choose THREE.)

Select 3 answers

A.Reduce the number of shuffle partitions.

B.Enable dynamic allocation of executors.

C.Disable speculative execution to reduce redundant tasks.

D.Use a larger instance type for core nodes.

E.Use EMRFS consistent view to ensure data consistency.

AnswersB, D, E

Dynamic allocation allows Spark to scale resources based on workload.

Why this answer

Enabling dynamic allocation of executors allows Amazon EMR to automatically scale the number of executors up or down based on workload demand. This prevents resource underutilization or over-provisioning, which can significantly improve Spark job performance by ensuring that the cluster's resources are efficiently matched to the processing needs of the transformations.

Exam trap

The trap here is that candidates often confuse 'reducing shuffle partitions' (Option A) as a universal performance fix, when in fact it can degrade performance due to data skew and memory issues, while the correct answer focuses on resource elasticity through dynamic allocation.

Practice this question →

324

MCQmedium

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The application reads from a Kinesis data stream and writes results to an Amazon S3 bucket. Recently, the application has been failing with 'ResourceNotFoundException' for the S3 bucket. What is the MOST likely cause?

A.The IAM role used by the application does not have s3:PutObject permission.

B.The S3 bucket ARN is incorrectly specified in the application configuration.

C.The Flink application code specifies the wrong AWS Region for the S3 bucket.

D.The S3 bucket has versioning disabled.

AnswerB

Correct. An incorrect S3 bucket name or ARN in the configuration causes the application to fail with 'ResourceNotFoundException' because the bucket cannot be located.

Why this answer

The 'ResourceNotFoundException' for the S3 bucket indicates that the application cannot find the bucket. This is most likely due to an incorrectly specified bucket name or ARN in the application configuration. If the bucket name is misspelled or the ARN is malformed, the Kinesis Data Analytics application cannot resolve the bucket resource.

Option A is incorrect because missing 's3:PutObject' permission would cause an 'AccessDenied' error, not 'ResourceNotFoundException'. Option C is incorrect because the AWS Region for S3 is determined by the bucket's location, and specifying a wrong region in Flink code would typically result in a different error (e.g., 'IllegalArgumentException' or cross-region access issues) but not 'ResourceNotFoundException'. Option D is incorrect because versioning is unrelated to bucket existence; disabling versioning does not cause a resource not found error.

Exam trap

Candidates often confuse permission errors with resource-not-found errors. A missing IAM permission typically results in 'AccessDenied', not 'ResourceNotFoundException'.

Practice this question →

325

MCQeasy

Refer to the exhibit. A data engineer runs this CLI command on an S3 bucket. The data is ingested from multiple sources. Which AWS service would be best to process these files in a single batch transformation?

A.AWS Lambda

B.Amazon Kinesis Data Analytics

C.Amazon Athena

D.AWS Glue

AnswerD

Glue can run batch ETL jobs on multiple files.

Why this answer

AWS Glue is designed for batch ETL processing, capable of handling multiple files of varying sizes in a single transformation job. Option A is wrong because Lambda has limitations on execution time (15 minutes) and memory (10 GB), making it unsuitable for large-scale batch transforms. Option B is wrong because Kinesis Data Analytics is for real-time stream processing, not batch.

Option C is wrong because Athena is an interactive query service for ad-hoc analysis of data in S3, not for batch transformations.

Practice this question →

326

Multi-Selecthard

A data engineering team is designing a batch processing workflow using AWS Glue. The job reads from an S3 bucket, transforms data, and writes to another S3 bucket. The job runs daily and processes new data incrementally. Which THREE features should they use to optimize performance and cost?

Select 3 answers

A.Convert all input data to Parquet format before processing.

B.Enable Glue job autoscaling.

C.Manually increase the number of DPUs for each run.

D.Use predicate pushdown and column pruning in the script.

E.Enable job bookmarks to process only new data.

AnswersB, D, E

Adjusts resources to workload.

Why this answer

Options B, D, and E are correct. Glue job autoscaling (B) dynamically adjusts resources, optimizing cost and performance. Predicate pushdown and column pruning (D) reduce data scanned, improving efficiency.

Job bookmarks (E) enable incremental processing, avoiding reprocessing of old data. Option A (converting to Parquet) is a good practice but not a Glue feature. Option C (manually increasing DPUs) is not optimal because it lacks flexibility and may waste resources.

Practice this question →

327

Multi-Selecteasy

A company is designing a data ingestion pipeline for real-time IoT sensor data. The data volume peaks at 10,000 messages per second. The pipeline must process messages in order per sensor and persist raw data to Amazon S3 for archival. Which TWO services should be used together to meet these requirements? (Choose TWO.)

Select 2 answers

A.Amazon Kinesis Data Firehose

B.Amazon Kinesis Data Streams

C.AWS Lambda

D.Amazon AppFlow

E.Amazon Simple Queue Service (Amazon SQS)

AnswersA, B

Delivers streaming data to S3.

Why this answer

Amazon Kinesis Data Streams (KDS) provides the ordered, real-time ingestion layer required for per-sensor message ordering, as it partitions data by a partition key (e.g., sensor ID) and guarantees order within a shard. Amazon Kinesis Data Firehose then reliably reads from the KDS stream and delivers the raw data to Amazon S3 for archival, handling buffering, compression, and automatic retries without requiring custom code.

Exam trap

The trap here is that candidates often choose AWS Lambda as a processing step without realizing it is not required for the core ingestion and archival pipeline, and they may overlook that Kinesis Data Streams is needed for ordering while Firehose handles the S3 delivery.

Practice this question →

328

MCQmedium

A data engineer is designing a pipeline to ingest change data capture (CDC) events from an Amazon RDS for PostgreSQL database into Amazon S3. The CDC events are captured using AWS DMS. The data must be available for querying within 5 minutes of the change. Which approach meets these requirements?

A.Export the database to S3 using pg_dump and then use AWS Glue to load into S3 in Parquet format.

B.Use AWS DMS to replicate data to Amazon Redshift, then unload to S3.

C.Use AWS DMS to replicate data directly to S3 in near real-time.

D.Use AWS DMS to replicate data to an SQS queue, then process with Lambda to write to S3.

AnswerC

DMS can write CDC to S3 with low latency.

Why this answer

AWS DMS supports continuous replication (change data capture) directly to Amazon S3 as a target endpoint. DMS can write CDC events to S3 in near real-time (typically seconds to minutes), meeting the 5-minute latency requirement without intermediate services. The data is stored in comma-separated value (CSV) or Parquet format, ready for querying via Athena or Glue.

Exam trap

The trap here is that candidates may overcomplicate the solution by introducing intermediate services (like Redshift or SQS) when DMS’s native S3 target endpoint already provides near-real-time CDC replication, and they may confuse pg_dump (a batch export tool) with a CDC mechanism.

How to eliminate wrong answers

Option A is wrong because pg_dump is a one-time logical backup tool, not a CDC solution; it cannot capture ongoing changes and would require full exports, exceeding the 5-minute latency window. Option B is wrong because replicating to Redshift then unloading to S3 adds unnecessary complexity and latency; Redshift is optimized for analytics, not as a CDC staging area, and the unload operation introduces additional delay. Option D is wrong because DMS does not natively support SQS as a target endpoint; DMS can write to S3, Kinesis, or other targets, but not directly to SQS, and the proposed architecture would require custom integration, breaking the near-real-time requirement.

Practice this question →

329

MCQeasy

A data engineer attached this IAM policy to a Lambda function used to transform data in S3. The function is unable to write output to the bucket. What is the most likely reason?

A.The resource ARN is missing the bucket-level ARN.

B.The policy does not allow the s3:DeleteObject action.

C.The policy does not allow the s3:ListBucket action on the bucket.

D.The policy does not allow the s3:PutObjectAcl action.

AnswerC

To write objects, the function needs ListBucket permission on the bucket itself.

Why this answer

The policy allows GetObject and PutObject on objects, but not the s3:ListBucket action required to check existence or list objects. The function likely needs ListBucket to write or verify.

Practice this question →

330

MCQeasy

An IAM policy includes the above resource ARN for CloudWatch Logs. A data engineer needs to allow a Lambda function to create log streams and put logs to the log group 'my-log-group'. However, the Lambda function is failing with access denied. What is the issue?

A.The region in the ARN does not match the Lambda function's region.

B.The Lambda function does not have an execution role.

C.The ARN does not include the log-stream portion.

D.The ARN is incorrectly formatted because of the wildcard.

AnswerC

The correct ARN for log streams should be 'arn:aws:logs:us-east-1:123456789012:log-group:my-log-group:log-stream:*'.

Why this answer

The ARN provided in the policy is for the log group itself (arn:aws:logs:region:account-id:log-group:my-log-group), but CloudWatch Logs requires separate permissions for the log-stream subresource. To allow a Lambda function to create log streams and put logs, the ARN must include the log-stream portion, typically with a wildcard like arn:aws:logs:region:account-id:log-group:my-log-group:log-stream:*. Without this, the IAM policy does not grant access to the log-stream actions (CreateLogStream, PutLogEvents), causing an access denied error even if the log group ARN is correct.

Exam trap

The trap here is that candidates assume granting access to the log group ARN implicitly covers log streams, but AWS IAM requires explicit resource-level permissions for each subresource in the ARN hierarchy.

How to eliminate wrong answers

Option A is wrong because the region in the ARN must match the Lambda function's region for the policy to apply, but the question does not indicate a region mismatch; the error is specifically about log-stream permissions. Option B is wrong because a Lambda function must have an execution role to access AWS resources, and the question implies a role exists (the policy is attached), but the role lacks the necessary log-stream permissions. Option D is wrong because the wildcard in the ARN is correctly placed for the log group name (my-log-group) and is not the cause of the failure; the issue is the missing log-stream component, not the wildcard format.

Practice this question →

331

MCQmedium

A data pipeline ingests CSV files from an S3 bucket into a Redshift table using the COPY command. Recently, files with inconsistent column delimiters (some use pipes, others use commas) have been arriving. The pipeline must handle both delimiters without manual intervention. What is the MOST efficient solution?

A.Configure the COPY command with a fixed delimiter (e.g., comma) and manually convert files with pipes before ingestion.

B.Create an AWS Lambda function triggered by S3 events that reads the first line of each file, detects the delimiter, and runs the COPY command with the appropriate DELIMITER option.

C.Use AWS Glue to crawl the S3 bucket and automatically detect the schema and delimiter before writing to Redshift.

D.Use Amazon Athena to query the files with the OpenCSVSerDe, which automatically detects delimiters, and then write the results to Redshift.

AnswerB

Lambda provides a lightweight, event-driven solution to dynamically detect and handle delimiters.

Why this answer

It provides an automated, event-driven solution that dynamically detects the delimiter of each incoming CSV file and executes the COPY command with the appropriate DELIMITER option. This approach eliminates manual intervention and leverages AWS Lambda's ability to process S3 events in near real-time, making it the most efficient for handling inconsistent delimiters.

Exam trap

The trap here is that candidates may assume AWS Glue or Athena are suitable for dynamic delimiter detection, but they lack the ability to directly and efficiently execute Redshift COPY commands with per-file delimiter customization without significant additional orchestration.

How to eliminate wrong answers

Option A is wrong because it requires manual conversion of files with pipes, which contradicts the requirement of no manual intervention and is not efficient for a data pipeline. Option C is wrong because AWS Glue crawlers are designed for schema discovery and cataloging, not for dynamically executing Redshift COPY commands with varying delimiters; they would require additional orchestration to handle the actual data ingestion. Option D is wrong because Amazon Athena with OpenCSVSerDe does not automatically detect delimiters; it requires the delimiter to be specified in the table DDL, and writing results to Redshift adds unnecessary complexity and latency compared to a direct COPY command.

Practice this question →

332

MCQmedium

Refer to the exhibit. A data engineer is troubleshooting an AWS Glue ETL job that fails with an access denied error when writing to S3. The IAM role attached to the Glue job has the policy shown. What is the most likely cause of the error?

A.The Glue job is missing the s3:ListBucket permission on the bucket

B.The Glue job is writing to an S3 bucket that is not included in the Resource ARN

C.The S3 bucket is encrypted with AWS KMS and the policy does not include kms:Decrypt permissions

D.The Glue job does not have permission to call glue:StartJobRun

AnswerB

The policy only grants access to my-data-bucket, not other buckets.

Why this answer

The IAM policy shown in the exhibit explicitly restricts the `s3:PutObject` action to a specific ARN pattern (e.g., `arn:aws:s3:::example-bucket/*`). If the Glue job attempts to write to a different S3 bucket—one not matching that ARN—the request will be denied with an access denied error. AWS Glue ETL jobs use the attached IAM role's permissions, and a mismatch between the target bucket and the resource ARN in the policy is the most common cause of such failures.

Exam trap

The trap here is that candidates often overlook the explicit resource ARN in the policy and assume the error is due to missing permissions like `s3:ListBucket` or KMS, when the real issue is a bucket name mismatch in the resource specification.

How to eliminate wrong answers

Option A is wrong because the `s3:ListBucket` permission is required for listing objects (e.g., `s3:ListBucket` on the bucket resource), not for writing objects; the error occurs during a write operation, and the policy already includes `s3:PutObject` on the bucket objects. Option C is wrong because the error message is specifically 'access denied when writing to S3', not a KMS-related error; if KMS encryption were the issue, the error would typically mention 'KMS' or 'decrypt' and the policy would need `kms:GenerateDataKey` or `kms:Decrypt`, not just `kms:Decrypt`. Option D is wrong because `glue:StartJobRun` is a permission to start a Glue job run, not a permission for writing to S3; the error occurs during the job execution (writing to S3), not during job initiation.

Practice this question →

333

MCQmedium

An IAM policy is attached to an AWS Glue job. The job needs to read from and write to S3 buckets, and also trigger other Glue jobs. The job is failing with an AccessDenied error when trying to write to a bucket named 'example-bucket'. What is the MOST likely cause?

A.The policy does not include s3:PutObject action.

B.The bucket name in the resource ARN does not match the actual bucket name.

C.The policy uses a resource ARN with a wildcard, which is not allowed.

D.The policy does not allow Glue actions.

AnswerB

The ARN uses 'example-bucket' but the actual bucket might have a different name.

Why this answer

The AccessDenied error when writing to 'example-bucket' most likely occurs because the resource ARN in the IAM policy specifies a different bucket name, causing the S3 service to deny the s3:PutObject action. IAM policies require exact ARN matches for resource-based permissions, and a mismatch between the ARN bucket name and the actual bucket name will result in an implicit deny, even if the action is allowed.

Exam trap

The trap here is that candidates assume the error is due to a missing action (like s3:PutObject) rather than a resource ARN mismatch, which is a subtle but common IAM misconfiguration that AWS explicitly tests in the DEA-C01 exam.

How to eliminate wrong answers

Option A is wrong because the error is specifically about writing (s3:PutObject), and if the policy lacked that action, the error would still occur, but the question asks for the 'most likely' cause given the policy is attached; the mismatch in the resource ARN is a more precise and common cause. Option C is wrong because wildcards in resource ARNs are allowed in IAM policies (e.g., 'arn:aws:s3:::example-bucket/*'), and using a wildcard would not cause an AccessDenied error if the bucket name matches. Option D is wrong because the job is failing on an S3 write, not on triggering other Glue jobs; Glue actions (like 'glue:StartJobRun') are separate and would produce a different error if missing.

Practice this question →

334

MCQhard

A company runs a nightly AWS Glue ETL job that reads from a JDBC source (PostgreSQL) and writes to S3 in Parquet format. The job takes over 6 hours, but the SLA requires completion within 4 hours. The source table has 500 million rows and is updated frequently. Which approach will most reliably reduce job duration?

A.Enable job bookmark and schedule the job to run more frequently.

B.Use multiple JDBC connections in parallel by setting 'hashexpression' and 'hashfield'.

C.Partition the source table by year and use pushdown predicates in the Glue job.

D.Increase the number of DPUs for the Glue job to 100.

AnswerC

This reduces the data scanned by filtering on partition columns.

Why this answer

Partitioning the source table by year and using pushdown predicates allows AWS Glue to read only the relevant partitions from PostgreSQL, drastically reducing the data scanned and transferred. This directly addresses the 500 million row volume by minimizing the JDBC read workload, which is the primary bottleneck. Option B, using multiple JDBC connections in parallel via hash partitioning, can improve performance but is less reliable: it significantly increases load on the source database, may hit connection limits, and its effectiveness depends on the JDBC driver's support for hashfield/hashexpression.

Moreover, it still reads all rows, whereas pushdown predicates reduce the data volume at the source, making option C more reliable for meeting the 4-hour SLA.

Exam trap

The trap here is that candidates often assume increasing DPUs (Option D) or adding parallelism (Option B) will linearly speed up JDBC reads, but they fail to recognize that the bottleneck is the source database's I/O and network throughput, not Glue's compute capacity, and that predicate pushdown is the only option that reduces the data volume at the source.

How to eliminate wrong answers

Option A is wrong because job bookmarks track previously processed data to avoid reprocessing, but they do not reduce the initial full load or the per-run data volume; scheduling more frequently would only compound the problem by running incomplete jobs. Option B is wrong because 'hashexpression' and 'hashfield' are not valid JDBC parallelism parameters in AWS Glue; the correct approach for parallel JDBC reads is to set 'hashfield' and 'hashpartitions' (not 'hashexpression'), and even then, parallelism alone cannot overcome the I/O bottleneck of scanning 500 million rows without filtering. Option D is wrong because increasing DPUs to 100 may improve compute parallelism for transformations, but the bottleneck is the JDBC read from PostgreSQL, which is constrained by the source database's network and query capacity, not by Glue's compute resources; excessive DPUs can also cause throttling or connection limits.

Practice this question →

335

Multi-Selectmedium

A data engineer is designing a data ingestion pipeline to load clickstream data from an Amazon S3 bucket into an Amazon Redshift cluster. The data arrives in 5-minute batches. Which TWO actions should the engineer take to ensure data consistency and avoid duplicates? (Select TWO.)

Select 2 answers

A.Define a SORTKEY on the target table to improve deduplication.

B.Disable workload management (WLM) to maximize resources.

C.Use the STL_LOAD_ERRORS system table to monitor and resolve load errors.

D.Load data into a single slice to maintain order.

E.Use a staging table and perform a MERGE operation to avoid duplicates.

AnswersC, E

Monitoring load errors helps catch and fix issues that could cause duplicates or missing data.

Why this answer

Options C and E are correct. Using the STL_LOAD_ERRORS system table (option C) allows the engineer to monitor and resolve load errors, ensuring data consistency. Using a staging table with a MERGE operation (option E) helps avoid duplicates by performing an upsert.

Option A (defining a SORTKEY) improves query performance but does not affect deduplication. Option B (disabling WLM) would reduce resource management and is not recommended. Option D (loading into a single slice) reduces performance and does not maintain order for consistency.

Practice this question →

336

MCQhard

A company uses Amazon Kinesis Data Streams with enhanced fan-out consumers. The stream has 5 shards. Each consumer reads from all shards. The total incoming data rate is 25 MB/s. What is the maximum read throughput per consumer if enhanced fan-out is enabled?

A.2 MB/s

B.10 MB/s

C.1 MB/s

D.5 MB/s

AnswerB

Each shard provides 2 MB/s read capacity with enhanced fan-out; 5 shards = 10 MB/s.

Why this answer

With enhanced fan-out, each consumer gets a dedicated 2 MB/s read throughput per shard. Since the stream has 5 shards, the maximum read throughput per consumer is 5 shards × 2 MB/s = 10 MB/s. This is because enhanced fan-out eliminates the 2 MB/s total read limit per shard shared among all consumers, providing each consumer with its own 2 MB/s pipe per shard.

Exam trap

The trap here is that candidates confuse the standard consumer throughput limit (2 MB/s total per shard shared among all consumers) with the enhanced fan-out limit (2 MB/s per shard per consumer), leading them to pick 2 MB/s or 5 MB/s instead of correctly multiplying by the number of shards.

How to eliminate wrong answers

Option A is wrong because 2 MB/s is the per-shard read throughput with enhanced fan-out, not the total for 5 shards. Option C is wrong because 1 MB/s is the per-shard read throughput for standard (non-enhanced) consumers, not for enhanced fan-out. Option D is wrong because 5 MB/s would be the total read throughput if each shard provided only 1 MB/s (standard) across 5 shards, but enhanced fan-out provides 2 MB/s per shard per consumer.

Practice this question →

337

MCQmedium

A company is ingesting log files from multiple EC2 instances into Amazon S3 using the CloudWatch agent. The logs are delivered to a CloudWatch Logs group, and a subscription filter sends them to a Lambda function for transformation, then to Firehose. The Firehose stream is configured with a buffer interval of 60 seconds and buffer size of 5 MB. The logs are critical and must be available in S3 within 5 minutes. What is the most cost-effective way to reduce the delivery latency?

A.Replace Firehose with Amazon Kinesis Data Streams

B.Increase the buffer size to 10 MB

C.Increase the buffer interval to 120 seconds

D.Decrease the buffer interval to 10 seconds

AnswerD

Lower buffer interval reduces delivery latency.

Why this answer

Decreasing the Firehose buffer interval to 10 seconds directly reduces the maximum time data waits in the buffer before being delivered to S3, ensuring logs reach S3 within the required 5-minute window. Since the current 60-second buffer interval is the primary contributor to latency, lowering it to 10 seconds minimizes delivery delay without incurring additional costs, as Firehose charges are based on data volume, not buffer frequency.

Exam trap

The trap here is that candidates may think increasing buffer size or interval improves throughput, but the question asks for reduced latency, so decreasing the buffer interval is the direct and cost-effective solution.

How to eliminate wrong answers

Option A is wrong because replacing Firehose with Kinesis Data Streams would require additional components (e.g., a consumer to write to S3) and increase cost and complexity, not reduce latency cost-effectively. Option B is wrong because increasing the buffer size to 10 MB would allow more data to accumulate before delivery, potentially increasing latency, not reducing it. Option C is wrong because increasing the buffer interval to 120 seconds would double the maximum buffering time, worsening delivery latency.

Practice this question →

338

Multi-Selectmedium

A company uses AWS Glue to perform ETL on data stored in Amazon S3. The Glue job reads CSV files, converts them to Parquet, and partitions by date. The job runs daily and processes about 500 GB of data. The team wants to optimize costs and performance. Which three actions should the team take? (Select THREE.)

Select 3 answers

A.Increase the Spark shuffle partitions to 500.

B.Use column pruning to read only necessary columns in the Glue script.

C.Use G.1X or G.2X worker types for better performance.

D.Increase the number of DPUs for the job.

E.Write the output as JSON instead of Parquet to avoid compression overhead.

AnswersB, C, D

Reduces data scanned and improves performance.

Why this answer

Column pruning in AWS Glue scripts reduces the amount of data read from Amazon S3 by specifying only the columns needed for the ETL transformation. This minimizes I/O and network overhead, directly lowering costs and improving job performance, especially when processing large CSV files.

Exam trap

The trap here is that candidates often confuse increasing DPUs or shuffle partitions as a universal performance fix, but AWS Glue's cost optimization relies on reducing data processed (column pruning) and choosing appropriate worker types for the workload, not simply scaling resources.

Practice this question →

339

MCQeasy

A company needs to transfer 10 TB of historical data from an on-premises HDFS cluster to Amazon S3. The data is stored on a single 20 TB disk. The network link to AWS has a bandwidth of 1 Gbps. The transfer must be completed within 2 days. Which solution meets these requirements?

A.Use AWS Snowball Edge to transfer the data physically.

B.Use Amazon Kinesis Data Streams to stream data to S3.

C.Use AWS DMS to migrate data from HDFS to S3.

D.Use AWS CLI to copy data directly to S3 over the network.

AnswerA

Snowball Edge provides fast, reliable transfer for large datasets.

Why this answer

AWS Snowball Edge is the correct solution because it can physically transfer 10 TB of data from a single 20 TB disk within the 2-day window, bypassing the network bandwidth limitation. With a 1 Gbps link, the theoretical maximum transfer of 10 TB would take approximately 22.2 hours under ideal conditions, but real-world overhead, latency, and potential congestion make it unreliable to complete within 2 days. Snowball Edge provides a petabyte-scale physical transport solution that avoids network constraints entirely.

Exam trap

The trap here is that candidates may calculate the theoretical transfer time (10 TB / 1 Gbps ≈ 22.2 hours) and assume it fits within 2 days, ignoring real-world network inefficiencies, disk I/O limits, and the fact that AWS CLI transfers over a single TCP connection cannot saturate a 1 Gbps link without tuning (e.g., multipart uploads, parallel connections).

How to eliminate wrong answers

Option B is wrong because Amazon Kinesis Data Streams is designed for real-time streaming of small records (up to 1 MB per record) and cannot handle bulk transfer of 10 TB of historical data from an HDFS cluster; it lacks the throughput and batch processing capability for such large volumes. Option C is wrong because AWS DMS (Database Migration Service) is intended for migrating databases (e.g., Oracle, MySQL) and does not support HDFS as a source endpoint; it cannot read data from HDFS or transfer it to S3. Option D is wrong because using AWS CLI to copy data directly over a 1 Gbps network link would take at least 22.2 hours under perfect conditions, but real-world factors like TCP overhead, retransmissions, and network congestion make it highly unlikely to complete within 2 days, especially with a single 20 TB disk that may have I/O bottlenecks.

Practice this question →

340

MCQhard

A company runs a data ingestion pipeline that uses AWS Glue to read 500 GB of JSON files from an S3 bucket (s3://raw-data/) every hour. The Glue ETL job transforms the data and writes Parquet files to another S3 bucket (s3://processed-data/). The job is triggered by a time-based CloudWatch Events rule. Recently, the job has started taking over 2 hours to complete, causing delays in downstream processes. The data volume has been consistent, and no changes have been made to the job code or infrastructure. The S3 bucket 's3://raw-data/' receives new files continuously, but the Glue job reads all files in the bucket each run (no incremental processing). The engineer suspects that the job is reprocessing old data. Which action should the engineer take FIRST to reduce the job duration?

A.Enable Glue job bookmarking and configure the job to process only new data.

B.Increase the parallelism of the Spark job by repartitioning the data.

C.Add partition pruning by modifying the S3 path to include date-based partitions.

D.Increase the number of DPUs for the Glue job to 100.

AnswerA

Bookmarking tracks processed files, so subsequent runs only process new files, drastically reducing runtime.

Why this answer

Enabling job bookmarking in Glue allows the job to process only new files since the last run, dramatically reducing processing time. Option D (increasing DPUs) would help with resource constraints but does not address the root cause of reprocessing old data. Option B (increasing parallelism) may help but not as much as eliminating reprocessing.

Option C (using partition pruning) assumes the data is partitioned, but the stem says all files are read; partitioning might not help if files are not organized by time. The most impactful first step is to enable bookmarking.

Practice this question →

341

MCQmedium

A data engineer is designing a data ingestion pipeline to load data from an on-premises Oracle database to Amazon S3. The pipeline should capture changes in near real-time (within minutes) and minimize impact on the source database. The source table has a 'last_modified' timestamp column. Which service combination would meet these requirements?

A.AWS DMS with a replication task in CDC mode, writing to S3 in Parquet format.

B.Amazon Kinesis Data Firehose with a Lambda function that queries Oracle.

C.AWS Data Pipeline with a periodic SQL query activity to copy full table snapshots.

D.AWS Glue with a JDBC connection to Oracle, running a crawler every 5 minutes.

AnswerA

DMS CDC captures minimal changes and writes to S3 with low latency.

Why this answer

AWS DMS with a replication task in CDC (Change Data Capture) mode is the correct choice because it continuously reads the Oracle redo logs to capture near real-time changes (within seconds to minutes) with minimal impact on the source database. It can directly write to S3 in Parquet format, meeting the requirement for low-latency ingestion without full table scans.

Exam trap

The trap here is that candidates assume a 'last_modified' timestamp column enables easy CDC via polling (options B, D), but the question tests that true near real-time CDC with minimal source impact requires reading database redo logs, not querying the table, and that services like Kinesis or Glue cannot perform log-based CDC without additional custom logic.

How to eliminate wrong answers

Option B is wrong because Kinesis Data Firehose with a Lambda function that queries Oracle would require periodic polling of the source table, which either misses changes between polls or causes high load on the database if polled too frequently, and it does not natively support CDC from Oracle redo logs. Option C is wrong because AWS Data Pipeline with a periodic SQL query activity to copy full table snapshots performs full table scans, which impacts the source database and cannot capture changes in near real-time (only at scheduled intervals). Option D is wrong because AWS Glue with a JDBC connection running a crawler every 5 minutes performs full table scans or incremental queries based on the 'last_modified' column, but this still causes repeated query load on Oracle and cannot achieve true near real-time CDC without reading redo logs.

Practice this question →

342

MCQhard

A data engineer is designing a data ingestion pipeline for clickstream data from a mobile app. The data volume varies, with occasional spikes up to 10 MB/s. The pipeline must persist the raw data in Amazon S3 and make it available for near-real-time analytics via Amazon Athena. Which combination of services minimizes cost and operational overhead?

A.Amazon Kinesis Data Streams with Amazon Kinesis Data Analytics, then Amazon S3

B.Amazon SQS with an Auto Scaling group of EC2 instances writing to Amazon S3

C.Amazon Kinesis Data Streams with AWS Lambda for transformation, then Amazon S3

D.Amazon Kinesis Data Firehose with direct delivery to Amazon S3, then Amazon Athena

AnswerD

Firehose is fully managed, scales automatically, and delivers to S3.

Why this answer

Amazon Kinesis Data Firehose is the most cost-effective and low-overhead solution for ingesting variable-volume clickstream data (up to 10 MB/s) into Amazon S3 because it is a fully managed service that automatically scales, buffers, and compresses data before delivery. It integrates directly with S3 without requiring custom code or infrastructure management, and the data is immediately queryable by Amazon Athena with no additional transformation steps.

Exam trap

The trap here is that candidates often choose Amazon Kinesis Data Streams with Lambda (Option C) because they think it provides more control, but they overlook Lambda's concurrency limits and the operational burden of managing stream shards, making Firehose the simpler and cheaper choice for raw data ingestion to S3.

How to eliminate wrong answers

Option A is wrong because Amazon Kinesis Data Analytics adds unnecessary cost and complexity for a pipeline that only needs to persist raw data to S3, and it requires a separate stream consumer. Option B is wrong because Amazon SQS with Auto Scaling EC2 instances introduces significant operational overhead for managing instance scaling, SQS polling, and potential data loss or duplication, and it is not optimized for near-real-time streaming at 10 MB/s. Option C is wrong because AWS Lambda has a maximum invocation duration of 15 minutes and a concurrency limit that can cause throttling during spikes, making it unsuitable for sustained 10 MB/s throughput without complex sharding and retry logic.

Practice this question →

343

MCQeasy

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3 on a nightly basis. The data volume is approximately 10 GB per night. The database is accessible over the internet. Which AWS service is MOST appropriate for this task?

A.AWS Glue ETL job with a JDBC connection

B.AWS DataSync

C.AWS Transfer Family

D.Amazon Kinesis Data Streams

AnswerA

AWS Glue ETL with JDBC can connect to Oracle and export data to S3, but it is less efficient than DMS for this use case. DMS handles schema extraction, data type conversion, and checkpoint resume automatically.

Why this answer

The most appropriate AWS service among the given options for nightly batch ingestion from an on-premises Oracle database into Amazon S3 is AWS Glue ETL job with a JDBC connection (A). AWS Glue can connect to the Oracle database via JDBC, extract data, and load it into S3 in a scheduled batch manner. While AWS Database Migration Service (DMS) is purpose-built for database migrations and would be the ideal service, it is not listed as an option.

AWS DataSync (B) is used for file and object storage transfers, not for database connections. AWS Transfer Family (C) provides managed file transfer protocols and cannot directly query databases. Amazon Kinesis Data Streams (D) is designed for real-time streaming data ingestion, not nightly batch loads.

Therefore, AWS Glue is the correct choice.

Exam trap

Candidates may assume that AWS Glue is the only option for batch ETL from databases, but AWS DMS is purpose-built for database migrations and is often the recommended service for migrating or replicating data from on-premises databases to AWS, including direct to S3.

Practice this question →

344

MCQeasy

A data engineer needs to ingest streaming data from thousands of IoT devices and immediately process each record with minimal latency. Which AWS service should be used as the ingestion point?

A.AWS Lambda

B.Amazon S3

C.Amazon Kinesis Data Streams

D.AWS Glue

AnswerC

Kinesis Data Streams ingests streaming data with low latency and can be consumed by multiple applications.

Why this answer

Amazon Kinesis Data Streams is designed for real-time streaming data ingestion with low latency. AWS Glue is for batch ETL, S3 is object storage, and Lambda is compute but not an ingestion endpoint itself.

Practice this question →

345

MCQhard

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is transformed using an AWS Lambda function. Recently, the transformation errors have increased due to Lambda timeouts. The data engineer needs to diagnose and resolve the issue without losing data. What should the engineer do?

A.Increase the Lambda function timeout and ensure that failed records are sent to a backup S3 bucket

B.Enable Amazon CloudWatch Logs for the Lambda function to capture errors and store failed records in CloudWatch

C.Configure the Lambda function to write failed records to an Amazon SQS queue for later reprocessing

D.Modify the Lambda function to store failed records in Amazon S3 before processing

AnswerA

Increasing timeout reduces failures, and configuring a backup bucket prevents data loss.

Why this answer

Increasing the Lambda function timeout directly addresses the root cause of transformation errors (timeouts), and configuring a backup S3 bucket for failed records ensures no data loss. Kinesis Data Firehose can be configured to send failed records to a separate S3 bucket as a dead-letter queue, which preserves the data for later reprocessing while the primary transformation pipeline is fixed.

Exam trap

The trap here is that candidates may confuse logging (CloudWatch Logs) with actual data preservation, or assume that SQS is a native Firehose failure destination, when in fact Firehose only supports S3 or Redshift as backup destinations for failed records.

How to eliminate wrong answers

Option B is wrong because enabling CloudWatch Logs captures error logs but does not store the actual failed records; it only provides visibility into errors without preventing data loss. Option C is wrong because Kinesis Data Firehose does not natively support sending failed records to an SQS queue; the Lambda function would need custom code to write to SQS, and this does not address the timeout issue. Option D is wrong because storing failed records in S3 before processing would require modifying the Lambda function to write to S3 first, which adds complexity and does not resolve the timeout; the records are already in the Firehose stream and need to be processed or redirected after failure.

Practice this question →

346

MCQeasy

The exhibit shows the output of describing an Amazon Kinesis Data Stream. A producer is sending records but the consumer is not receiving all records. What is the most likely cause?

A.The stream has only one shard, causing write throttling

B.The stream is in ACTIVE status, which prevents reading

C.The retention period is too short

D.The hash key range is too wide

AnswerA

With one shard, write throughput is limited; exceeding it causes throttling and missed records.

Why this answer

The stream has only one shard, which provides a maximum throughput of 1 MB/s or 1000 records/s for writes. If the producer exceeds this, records will be throttled. The retention period is 24 hours, which is fine.

The stream status is ACTIVE. There is no indication of a faulty shard. The consumer might be slow, but the question asks for cause of not receiving all records; throttling due to insufficient shards is a common issue.

Practice this question →

347

MCQeasy

A company needs to ingest data from a MySQL database into Amazon S3 in near real-time. The database is running on EC2. The data engineer wants to minimize the impact on the source database. Which service should be used?

A.AWS Database Migration Service (DMS) with ongoing replication

B.AWS Glue ETL job with a JDBC connection

C.Amazon RDS for MySQL with read replica

D.AWS Schema Conversion Tool (SCT)

AnswerA

DMS CDC uses binary logs to capture changes with minimal overhead.

Why this answer

AWS DMS with ongoing replication (change data capture) is the correct choice because it can continuously replicate changes from a MySQL source database to Amazon S3 with minimal performance impact. DMS uses a transactional log-based approach (MySQL binlog) to capture changes as they occur, avoiding heavy SELECT queries on the source. This enables near real-time ingestion without adding significant load to the production database.

Exam trap

The trap here is that candidates often confuse AWS Glue's batch JDBC capabilities with streaming ingestion, or assume that a read replica can directly feed data into S3 without an intermediary service like DMS or Kinesis.

How to eliminate wrong answers

Option B is wrong because AWS Glue ETL jobs with JDBC connections run batch queries that pull full table snapshots or large result sets, which can cause significant performance degradation on the source MySQL database and cannot achieve near real-time latency. Option C is wrong because Amazon RDS for MySQL with a read replica is a database migration or read scaling solution, not a data ingestion service to S3; it does not natively stream data to S3 without additional tooling. Option D is wrong because AWS Schema Conversion Tool (SCT) is designed for converting database schemas between different database engines (e.g., Oracle to Aurora), not for ingesting data into S3.

Practice this question →

348

MCQhard

A data pipeline uses Amazon Kinesis Data Firehose to ingest log data from web servers and deliver it to Amazon S3. The data is then transformed by an AWS Glue job before being loaded into Amazon Redshift. The pipeline must handle a sudden spike in log volume without data loss. Which configuration change is MOST appropriate?

A.Increase the AWS Glue job timeout and allocate more DPUs.

B.Configure Kinesis Data Firehose to back up all data to S3 in case of delivery failures.

C.Increase the number of nodes in the Redshift cluster to handle higher load.

D.Increase the S3 bucket size limit and enable versioning.

AnswerB

S3 backup for failed records ensures no data loss.

Why this answer

Kinesis Data Firehose can be configured to back up all data to Amazon S3 in case of delivery failures, ensuring no data loss during spikes. This feature writes incoming data to a separate S3 bucket as a safety net when the primary destination (e.g., Redshift via Glue) is unavailable or overwhelmed, directly addressing the requirement to handle sudden volume spikes without data loss.

Exam trap

The trap here is that candidates confuse downstream scaling (Redshift or Glue) with ingestion-layer fault tolerance, overlooking that Kinesis Data Firehose’s S3 backup directly addresses data loss at the point of delivery failure.

How to eliminate wrong answers

Option A is wrong because increasing AWS Glue job timeout and DPUs improves processing capacity but does not prevent data loss during ingestion spikes; data can still be lost if Firehose delivery fails before Glue runs. Option C is wrong because scaling Redshift nodes handles downstream load but does not protect against data loss at the ingestion layer; data may be dropped before it reaches Redshift. Option D is wrong because S3 bucket size limit and versioning are irrelevant to data loss prevention during spikes; S3 has no practical size limit, and versioning protects against accidental deletion, not ingestion failures.

Practice this question →

349

Multi-Selectmedium

A data engineer is designing a data ingestion pipeline that uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The engineer wants to ensure that the data is organized in a directory structure by year, month, day, and hour. Which TWO configurations should the engineer set on the Firehose delivery stream? (Choose TWO.)

Select 2 answers

A.Enable dynamic partitioning

B.Set a custom prefix with '!{timestamp:yyyy}/!{timestamp:MM}/!{timestamp:dd}/!{timestamp:HH}/'

C.Use an AWS Lambda function to write to S3 with the desired prefix

D.Enable format conversion to Parquet

E.Configure an S3 bucket with versioning enabled

AnswersA, B

Dynamic partitioning allows Firehose to partition data based on keys.

Why this answer

Correct options: A and B. Option A is correct because enabling dynamic partitioning allows Firehose to use partition keys from the data, such as timestamps, to automatically create folder structures in S3. Option B is correct because setting a custom prefix with the expression '!{timestamp:yyyy}/!{timestamp:MM}/!{timestamp:dd}/!{timestamp:HH}/' defines the directory hierarchy by year, month, day, and hour.

Option C is incorrect because using a Lambda function to write to S3 with a desired prefix is not necessary; Firehose handles the prefix natively. Option D is incorrect because format conversion to Parquet is unrelated to directory organization. Option E is incorrect because S3 bucket versioning does not affect directory structure.

Practice this question →

350

MCQhard

Refer to the exhibit. A data engineer runs the describe-stream command and sees this output. The application is writing records to the stream but is experiencing high write latency. The average record size is 50 KB, and the write rate is 1500 records per second. What is the MOST likely cause of the latency?

A.The application is running in a different AWS region.

B.The application is exceeding the DynamoDB provisioned throughput.

C.The Kinesis stream is throttling the application because of a hot shard.

D.The stream does not have enough shards to handle the write throughput.

AnswerD

2 shards provide 2 MB/s write capacity; the application requires 75 MB/s.

Why this answer

The stream does not have enough shards to handle the write throughput. Each Kinesis shard can ingest up to 1 MB/s or 1000 records per second (for up to 1 KB record size). With 2 shards, the total write capacity is 2 MB/s or 2000 records/s.

However, the average record size is 50 KB, and the write rate is 1500 records/s. The throughput in MB/s is 1500 * 50 KB = 75 MB/s, which far exceeds the 2 MB/s total capacity. Therefore, the stream is under-provisioned, causing high write latency.

Option A is incorrect because the region of the application does not directly cause high latency; cross-region latency might be a factor but not the most likely given the throughput metrics. Option B is incorrect because DynamoDB provisioned throughput is not relevant to Kinesis streams. Option C is incorrect because throttling due to a hot shard would occur if data is unevenly distributed, but the issue here is overall insufficient shard count given the high throughput demand.

Practice this question →

351

MCQmedium

A data engineer is designing a data ingestion pipeline for real-time clickstream data using Amazon Kinesis Data Streams. The data must be transformed using AWS Lambda and then stored in Amazon S3 in Parquet format. Which Kinesis client library configuration should be used to minimize the number of Lambda invocations while ensuring data is processed within 60 seconds?

A.Set batch size to 100 records and disable batch window

B.Set batch size to 10000 records and batch window to 60 seconds

C.Set batch size to 100 records and batch window to 0 seconds

D.Set batch size to 10000 records and batch window to 5 seconds

AnswerB

Set batch size to 10000 records and batch window to 60 seconds maximizes records per invocation while respecting the 60-second requirement, minimizing invocations.

Why this answer

A batch size of 10,000 records combined with a batch window of 60 seconds maximizes the number of records per Lambda invocation while respecting the 60-second processing requirement, thereby minimizing the total number of invocations. Option A (batch size 100, no batch window) leads to many small invocations. Option C (batch window 0 seconds) triggers immediate processing, increasing invocation frequency.

Option D (batch window 5 seconds) forces more frequent invocations than necessary, even with a large batch size.

Practice this question →

352

MCQhard

A data engineer is designing a streaming pipeline that ingests IoT sensor data from 10,000 devices. Each device sends a 1 KB message every second. The data must be processed in near real-time and stored in S3 for analytics. Which combination of services provides the most cost-effective solution?

A.AWS Data Pipeline with periodic S3 copy.

B.Amazon Kinesis Data Streams with Kinesis Data Firehose delivery to S3.

C.Amazon MSK (Managed Streaming for Kafka) with Kafka Connect S3 sink.

D.Amazon SQS FIFO queue with Lambda consumers writing to S3.

AnswerB

Handles high throughput, Firehose batches to S3.

Why this answer

B is correct because Kinesis Data Streams ingests high-throughput IoT data (10,000 messages/sec at 1 KB each) with low latency, and Kinesis Data Firehose automatically batches and compresses data before delivering it to S3, eliminating the need for custom code or manual scaling. This combination provides the most cost-effective near-real-time solution by leveraging Firehose's built-in buffering and compression to minimize S3 storage costs and reduce the number of PUT requests.

Exam trap

The trap here is that candidates often choose MSK (Option C) thinking it is more scalable or flexible, but they overlook the higher operational cost and complexity for a simple S3 sink use case, where Kinesis Data Firehose's fully managed batching, compression, and direct S3 integration is more cost-effective and simpler to maintain.

How to eliminate wrong answers

Option A is wrong because AWS Data Pipeline is a batch-oriented orchestration service, not designed for near-real-time streaming ingestion; it would introduce significant latency and require periodic polling, failing the near-real-time requirement. Option C is wrong because Amazon MSK (Managed Streaming for Apache Kafka) introduces unnecessary operational overhead and cost for this use case, as it requires managing Kafka clusters, brokers, and the Kafka Connect S3 sink, while Kinesis Data Firehose provides a simpler, fully managed, and more cost-effective direct S3 delivery. Option D is wrong because SQS FIFO queues are designed for exactly-once processing and low throughput (300 transactions per second by default), making them unsuitable for 10,000 messages per second; additionally, Lambda consumers would incur high costs due to the large number of invocations and lack built-in batching and compression for S3 writes.

Practice this question →

353

MCQhard

A team is designing a data ingestion pipeline to load JSON files from an Amazon S3 bucket into Amazon Redshift. The files arrive every 5 minutes, and each file is between 10 MB and 50 MB. The team wants to minimize the time between file arrival and data availability in Redshift. Which approach should the team use?

A.Schedule an AWS Glue job to run every 5 minutes to load the data.

B.Use S3 Event Notifications to trigger an AWS Lambda function that runs the COPY command to load data into Redshift.

C.Use Amazon Redshift Spectrum to query the data directly from S3 without loading.

D.Configure Amazon Kinesis Data Firehose to stream data from S3 to Redshift.

AnswerB

Lambda responds quickly to S3 events and runs COPY for efficient bulk loading.

Why this answer

S3 Event Notifications can trigger an AWS Lambda function that executes the COPY command, loading data into Redshift with minimal latency. This approach avoids the overhead of scheduling or batching, directly responding to each file arrival to meet the 5-minute frequency and file size requirements.

Exam trap

The trap here is that candidates may confuse Redshift Spectrum (which queries external data without loading) with the requirement to have data available in Redshift tables, or they may overestimate the suitability of scheduled Glue jobs for low-latency, frequent ingestion.

How to eliminate wrong answers

Option A is wrong because scheduling an AWS Glue job every 5 minutes introduces unnecessary startup overhead and may not achieve the lowest latency, as Glue jobs have a cold start time and are better suited for larger batch processing. Option C is wrong because Redshift Spectrum queries data directly from S3 without loading it into Redshift tables, which does not make the data available in Redshift for fast, indexed queries and can incur higher latency for repeated access. Option D is wrong because Amazon Kinesis Data Firehose cannot directly ingest data from S3; it is designed to stream data into S3 or Redshift from producers like Kinesis Data Streams, not to read existing S3 files.

Practice this question →

354

Multi-Selecthard

A data engineering team is designing a near-real-time data ingestion pipeline for IoT sensor data. The data must be processed and stored in Amazon S3, with transformations applied before storage. The team needs to handle potential duplicates and ensure exactly-once processing semantics. Which TWO AWS services should be used together? (Choose TWO.)

Select 2 answers

A.Amazon Kinesis Data Firehose

B.Amazon Simple Queue Service (SQS)

C.Amazon Kinesis Data Analytics for Apache Flink

D.Amazon Kinesis Data Streams

E.AWS Database Migration Service (DMS)

AnswersC, D

Flink can provide exactly-once semantics with checkpointing.

Why this answer

For near-real-time IoT sensor data ingestion with exactly-once processing, Amazon Kinesis Data Streams (option D) provides ordered, durable, and replayable data ingestion. Amazon Kinesis Data Analytics for Apache Flink (option C) consumes from the stream and supports exactly-once semantics through checkpointing and idempotent sinks, enabling duplicate handling. Amazon Kinesis Data Firehose (option A) offers at-least-once delivery only.

Amazon SQS (option B) standard queues do not guarantee exactly-once, and FIFO queues cannot handle high-throughput IoT data. AWS DMS (option E) is for database replication, not streaming ingestion.

Practice this question →

355

MCQmedium

A data engineer needs to ingest streaming data from thousands of IoT devices into AWS for near-real-time analytics. The data volume varies significantly and can spike unpredictably. The engineer wants to minimize operational overhead and ensure that data is durably stored as soon as it arrives. Which AWS service combination should the engineer use?

A.Use Amazon S3 Transfer Acceleration with S3 Event Notifications to trigger AWS Lambda for processing.

B.Use Amazon Kinesis Data Firehose to ingest data into Amazon S3 and use AWS Lambda to transform data during delivery.

C.Use Amazon Simple Queue Service (SQS) to buffer the streaming data and configure an Auto Scaling group of EC2 instances to poll and process the data.

D.Use Amazon Kinesis Data Streams to ingest the data and AWS Lambda to process records in real-time with automatic scaling.

AnswerD

Kinesis Data Streams provides durable, scalable, low-latency ingestion; Lambda can process each shard in parallel and scales automatically.

Why this answer

Amazon Kinesis Data Streams (KDS) is designed for ingesting large volumes of streaming data with automatic scaling (via shard splitting/merging) and provides durable storage (default 24-hour retention, extendable to 365 days) as soon as records are received. AWS Lambda can be subscribed to the stream to process records in near-real-time, scaling automatically based on the number of shards, which minimizes operational overhead and handles unpredictable spikes without manual intervention.

Exam trap

The trap here is that candidates often confuse Kinesis Data Firehose (which batches and delivers to destinations like S3) with Kinesis Data Streams (which provides real-time, durable storage and processing), leading them to choose Option B despite its lack of true near-real-time ingestion and automatic scaling for spikes.

How to eliminate wrong answers

Option A is wrong because S3 Transfer Acceleration is for speeding up uploads over long distances, not for streaming ingestion, and S3 Event Notifications have at-least-once delivery with potential latency, not providing immediate durable storage upon arrival. Option B is wrong because Kinesis Data Firehose batches data before delivering to S3, introducing latency and not providing true near-real-time ingestion; it also does not offer the same level of automatic scaling for unpredictable spikes as KDS. Option C is wrong because SQS is a message queue with no native streaming shard model, and using an Auto Scaling group of EC2 instances adds significant operational overhead (managing instances, scaling policies, polling logic) and does not guarantee data durability as soon as it arrives (messages are stored but processing is decoupled).

Practice this question →

356

MCQmedium

A data engineer needs to ingest data from an Amazon RDS for MySQL database into Amazon S3 on a daily basis. The data volume is about 50 GB per day. The engineer wants to minimize the impact on the source database. Which AWS service should be used?

A.AWS Glue with a JDBC connection

B.Amazon Athena Federated Query

C.AWS Database Migration Service (DMS)

D.AWS DataSync

AnswerC

DMS is optimized for database migrations with minimal impact.

Why this answer

AWS DMS can perform full load and ongoing replication with minimal impact on the source database. Option A is wrong because AWS Glue with a JDBC connection can impact the source due to high query load. Option B is wrong because Amazon Athena Federated Query reads data directly from RDS, which can cause performance issues.

Option D is wrong because AWS DataSync is designed for file storage, not databases.

Practice this question →

357

Multi-Selectmedium

A company is ingesting IoT sensor data from thousands of devices using Amazon Kinesis Data Streams. The data is consumed by a Lambda function that transforms and writes to Amazon S3. The company notices that occasionally records are dropped. The data engineer needs to identify the cause and prevent data loss. Which TWO actions should the data engineer take? (Choose TWO.)

Select 2 answers

A.Enable CloudWatch Logs on the Kinesis stream to log all records.

B.Decrease the Lambda batch size to process records more frequently.

C.Add an Amazon SQS queue between Kinesis and Lambda to buffer records.

D.Increase the number of shards in the Kinesis data stream.

E.Configure a dead-letter queue on the Lambda function to capture failed records.

AnswersD, E

More shards provide higher throughput, reducing throttling.

Why this answer

Increasing the number of shards in the Kinesis data stream raises the total read and write capacity, reducing the likelihood of throttling that can cause records to be dropped. Option E is correct because configuring a dead-letter queue (DLQ) on the Lambda function captures records that fail processing after all retries, preventing data loss and enabling reprocessing.

Exam trap

The trap here is that candidates may think adding a buffer (SQS) or reducing batch size solves the issue, but the real cause is often shard throttling or processing failures, which require scaling shards and using a DLQ respectively.

Practice this question →

358

MCQhard

A data engineer is troubleshooting a Lambda function that reads from a Kinesis Data Stream, processes records, and writes to a Kinesis Data Firehose delivery stream. The Firehose delivery stream is configured to deliver data to an S3 bucket. The Lambda function is failing with an access denied error. The IAM policy attached to the Lambda execution role is shown in the exhibit. Which permission is missing?

A.firehose:PutRecord on the Firehose delivery stream

B.firehose:DescribeDeliveryStream on the Firehose delivery stream

C.logs:CreateLogGroup and logs:CreateLogStream on the CloudWatch log group

D.s3:PutObjectAcl on the S3 bucket

AnswerA

Correct. firehose:PutRecord is the IAM action needed to send records to a Kinesis Data Firehose delivery stream.

Why this answer

The Lambda function is failing due to a missing permission to write to the Kinesis Data Firehose delivery stream. The required permission is firehose:PutRecord (or firehose:PutRecordBatch). The IAM policy likely lacks this permission, causing an access denied error when the Lambda attempts to write records.

Option A correctly identifies this missing permission.

Exam trap

Candidates often confuse permissions for Kinesis Data Streams and Kinesis Data Firehose. Writing to a Firehose delivery stream requires firehose:PutRecord, not kinesis:PutRecord, which is for Kinesis Data Streams.

How to eliminate wrong answers

Option A is wrong because the Lambda function reads from the Kinesis Data Stream (requiring `kinesis:GetRecords`, `kinesis:DescribeStream`, etc.), not writes to it, so `kinesis:PutRecord` is irrelevant. Option C is wrong because CloudWatch Logs permissions (`logs:CreateLogGroup`, `logs:CreateLogStream`) are needed for logging but would cause a different error (e.g., 'Unable to create log stream'), not an access denied on Firehose. Option D is wrong because `s3:PutObjectAcl` is not required for Firehose to deliver to S3; Firehose uses `s3:PutObject` with bucket owner full control by default, and ACLs are not involved in this scenario.

Practice this question →

359

MCQeasy

A company wants to ingest streaming data from thousands of IoT devices into AWS for real-time processing. Each device sends JSON payloads of about 2 KB at a rate of 1 message per second. The data must be processed with a durable, ordered stream per device. Which service should the company use as the ingestion layer?

A.Amazon Simple Queue Service (Amazon SQS) with a FIFO queue.

B.Amazon Kinesis Data Streams.

C.Amazon Simple Notification Service (Amazon SNS) with a Lambda subscriber.

D.Amazon Kinesis Data Firehose with Direct Put.

AnswerB

Provides ordered, durable streaming.

Why this answer

Amazon Kinesis Data Streams is the correct choice because it provides durable, ordered stream processing per shard, which can be partitioned by device ID to maintain message order for each device. It supports real-time ingestion from thousands of IoT devices at 2 KB per message per second, with a retention period of up to 365 days and the ability to reprocess data via multiple consumers.

Exam trap

The trap here is that candidates often confuse Kinesis Data Streams with Kinesis Data Firehose, assuming Firehose can handle real-time ordered streams, but Firehose does not provide per-record ordering or real-time consumer access; it is a delivery stream, not a stream processing layer.

How to eliminate wrong answers

Option A is wrong because Amazon SQS FIFO queues guarantee exactly-once processing and strict ordering within a message group, but they are designed for decoupling microservices, not for high-throughput streaming ingestion from thousands of devices; FIFO throughput is limited to 300 transactions per second (with batching) and does not support multiple consumers reading the same stream in real-time. Option C is wrong because Amazon SNS is a pub/sub messaging service that does not provide ordered delivery or durable stream storage; it pushes messages to subscribers like Lambda, but ordering is not guaranteed and messages are not persisted for replay. Option D is wrong because Amazon Kinesis Data Firehose is a near-real-time delivery service that buffers data and writes it to destinations like S3 or Redshift, but it does not support ordered stream processing per device and cannot be consumed by multiple real-time applications directly; it is designed for batch loading, not for real-time ordered stream consumption.

Practice this question →

360

MCQmedium

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data is then processed by an AWS Lambda function that transforms the records and writes them to an Amazon S3 bucket. Recently, the Lambda function has been timing out and the S3 bucket is not receiving all expected data. The Kinesis stream is not throttling and has sufficient shards. Which step should the company take to resolve this issue?

A.Increase the Lambda function's reserved concurrency.

B.Increase the Lambda function's timeout and memory allocation.

C.Increase the number of shards in the Kinesis stream.

D.Enable enhanced fan-out on the Kinesis stream to reduce latency.

AnswerB

Increasing timeout and memory allows the function to run longer and with more compute resources.

Why this answer

The Lambda function is timing out, indicating that it cannot process the records within the allotted time. Increasing the Lambda function's timeout and memory allocation (Option B) provides more execution time and CPU resources, which can help process larger or more frequent records before timing out. Option A is incorrect because reserved concurrency limits the maximum number of concurrent instances, not the execution duration of a single instance.

Option C is incorrect because increasing shards is unnecessary; the stream is not throttling, and more shards do not resolve Lambda timeouts. Option D is incorrect because enhanced fan-out reduces latency for multiple consumers but does not prevent a single Lambda function from timing out due to processing time.

Practice this question →

361

MCQhard

A CloudFormation template defines an AWS Glue job. The job fails during execution with the error 'Unable to locate script: s3://scripts-bucket/etl-script.py'. The S3 bucket 'scripts-bucket' exists and the script file is present. What is the most likely cause?

A.The script location path is incorrect; it should include the bucket's region.

B.The IAM role for the Glue job does not have s3:GetObject permission on the scripts bucket.

C.The Glue job requires Python version 2, but the script uses Python 3 syntax.

D.The S3 bucket is in a different AWS region than the Glue job.

AnswerB

Glue needs to read the script from S3.

Why this answer

The Glue job fails to locate the script because the IAM role assigned to the job lacks the s3:GetObject permission on the scripts-bucket. Even though the bucket and object exist, AWS Glue requires the execution role to have explicit read access to the S3 object to download and execute the script. Without this permission, the job cannot retrieve the file, resulting in the 'Unable to locate script' error.

Exam trap

The trap here is that candidates assume the error is about the script path or region mismatch, but the real cause is almost always missing IAM permissions for the Glue execution role to read the script from S3.

How to eliminate wrong answers

Option A is wrong because S3 object paths do not include the bucket's region; the path format is s3://bucket-name/key, and region is irrelevant to the path. Option C is wrong because Python version compatibility would cause a syntax error during execution, not a 'Unable to locate script' error, which is a file access issue. Option D is wrong because S3 buckets and Glue jobs can operate across regions as long as the Glue job's IAM role has appropriate cross-region permissions; the error message specifically indicates a missing object, not a region mismatch.

Practice this question →

362

MCQmedium

A company uses AWS Data Pipeline to copy data from DynamoDB to S3 daily. Recently, the pipeline started failing with 'ThrottlingException' errors. The DynamoDB table has on-demand capacity. Which action should be taken to resolve the issue?

A.Increase the write capacity units of the DynamoDB table.

B.Replace Data Pipeline with AWS Glue using a DynamoDB connector.

C.Configure the pipeline to use a retry strategy with exponential backoff.

D.Disable the pipeline's retry logic and increase the timeout.

AnswerC

Retries with backoff alleviate throttling by slowing down requests.

Why this answer

ThrottlingException errors in AWS Data Pipeline when reading from DynamoDB indicate that the pipeline's read requests are exceeding the table's available throughput. Since the table uses on-demand capacity, which can handle spikes but has a per-second throughput limit, implementing exponential backoff in the pipeline's retry strategy allows it to reduce request rate upon throttling, aligning with AWS SDK best practices for handling DynamoDB throttling.

Exam trap

The trap here is that candidates assume on-demand capacity eliminates all throttling, but it only handles traffic spikes within a per-second limit, so throttling can still occur with sustained high read rates, and the correct fix is to implement exponential backoff in the pipeline's retry strategy rather than modifying capacity or switching tools.

How to eliminate wrong answers

Option A is wrong because DynamoDB on-demand capacity does not use provisioned write capacity units; increasing write capacity units is irrelevant and would require switching to provisioned mode, which is unnecessary. Option B is wrong because replacing Data Pipeline with AWS Glue using a DynamoDB connector does not inherently resolve throttling; Glue also uses the same DynamoDB read APIs and would face the same throttling issue without proper retry handling. Option D is wrong because disabling retry logic and increasing the timeout would cause the pipeline to fail permanently on the first throttling error, as it would not retry the request, and a longer timeout does not prevent throttling.

Practice this question →

363

MCQmedium

A real-time analytics application uses Amazon Kinesis Data Streams. The consumer application falls behind, causing increased latency. Which action would MOST effectively improve throughput?

A.Reduce the RecordMaxBufferedTime parameter in the Firehose delivery stream.

B.Increase the number of shards in the data stream.

C.Increase the batch size in the Kinesis Producer Library.

D.Use enhanced fan-out to dedicate a shard per consumer.

AnswerB

More shards increase parallelism and throughput capacity.

Why this answer

Increasing the number of shards in the Kinesis Data Stream directly increases the stream's read capacity (each shard supports up to 2 MB/s read and 5 transactions per second for shared throughput). This allows the consumer application to process more data in parallel, reducing the backlog and latency. The question specifies a consumer application falling behind, which is a read-throughput bottleneck, and scaling shards is the most effective way to address it.

Exam trap

The trap here is that candidates confuse producer-side optimizations (like KPL batch size or Firehose buffering) with consumer-side throughput issues, or they assume enhanced fan-out alone solves a shard-scaling problem without recognizing that the root cause is insufficient shard count for the consumer's processing rate.

How to eliminate wrong answers

Option A is wrong because RecordMaxBufferedTime is a Kinesis Firehose parameter that controls how long data is buffered before delivery to a destination; it does not affect Kinesis Data Streams consumer throughput or latency. Option C is wrong because increasing the batch size in the Kinesis Producer Library (KPL) improves write efficiency by aggregating records, but the problem is with the consumer falling behind, not the producer. Option D is wrong because enhanced fan-out provides dedicated 2 MB/s read throughput per consumer per shard, but it does not increase the total number of shards; if the consumer is already saturated on a single shard, enhanced fan-out helps only if multiple consumers exist, but the core issue of insufficient shard count remains.

Practice this question →

364

MCQhard

A data engineer is designing a data ingestion pipeline for real-time financial transactions. The pipeline must ensure exactly-once processing semantics and must handle duplicate records that may occur due to retries. Which combination of AWS services can achieve exactly-once processing?

A.Amazon Kinesis Data Streams with Amazon Kinesis Data Analytics for Apache Flink

B.Amazon SQS with AWS Lambda

C.Amazon MSK with AWS Lambda

D.Amazon Kinesis Data Firehose with AWS Lambda

AnswerA

Flink supports exactly-once processing with KDS.

Why this answer

Amazon Kinesis Data Streams with Kinesis Data Analytics for Apache Flink enables exactly-once processing by leveraging Flink's built-in checkpointing and two-phase commit protocols. Kinesis Data Streams provides a durable, ordered record store, while Flink uses its internal state and idempotent sinks to deduplicate records that arise from retries, ensuring each record is processed exactly once.

Exam trap

The trap here is that candidates often assume SQS FIFO or Lambda's idempotency can achieve exactly-once, but they overlook that without a stream processing framework with checkpointing and transactional sinks, retries can still introduce duplicates in distributed systems.

How to eliminate wrong answers

Option B is wrong because Amazon SQS with AWS Lambda provides at-least-once delivery by default; SQS does not guarantee deduplication without a FIFO queue and idempotent Lambda logic, and even then, exactly-once is not natively supported across retries. Option C is wrong because Amazon MSK (Kafka) with AWS Lambda requires custom checkpointing and idempotency logic in the Lambda function; MSK does not natively provide exactly-once semantics without a stream processing framework like Flink or Kafka Streams. Option D is wrong because Amazon Kinesis Data Firehose with AWS Lambda delivers records at least once and cannot guarantee exactly-once processing; Firehose buffers and batches data but does not support transactional commits or deduplication across retries.

Practice this question →

365

Multi-Selectmedium

A data engineering team is designing a data ingestion pipeline for a social media analytics platform. The pipeline must handle up to 100,000 events per second with less than 1 second processing latency. Which TWO services should be used together to meet these requirements?

Select 2 answers

A.AWS Glue streaming ETL

B.Amazon Kinesis Data Firehose

C.Amazon SQS

D.Amazon Kinesis Data Analytics for Apache Flink

E.Amazon Kinesis Data Streams

AnswersD, E

Flink can process streaming data with sub-second latency.

Why this answer

Amazon Kinesis Data Streams (Option E) is designed for real-time data ingestion at scale, supporting up to 1,000 records per second per shard with sub-second latency, making it suitable for 100,000 events per second when provisioned with sufficient shards. Amazon Kinesis Data Analytics for Apache Flink (Option D) can consume data directly from Kinesis Data Streams and perform low-latency stream processing (e.g., aggregations, filtering) with exactly-once semantics, meeting the <1 second processing latency requirement. Together, they form a fully managed, scalable pipeline for high-throughput, low-latency streaming analytics.

Exam trap

The trap here is that candidates often confuse Kinesis Data Firehose (near-real-time, buffered delivery) with Kinesis Data Streams (real-time, unbuffered ingestion), or assume AWS Glue streaming ETL can achieve sub-second latency when it is actually optimized for micro-batch processing with higher overhead.

Practice this question →

366

Multi-Selectmedium

A data engineering team is building a pipeline to transform CSV files uploaded to Amazon S3 into Parquet format using AWS Glue. The transformation must be serverless and handle files that arrive at irregular intervals. Which TWO actions should the team take? (Choose two.)

Select 2 answers

A.Configure an Amazon EMR cluster with Apache Spark for on-demand transformation.

B.Use an AWS Glue ETL job to convert CSV to Parquet.

C.Use AWS Data Pipeline to schedule a periodic transformation.

D.Use Amazon Redshift Spectrum to convert files during query execution.

E.Set up an S3 event notification to invoke an AWS Lambda function that triggers the Glue job.

AnswersB, E

Glue ETL jobs are serverless and can transform data formats.

Why this answer

AWS Glue ETL jobs provide a serverless Spark-based environment that can directly read CSV files from S3 and write them as Parquet, meeting the requirement for serverless transformation. Glue handles schema inference and conversion without managing any infrastructure, making it ideal for irregularly scheduled data.

Exam trap

The trap here is that candidates may confuse serverless with managed services like EMR or Data Pipeline, or assume Redshift Spectrum can transform data, when in fact it only queries external formats without writing back converted files.

Practice this question →

367

MCQeasy

A company needs to ingest data from multiple SaaS sources (e.g., Salesforce, Marketo) into Amazon S3 for analytics. Which AWS service is designed for this purpose?

A.AWS Transfer Family

B.AWS Glue

C.Amazon AppFlow

D.AWS DataSync

AnswerC

AppFlow is purpose-built for SaaS data ingestion.

Why this answer

Amazon AppFlow is a fully managed integration service specifically designed to securely transfer data between SaaS applications (like Salesforce, Marketo, Slack, and Zendesk) and AWS services such as Amazon S3 and Amazon Redshift. It supports scheduled, event-driven, or on-demand data ingestion with built-in transformations, filtering, and validation, making it the ideal choice for this use case.

Exam trap

The trap here is that candidates often confuse AWS Glue's ETL capabilities with native SaaS connectivity, but Glue requires custom connectors or AWS Glue Studio's visual ETL jobs to connect to SaaS sources, whereas AppFlow is purpose-built for this task.

How to eliminate wrong answers

Option A is wrong because AWS Transfer Family is used for transferring files into and out of Amazon S3 or Amazon EFS using SFTP, FTPS, or FTP protocols, not for integrating with SaaS APIs. Option B is wrong because AWS Glue is a serverless data integration service for ETL (extract, transform, load) jobs, but it does not natively connect to SaaS sources like Salesforce or Marketo without custom connectors or third-party libraries. Option D is wrong because AWS DataSync is designed for moving large volumes of data between on-premises storage and AWS (e.g., NFS, SMB, S3) or between AWS storage services, not for ingesting data from SaaS applications.

Practice this question →

368

MCQhard

A financial services company ingests stock trade data from multiple exchanges into an Amazon S3 bucket (trade-bucket). Each exchange sends a CSV file every 5 minutes. The data must be transformed into Parquet format and partitioned by exchange and date (trade_date) for efficient querying using Amazon Athena. The pipeline must handle late-arriving data (files up to 2 hours late) and ensure exactly-once processing to avoid duplicates. Currently, a scheduled AWS Glue ETL job runs every hour, reads new CSV files, converts them to Parquet, and writes to an output bucket. However, the team is experiencing data duplication: if the job fails midway, upon retry it reprocesses all files in the input folder, causing duplicates in the output. Additionally, the job takes too long because it scans all files each run. The engineer must redesign the pipeline to eliminate duplicates and improve efficiency. What should the engineer do?

A.Use AWS Glue Workflows to orchestrate the job and add a condition to check for duplicates before writing.

B.Set up an S3 event notification to invoke an AWS Lambda function that starts a Glue job with a parameter containing the S3 object key of the new file; modify the Glue job to process only that file and use the file key to avoid duplicates.

C.Modify the Glue job to move processed CSV files to an archive folder after successful transformation, and process only unprocessed files.

D.Replace Glue with Amazon EMR and use Spark Structured Streaming with checkpointing to process files incrementally.

AnswerB

This ensures each file is processed exactly once, and the job runs only on new files, improving efficiency.

Why this answer

The best approach. By setting up an S3 event notification to invoke a Lambda function that triggers a Glue job with the new file's S3 key as a parameter, the job processes only that specific file. Using the file key in the job logic ensures idempotency—if the job fails and retries, it reprocesses the same file key, and deduplication can be handled (e.g., by checking if the output partition already contains that file's data or using a job bookmark on the file key).

This achieves exactly-once processing and incremental processing (no full scans), improving efficiency. Option A (Glue Workflows) still processes all files each run and doesn't prevent duplicates if a file arrives after the job starts. Option C (moving CSV files to an archive folder) risks race conditions if late-arriving data comes while the job is running, and does not guarantee exactly-once if the job fails mid-way.

Option D (EMR with Spark Structured Streaming) is overly complex and expensive for a 5-minute CSV batch ingestion; checkpointing can handle failures but adds significant operational overhead.

Practice this question →

369

Multi-Selecteasy

A company is building a data lake on Amazon S3 and needs to ingest data from various on-premises sources. Which TWO AWS services can be used to transfer data securely over the internet?

Select 2 answers

A.AWS Snowcone

B.AWS DataSync

C.Amazon Kinesis Data Firehose

D.AWS Direct Connect

E.AWS CLI

AnswersB, E

DataSync can transfer data over the internet.

Why this answer

AWS DataSync is designed to securely transfer large amounts of data from on-premises storage to AWS over the internet. It uses TLS encryption for data in transit and can automate and accelerate transfers by leveraging parallel multi-threading and incremental updates. This makes it a correct choice for securely ingesting data into an S3-based data lake.

Exam trap

The trap here is that candidates often confuse AWS DataSync with AWS Direct Connect, thinking both are required for secure transfers, but Direct Connect is a network service that bypasses the internet entirely, while DataSync is a data transfer service that works over the internet or Direct Connect.

Practice this question →

370

Multi-Selectmedium

A company is building a data lake on Amazon S3. They need to ingest data from multiple sources, including relational databases, streaming data, and log files. Which THREE AWS services can be used to ingest data into the data lake?

Select 3 answers

A.Amazon Kinesis Data Firehose

B.AWS Database Migration Service (DMS)

C.Amazon Athena

D.Amazon Redshift Spectrum

E.AWS Glue

AnswersA, B, E

Ingests streaming data into S3.

Why this answer

Amazon Kinesis Data Firehose is a fully managed service for streaming data ingestion that can capture, transform, and load streaming data into Amazon S3 in near real-time. It supports sources like Amazon CloudWatch Logs, AWS IoT, and custom producers via the Kinesis Agent, making it ideal for log files and streaming data.

Exam trap

The trap here is confusing query engines (Athena, Redshift Spectrum) with ingestion services, as candidates often assume any service that touches S3 can be used for data loading.

Practice this question →

371

Multi-Selecteasy

A data engineer needs to ingest streaming data from an e-commerce application into Amazon S3 for near-real-time analytics. The solution must handle variable throughput and allow reprocessing of failed records. Which TWO AWS services should the engineer use? (Choose two.)

Select 2 answers

A.Amazon Kinesis Data Streams

B.Amazon SQS

C.Amazon DynamoDB Streams

D.Amazon Kinesis Data Firehose

E.Amazon RDS

AnswersA, D

Kinesis Data Streams ingests and stores streaming data in shards for real-time processing.

Why this answer

Amazon Kinesis Data Streams is correct because it provides a durable, scalable, real-time data ingestion layer that can handle variable throughput by sharding, and it retains data for up to 365 days, enabling reprocessing of failed records by replaying from a specific sequence number or timestamp.

Exam trap

The trap here is that candidates often confuse Amazon SQS with a streaming service, but SQS is a pull-based queue with no replay capability, whereas Kinesis Data Streams provides the persistent, replayable stream needed for reprocessing failed records.

Practice this question →

372

MCQmedium

Refer to the exhibit. A data engineer is attaching this IAM policy to an IAM role used by an AWS Glue job. The job reads from a Kinesis Data Streams stream and writes transformed data to an S3 bucket. When the job runs, it fails with an AccessDenied error for the Kinesis stream. What is the MOST likely cause?

A.The stream ARN in the policy is incorrect.

B.The IAM policy is missing the 'kinesis:DescribeStream' action.

C.The Glue job does not have permissions to call 'kinesis:PutRecord'.

D.The S3 bucket policy blocks the PutObject action from the Glue role.

AnswerB

Required to read stream metadata.

Why this answer

When an AWS Glue job reads from Kinesis Data Streams, it requires the `kinesis:DescribeStream` action to retrieve stream metadata such as shard IDs and the stream's current state. Without this permission, the Glue job cannot discover the stream's shards and fails with an AccessDenied error, even if other Kinesis actions like `GetRecords` are allowed. The error occurs because the IAM policy attached to the Glue role is missing this prerequisite action.

Exam trap

The trap here is that candidates assume only data-plane actions like `GetRecords` are needed for reading, forgetting that the Glue job's underlying KCL requires the control-plane `DescribeStream` action to discover shard topology before it can read any data.

How to eliminate wrong answers

Option A is wrong because if the stream ARN were incorrect, the error would typically be a 'ResourceNotFoundException' or 'InvalidArgumentException', not an AccessDenied error. Option C is wrong because the Glue job is reading from the stream, not writing to it; `kinesis:PutRecord` is for writing data, and the job would need `kinesis:GetRecords` and `kinesis:GetShardIterator` instead. Option D is wrong because the error is specifically for the Kinesis stream, not for S3; an S3 bucket policy blocking PutObject would cause an AccessDenied error on the S3 PutObject operation, not on the Kinesis stream.

Practice this question →

373

MCQeasy

An organization needs to ingest data from on-premises databases into AWS S3 for archival purposes. The data volume is several TB per day, and the network has moderate bandwidth. Which AWS service is BEST suited for this bulk data transfer?

A.AWS Direct Connect

B.Amazon S3 Transfer Acceleration

C.AWS Snowball

D.AWS DataSync

AnswerD

DataSync automates and accelerates moving data from on-premises to AWS.

Why this answer

AWS DataSync is the best choice because it is designed for efficiently moving large volumes of data (several TB per day) from on-premises storage to AWS, including S3. It automates and accelerates data transfer over the existing network using a purpose-built protocol and parallel multi-threading, while handling encryption, validation, and scheduling. For moderate bandwidth, DataSync optimizes throughput and can resume interrupted transfers, making it ideal for ongoing archival ingestion.

Exam trap

The trap here is that candidates often confuse AWS DataSync with AWS Direct Connect, thinking a dedicated network link is required for large transfers, but DataSync is optimized to work over existing internet connections with moderate bandwidth, making it the more practical and cost-effective service for this use case.

How to eliminate wrong answers

Option A is wrong because AWS Direct Connect provides a dedicated network connection, but it is a network service, not a data transfer service; it requires significant setup and cost, and does not include built-in data management features like scheduling or validation for bulk ingestion. Option B is wrong because Amazon S3 Transfer Acceleration only speeds up uploads to S3 over the public internet using edge locations, but it does not handle the initial data extraction from on-premises databases or provide the orchestration needed for multi-TB daily transfers. Option C is wrong because AWS Snowball is a physical device for offline data transfer, which is suitable for very large datasets (petabytes) or low-bandwidth scenarios, but it introduces latency for shipping and is not ideal for daily recurring transfers of several TB per day.

Practice this question →

374

MCQmedium

A company uses AWS DMS to migrate data from Oracle to Aurora MySQL. During the ongoing replication, the target table shows duplicate primary key errors. What is the most likely cause?

A.DMS is using 'Limited LOB mode' and truncating LOB data, causing row mismatches.

B.The source table has a trigger that inserts additional rows.

C.The target table has an auto-increment column, and DMS is inserting explicit values that conflict.

D.The DMS task is configured with 'Parallel apply' threads that cause race conditions.

AnswerC

DMS inserts values for the PK, but auto-increment may also generate values, causing duplicates.

Why this answer

When DMS replicates into a target table with an auto-increment column, it attempts to insert explicit values from the source. If the target's auto-increment counter has already generated a value matching one of those explicit inserts, a duplicate primary key error occurs. This is the most likely cause because DMS does not automatically skip or re-map auto-increment columns unless explicitly configured.

Exam trap

The DEA-C01 exam often tests the misconception that duplicate key errors are caused by data type mismatches or LOB truncation, but the real trap is forgetting that auto-increment columns on the target can conflict with explicit primary key values from the source during ongoing replication.

How to eliminate wrong answers

Option A is wrong because Limited LOB mode truncates LOB data to a maximum size, which can cause data loss or row mismatches, but it does not produce duplicate primary key errors. Option B is wrong because source triggers that insert additional rows would add extra rows to the source, but DMS captures changes from the source transaction logs (e.g., Oracle Redo Logs) and would replicate those additional inserts as separate events; they would not cause duplicate key errors on the target unless the inserted rows have the same primary key as existing rows, which is not a typical trigger behavior. Option D is wrong because Parallel apply threads improve performance by applying changes concurrently, but DMS handles conflict resolution and ordering to prevent race conditions; duplicate key errors from parallel apply are extremely rare and not the most likely cause.

Practice this question →

375

MCQhard

A company uses AWS Glue to run ETL jobs that process data from Amazon RDS to Amazon S3. The jobs run nightly and take 3 hours to complete. The data volume is growing by 20% each month. The engineer needs to reduce job runtime and cost. The source RDS is a db.r5.large instance. Which approach would be MOST effective?

A.Reduce the number of DPUs to lower cost and accept longer runtime.

B.Increase the number of Glue workers and choose a G.1X or G.2X worker type.

C.Create a read replica of the RDS instance and point the Glue job to the replica.

D.Enable S3 Transfer Acceleration on the destination bucket.

AnswerB

More workers increase parallelism.

Why this answer

Increasing the number of Glue workers and choosing a G.1X or G.2X worker type directly increases parallelism and provides more memory per worker, reducing job runtime at a manageable cost increase. Option A is wrong; reducing DPUs would increase runtime, not reduce it. Option C is wrong because a read replica does not improve Glue processing speed; the bottleneck is Glue's processing capacity, not the source database's read capacity.

Option D is wrong because S3 Transfer Acceleration improves upload speed to S3, not Glue job processing.

Practice this question →