Knowledge + Practice

CCNA Data Ingestion and Transformation Questions

66 of 591 questions · Page 8/8 · Data Ingestion and Transformation · Answers revealed

Practice these questions Domain overview All questions

526

MCQmedium

A company uses AWS Glue ETL jobs to transform data in S3. The job runs successfully but takes longer than expected. The data is in Parquet format and partitioned by date. Which change would most improve performance without increasing cost?

A.Repartition the data by a different column.

B.Convert Parquet to CSV for faster serialization.

C.Increase the number of DPUs for the job.

D.Enable pushdown predicates to filter partitions early.

AnswerD

Reduces data scanned, improving performance.

Why this answer

Pushdown predicates allow AWS Glue to filter data at the storage layer (e.g., S3 partition pruning) before reading it into memory. Since the data is partitioned by date, enabling pushdown predicates reduces the amount of data scanned, which directly decreases job runtime without requiring additional DPUs or changing the data format.

Exam trap

The trap here is that candidates often assume performance issues are solved by adding more resources (DPUs) or changing file formats, when the real bottleneck is reading unnecessary data due to lack of partition pruning.

How to eliminate wrong answers

Option A is wrong because repartitioning by a different column would likely increase shuffle overhead and may not align with the existing partition structure, potentially worsening performance. Option B is wrong because converting Parquet to CSV would increase data size and I/O due to CSV's lack of compression and columnar storage, making the job slower and more expensive. Option C is wrong because increasing DPUs would raise cost without addressing the root cause (scanning unnecessary partitions), and the question explicitly asks for a change that does not increase cost.

Practice this question →

527

Multi-Selecteasy

A data engineer is designing a data ingestion pipeline that uses AWS Lambda to process records from a Kinesis Data Stream and write to DynamoDB. Which TWO strategies can help handle increased throughput and prevent data loss? (Choose TWO.)

Select 2 answers

A.Configure the Lambda event source mapping with a batch window and set the number of concurrent batches per shard

B.Use synchronous invocation of Lambda from the producer

C.Increase the number of shards in the Kinesis data stream

D.Configure a dead-letter queue (DLQ) for the Lambda function

E.Increase the Lambda function timeout

AnswersA, C

This improves throughput and handles spikes.

Why this answer

Configuring a batch window allows Lambda to accumulate records from the Kinesis stream for up to 300 seconds before invoking the function, which helps smooth out traffic spikes and reduces the number of invocations. Setting the number of concurrent batches per shard (via the ParallelizationFactor, up to 10) enables Lambda to process multiple batches from the same shard in parallel, increasing throughput without data loss. This combination ensures that records are processed efficiently even under high load, as Lambda can handle more concurrent executions per shard while batching reduces the risk of throttling.

Exam trap

The trap here is that candidates often think a dead-letter queue (DLQ) prevents data loss during high throughput, but DLQs only capture records after processing failures, not during ingestion spikes, and they confuse synchronous invocation (Option B) with the actual asynchronous event source mapping used by Kinesis.

Practice this question →

528

MCQmedium

A company uses AWS Glue for ETL jobs. The data engineer needs to ensure that the Glue job can access an S3 bucket in another account. What is the recommended approach?

A.Create an IAM role in the target account and have the Glue job assume that role

B.Assign an IAM role to the Glue job with permissions to access the bucket, and configure the bucket policy to allow the role

C.Configure the S3 bucket policy to allow the Glue job's IAM role and also set the Glue job's resource-based policy

D.Store AWS access keys for the target account in AWS Secrets Manager and have the Glue job retrieve them

AnswerB

Correct. To allow a Glue job in one account to access an S3 bucket in another account, the Glue job's IAM role must have the necessary S3 permissions, and the bucket policy in the target account must explicitly grant those permissions to the role's ARN. This is the recommended cross-account access pattern.

Why this answer

For a Glue job in one account to access an S3 bucket in another account, the standard approach is to grant the Glue job's IAM role permissions to perform S3 actions on the bucket and configure the bucket policy in the target account to allow that role's ARN. Option A is technically possible if the target account's role trusts the Glue job's account and the job has sts:AssumeRole permissions, but this adds unnecessary complexity and is not the recommended method. Option C is incorrect because Glue jobs do not have resource-based policies.

Option D is incorrect because Glue jobs cannot use static access keys.

Practice this question →

529

MCQmedium

A company is using Amazon Kinesis Data Firehose to deliver streaming data to an Amazon S3 bucket. The data is delivered in JSON format. The company wants to convert the data to Apache Parquet format before delivery to reduce storage costs and improve query performance. How can this be achieved?

A.Deliver data to S3 as JSON, then use Amazon Athena to convert to Parquet.

B.Use the AWS Glue Data Catalog to define a schema and configure Firehose to use it for Parquet conversion.

C.Write an AWS Lambda function to transform the data to Parquet and deliver it to S3.

D.Configure the Firehose stream to convert data to Parquet automatically without any additional setup.

AnswerB

Correct. Kinesis Data Firehose can convert incoming data to Parquet or ORC using a schema from the AWS Glue Data Catalog. This is the recommended method.

Why this answer

Kinesis Data Firehose can convert the input data to Parquet or ORC format using a schema from the AWS Glue Data Catalog. Option A is incorrect because delivering as JSON then converting with Athena is an extra step after storage, not before delivery, and does not reduce storage costs from the outset. Option C is incorrect because Lambda can be used for custom transformations, but Firehose natively supports Parquet conversion using Glue.

Option D is incorrect because Firehose cannot convert to Parquet automatically without a schema; you must provide a schema (e.g., from Glue).

Practice this question →

530

MCQmedium

A data engineer is troubleshooting a Lambda function that reads from the Kinesis stream 'my-data-stream'. The Lambda function is able to read data but occasionally fails with 'KMS.AccessDeniedException'. What is the most likely cause?

A.The Lambda function's execution role does not have kms:Decrypt permission for the KMS key.

B.The retention period is too short; increase it.

C.The stream has too few shards; increase shard count.

D.The Lambda function is not authorized to consume from Kinesis streams.

AnswerA

Kinesis uses KMS for encryption; consumers need decrypt permission.

Why this answer

The KMS.AccessDeniedException indicates that the Lambda function's execution role lacks the kms:Decrypt permission for the AWS KMS key used to encrypt the Kinesis stream. When a Kinesis stream is encrypted with a customer managed KMS key, the consumer (Lambda) must have explicit decrypt permissions on that key to read the data records.

Exam trap

The trap here is that candidates may confuse KMS permissions with Kinesis stream permissions, assuming the error is about stream consumption authorization rather than decryption of encrypted data.

How to eliminate wrong answers

Option B is wrong because a short retention period would cause data to expire, not produce a KMS.AccessDeniedException. Option C is wrong because insufficient shards would cause throttling or throughput issues, not a KMS access error. Option D is wrong because the Lambda function is already able to read data (as stated), so it has Kinesis consumption authorization; the error is specifically about KMS decryption, not stream-level permissions.

Practice this question →

531

MCQmedium

A data engineer notices that an AWS Glue job writing to Amazon S3 in Parquet format creates many small files (less than 1 MB each). This leads to poor query performance in Amazon Athena. What is the BEST way to reduce the number of output files?

A.Enable 'groupFiles' in the Glue job's S3 target configuration.

B.Use 'coalesce(1)' at the end of the ETL script.

C.Use 'repartition(100)' to increase parallelism.

D.Configure an S3 lifecycle policy to delete small files.

AnswerA

Glue's groupFiles option merges small files during write.

Why this answer

Enabling 'groupFiles' in the AWS Glue job's S3 target configuration instructs Glue to coalesce small files into larger ones (default target size ~128 MB) during the write phase. This directly reduces the number of small Parquet files written to S3, improving Athena query performance by minimizing S3 LIST and GET overhead.

Exam trap

The trap here is that candidates often confuse 'coalesce(1)' or 'repartition()' as file-size solutions, but these operations control the number of Spark partitions, not the final file size, and can actually worsen the problem or cause job failures.

How to eliminate wrong answers

Option B is wrong because 'coalesce(1)' forces all data into a single partition, which can cause out-of-memory errors or severe performance degradation in distributed Spark jobs, and it does not address the root cause of small files from multiple tasks. Option C is wrong because 'repartition(100)' increases parallelism, which would create even more output files (up to 100), worsening the small-file problem. Option D is wrong because an S3 lifecycle policy deletes files after a set time period, but it does not consolidate existing small files or prevent them from being created; it only removes them after the fact, which does not solve the immediate query performance issue.

Practice this question →

532

MCQmedium

A data engineer is troubleshooting an AWS Glue job that is failing with an Access Denied error when trying to read data from an S3 bucket. The IAM policy attached to the Glue job's IAM role is shown in the exhibit. What is the likely cause of the failure?

A.The policy does not include s3:GetObject or s3:PutObject permissions.

B.The policy does not include glue:StartJobRun permission.

C.The policy does not include s3:ListBucket permission on the bucket.

D.The policy does not include glue:GetJobRun permission.

AnswerC

Glue needs s3:ListBucket to enumerate objects in the bucket before reading.

Why this answer

The IAM policy grants s3:GetObject and s3:PutObject on the bucket objects, but it does not include s3:ListBucket permission on the bucket itself. When AWS Glue reads data from S3, it needs to list the objects in the bucket first (s3:ListBucket) to discover which objects to read. Without this permission, the Glue job fails with an Access Denied error even though it has GetObject permission.

Option A is incorrect because the policy does include s3:GetObject (implied for objects) and the issue is not about PutObject. Option B is incorrect because glue:StartJobRun is not related to S3 access; the error is S3-specific. Option D is incorrect because glue:GetJobRun is unrelated to the S3 access issue.

Practice this question →

533

MCQmedium

Refer to the exhibit. A data engineer is creating an IAM policy for an application that sends data to a Kinesis stream and stores processed data in S3. The policy is attached to an IAM role used by an EC2 instance. The application fails to write to S3 with an access denied error. What is the cause?

A.The policy does not allow s3:ListBucket on the bucket.

B.The IAM role is not attached to the EC2 instance profile.

C.The policy does not allow kinesis:PutRecord on the stream.

D.The EC2 instance does not have an internet gateway to reach S3.

AnswerA

Some operations require ListBucket permission; without it, the SDK may fail.

Why this answer

The error occurs because the IAM policy grants s3:PutObject but not s3:ListBucket on the target S3 bucket. When the application writes to S3, the AWS SDK often performs a ListBucket operation first to verify bucket existence or to handle multipart uploads, and without s3:ListBucket permission, the request is denied with an access denied error.

Exam trap

The DEA-C01 exam often tests the nuance that S3 write operations (PutObject) require the s3:ListBucket permission on the bucket for SDK-level operations, even though the explicit API call is only PutObject.

How to eliminate wrong answers

Option B is wrong because the question states the policy is attached to an IAM role used by the EC2 instance, and the error is specifically about S3 access, not about role attachment. Option C is wrong because the application fails to write to S3, not to Kinesis, so kinesis:PutRecord permissions are irrelevant to this error. Option D is wrong because EC2 instances can access S3 via a VPC endpoint or through the public internet using a NAT gateway or internet gateway, but the error is an IAM permissions issue, not a network connectivity issue.

Practice this question →

534

MCQmedium

A company is ingesting streaming data into Kinesis Data Streams. The consumer application experiences high latency due to a single shard bottleneck. What is the most effective way to reduce latency?

A.Increase the number of shards in the data stream.

B.Wait for automatic scaling to add shards.

C.Use the Kinesis Client Library (KCL) to process records.

D.Switch to Amazon Kinesis Data Firehose.

AnswerA

More shards increase parallelism and throughput, reducing latency.

Why this answer

Increasing the number of shards increases throughput and reduces latency. Waiting for autoscaling is passive, using KCL is for processing, and switching to Firehose changes the architecture.

Practice this question →

535

MCQhard

A company is ingesting streaming data from social media feeds using Amazon Kinesis Data Streams. The data volume peaks at 10,000 records per second, and each record is up to 1 KB. The company needs to archive the raw data in Amazon S3 in near real-time and also make it available for real-time analytics using Amazon Kinesis Data Analytics. What is the MOST efficient architecture to meet these requirements?

A.Use Kinesis Data Streams as the ingestion point. Use Kinesis Data Firehose to read from the stream, convert to Parquet, and write to S3. Use a Lambda function to send data to Kinesis Data Analytics.

B.Use Kinesis Data Streams as the ingestion point. Use a Lambda function to read from the stream, write to S3, and send data to Kinesis Data Analytics.

C.Use two Kinesis Data Streams: one for S3 delivery and one for Kinesis Data Analytics.

D.Use Kinesis Data Streams as the ingestion point. Use Kinesis Data Firehose to read from the stream and write to S3. Use Kinesis Data Analytics to read directly from the same stream.

AnswerD

Firehose can read from the stream and write to S3; Kinesis Data Analytics can read from the same stream for real-time analytics.

Why this answer

Kinesis Data Streams can serve as a single ingestion point, with Kinesis Data Firehose reading from the stream to deliver data to S3 (with optional transformation) and Kinesis Data Analytics reading directly from the same stream for real-time analytics. This avoids unnecessary duplication of streams or Lambda-based processing, which would add latency and complexity. The architecture is the most efficient as it leverages native integrations without intermediate compute.

Exam trap

The trap here is that candidates often overcomplicate the architecture by adding unnecessary Lambda functions or duplicate streams, not realizing that Kinesis Data Firehose and Kinesis Data Analytics can both consume from the same Kinesis Data Stream natively.

How to eliminate wrong answers

Option A is wrong because it suggests using a Lambda function to send data to Kinesis Data Analytics, which is unnecessary and introduces additional cost and latency; Kinesis Data Analytics can read directly from the Kinesis Data Stream. Option B is wrong because using a Lambda function to write to S3 and send data to Kinesis Data Analytics adds processing overhead and potential throughput limitations, whereas Kinesis Data Firehose is purpose-built for streaming to S3 with near-real-time delivery. Option C is wrong because using two separate Kinesis Data Streams is redundant and increases cost and management overhead; a single stream can be consumed by both Firehose and Kinesis Data Analytics simultaneously.

Practice this question →

536

MCQeasy

A company uses AWS Lake Formation to manage data lake permissions. A data engineer needs to grant an IAM role 'Read' access to a specific database and all its tables in the Data Catalog. What is the MOST efficient way to achieve this?

A.Grant 'Super' permission on the Data Catalog

B.Add the IAM role to the Lake Formation administrators group

C.Grant 'Select' on the database and select 'Include' to apply to all tables

D.Grant 'Describe' on the database and 'Select' on each table individually

AnswerC

This grants read access to all tables in one operation.

Why this answer

Lake Formation allows granting 'Select' permission on a database with the 'Include' option, which automatically applies the same permission to all current and future tables in that database. This is the most efficient way to grant read access to an IAM role for a specific database and all its tables in the Data Catalog, as it avoids manual per-table grants.

Exam trap

The trap here is that candidates may confuse 'Super' or admin roles with a simple read grant, or think that granting 'Describe' on the database is sufficient for read access, when in fact 'Select' on the tables is required for data access.

How to eliminate wrong answers

Option A is wrong because 'Super' permission grants full administrative access to the Data Catalog, which is excessive and violates the principle of least privilege. Option B is wrong because adding the IAM role to the Lake Formation administrators group grants full administrative permissions over all Lake Formation resources, not just read access to a specific database. Option D is wrong because granting 'Select' on each table individually is inefficient and does not cover future tables, while 'Describe' on the database alone does not grant read access to table data.

Practice this question →

537

Multi-Selectmedium

A company needs to ingest streaming data from an existing Amazon Kinesis Data Streams into Amazon S3 with partitioning by date. Which TWO services can accomplish this with minimal coding? (Choose two.)

Select 2 answers

A.Amazon Kinesis Data Firehose

B.AWS Glue Streaming

C.AWS Lambda

D.Amazon Kinesis Data Analytics

E.Amazon S3 Transfer Acceleration

AnswersA, C

Firehose can read from a Kinesis stream and deliver to S3 with partitioning.

Practice this question →

538

Multi-Selecthard

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams with multiple consumers. The data must be processed by a Lambda function for real-time alerts and also stored in Amazon S3 for historical analysis. Which THREE components are needed to implement this architecture? (Choose THREE.)

Select 3 answers

A.Amazon Kinesis Data Firehose

B.Amazon Kinesis Data Streams

C.AWS Lambda function

D.Amazon Kinesis Data Analytics

E.Amazon SQS queue

AnswersA, B, C

Reads from the stream and delivers to S3.

Why this answer

Amazon Kinesis Data Firehose is the correct component because it is the fully managed service designed to load streaming data into Amazon S3 without requiring custom code. It can directly subscribe to a Kinesis Data Stream as its source and automatically buffer, batch, compress, and deliver records to S3 for historical analysis, making it the ideal choice for the storage leg of this architecture.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Analytics as a necessary component for any streaming pipeline, but it is only required when you need to perform real-time analytics on the stream, not for simple data movement to S3.

Practice this question →

539

MCQhard

A company runs a real-time analytics platform that ingests data from thousands of sensors via Amazon Kinesis Data Streams. Each sensor sends a JSON payload every second. The data is consumed by a fleet of EC2 instances running a custom consumer application. Recently, the consumer has been falling behind, with the iterator age exceeding 10 minutes. The company has already increased the number of shards to 100, but the problem persists. The consumer application is single-threaded per shard and uses the Kinesis Client Library (KCL). The CPU utilization on the EC2 instances is below 30%. What should the data engineer do to reduce the iterator age?

A.Increase the number of shards to 200

B.Use larger EC2 instances with more vCPUs

C.Modify the consumer to use multiple worker threads per shard

D.Replace the EC2 consumer with AWS Lambda functions

AnswerC

Increases processing parallelism within each shard.

Why this answer

The correct solution. The consumer is single-threaded per shard, which limits processing throughput despite low CPU utilization. Using multiple worker threads per shard allows concurrent processing of records from the same shard, reducing iterator age.

Option A (more shards) was already tried and did not resolve the issue. Option B (larger EC2 instances) is unlikely to help because CPU is not the bottleneck. Option D (Lambda) may not handle the high-frequency sensor data efficiently and can introduce additional latency.

Practice this question →

540

MCQeasy

A data pipeline ingests streaming data from thousands of IoT devices into Kinesis Data Streams. The data must be transformed using a simple field mapping before being stored in S3. Which service should be used to perform the transformation with minimal operational overhead?

A.AWS Lambda function invoked by the Kinesis stream

B.AWS Glue ETL job

C.Kinesis Data Analytics

D.Kinesis Data Firehose with a Lambda transformation

AnswerD

Firehose can invoke a Lambda function for simple transformations before delivery.

Why this answer

Kinesis Data Firehose can invoke a Lambda function to perform simple field mapping transformations before delivering data to S3, minimizing operational overhead. Option A is wrong because AWS Lambda invoked directly by the Kinesis stream requires custom logic for S3 delivery and stream management, increasing overhead. Option B is wrong because AWS Glue ETL jobs are designed for batch processing and are more complex to set up for streaming transformations.

Option C is wrong because Kinesis Data Analytics is used for real-time analytics with SQL or Flink, not simple field mapping transformations.

Practice this question →

541

MCQeasy

A data engineer needs to ingest data from an external FTP server into S3 on a schedule. The FTP server is only accessible via VPN. Which AWS service is best suited for this task?

A.AWS Transfer Family

B.AWS Snowcone

C.AWS Glue with a Python shell

D.AWS DataSync

AnswerA

Supports FTP and integrates with VPN.

Why this answer

AWS Transfer Family supports FTP, FTPS, and SFTP protocols and can be integrated with a VPC using Elastic IPs or a VPC endpoint, enabling secure access to an FTP server reachable only via VPN. It automates the transfer of files from the external FTP server to Amazon S3 on a schedule without requiring custom code or infrastructure management.

Exam trap

The DEA-C01 exam often tests the distinction between services that support FTP natively (Transfer Family) versus those that only handle file transfers over NFS/SMB or via physical devices, leading candidates to mistakenly choose DataSync or Snowcone for FTP-based ingestion.

How to eliminate wrong answers

Option B is wrong because AWS Snowcone is a physical edge device used for offline data migration or edge computing, not for scheduled online transfers from an FTP server. Option C is wrong because AWS Glue with a Python shell is a serverless ETL service that can run custom scripts, but it lacks native FTP protocol support and would require complex, non-scalable workarounds to handle FTP transfers over VPN. Option D is wrong because AWS DataSync is designed for high-speed transfers between on-premises storage and AWS, but it does not support FTP protocol and cannot connect to an FTP server directly.

Practice this question →

542

MCQmedium

A company wants to ingest data from SaaS applications (e.g., Salesforce, Marketo) into Amazon S3 for analytics. The data volume is moderate and updates occur frequently. Which AWS service is BEST suited for this task?

A.Amazon Kinesis Data Streams

B.Amazon AppFlow

C.AWS Database Migration Service (DMS)

D.AWS Glue

AnswerB

AppFlow supports many SaaS sources and can write to S3.

Why this answer

Amazon AppFlow is specifically designed to securely transfer data from SaaS applications like Salesforce and Marketo to AWS services such as Amazon S3. It handles authentication, data transformation, and scheduling. Amazon Kinesis Data Streams is for real-time streaming data, not batch ingestion from SaaS.

AWS Database Migration Service (DMS) is for migrating databases, not SaaS data. AWS Glue is an ETL service that can process data but is not optimized for direct SaaS ingestion.

Practice this question →

543

MCQeasy

A data pipeline ingests daily CSV files from an FTP server into an Amazon S3 bucket. The files must be converted to Parquet format and partitioned by date for efficient querying using Amazon Athena. Which AWS service is most suitable for this transformation?

A.Amazon Kinesis Data Firehose

B.Amazon EMR

C.AWS Glue

D.AWS Lambda

AnswerC

Glue provides a serverless Spark environment that can transform CSV to Parquet and partition data efficiently.

Why this answer

AWS Glue is the most suitable service because it provides a fully managed ETL (Extract, Transform, Load) capability that can natively read CSV files from S3, convert them to Parquet format, and write the output partitioned by date. Glue's built-in transform 'ConvertToParquet' and dynamic frame partitioning make this a straightforward, serverless solution without needing to manage infrastructure.

Exam trap

The trap here is that candidates often confuse AWS Glue with Amazon EMR, thinking EMR is always needed for Parquet conversion, but Glue's serverless ETL is more appropriate for scheduled batch jobs without cluster management overhead.

How to eliminate wrong answers

Option A is wrong because Amazon Kinesis Data Firehose is designed for streaming data ingestion, not batch processing of daily CSV files from an FTP server; it lacks native support for reading from S3 as a source and performing complex transformations like CSV-to-Parquet conversion with custom partitioning. Option B is wrong because Amazon EMR is a managed Hadoop cluster that can perform this transformation, but it requires provisioning and managing EC2 instances, which is overkill for a simple daily batch job and not the most suitable service for a serverless, cost-effective solution. Option D is wrong because AWS Lambda has a maximum execution time of 15 minutes and a limited memory capacity (up to 10 GB), which is insufficient for processing large daily CSV files (e.g., gigabytes in size) and performing efficient Parquet conversion with partitioning.

Practice this question →

544

MCQhard

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams. The data must be enriched with reference data from a DynamoDB table before being written to S3. The engineer wants to minimize latency. Which architecture is BEST?

A.Use AWS Glue streaming ETL to read from Kinesis, enrich, and write to S3.

B.Use Kinesis Data Analytics for Apache Flink to enrich and output to Firehose.

C.Use Kinesis Data Firehose with a Lambda function for enrichment.

D.Use a Lambda function to poll the stream, enrich, and write to Firehose.

AnswerB

Flink provides low-latency streaming enrichment with external sources.

Why this answer

(Kinesis Data Analytics for Apache Flink) is the best choice because it supports low-latency enrichment using external sources like DynamoDB via asynchronous I/O, meeting the requirement to minimize latency. Option A (AWS Glue streaming ETL) is designed for batch-oriented processing and introduces higher latency. Option C (Kinesis Data Firehose with a Lambda function) may experience cold starts and limited concurrency, increasing latency.

Option D (Lambda polling the stream) also suffers from cold starts and scalability issues, making it less suitable for low-latency enrichment.

Practice this question →

545

MCQmedium

A company has a Kinesis Data Firehose delivery stream that receives JSON data from IoT devices. The data is delivered to an S3 bucket. The company notices that the data in S3 is delayed by up to 30 minutes. The Firehose stream is configured with a buffer size of 1 MB and a buffer interval of 60 seconds. The incoming data rate is approximately 100 KB per second. The company needs to reduce the delivery latency to under 5 minutes. Which action should the company take?

A.Enable Lambda transformation to process data faster.

B.Increase the buffer interval to 300 seconds.

C.Change the compression format from GZIP to Snappy.

D.Decrease the buffer size to 256 KB.

AnswerD

Smaller buffer size causes more frequent deliveries, reducing latency.

Why this answer

The observed latency of up to 30 minutes is likely due to the buffer size being too large relative to the data rate, causing long waits to fill the buffer. With a data rate of 100 KB/s and a buffer size of 1 MB, the buffer fills in approximately 10 seconds, but the buffer interval of 60 seconds already limits delivery to at most 60 seconds. However, the 30-minute delay suggests additional issues such as backlog or configuration errors.

Decreasing the buffer size to 256 KB will cause more frequent deliveries (every ~2.5 seconds), reducing latency. Option A (Lambda transformation) adds processing time and increases latency. Option B (increase buffer interval to 300 seconds) would increase latency.

Option C (change compression) does not affect delivery frequency. Therefore, option D is correct.

Practice this question →

546

Multi-Selecthard

A data engineer is building a pipeline to ingest data from an on-premises Oracle database into Amazon S3. The pipeline must capture change data (CDC) in near real-time and handle schema changes. Which TWO AWS services should the engineer use?

Select 2 answers

A.AWS Glue Schema Registry

B.AWS Snowball Edge

C.Amazon AppFlow

D.Amazon Kinesis Data Streams with Kinesis Agent

E.AWS Database Migration Service (DMS) with CDC

AnswersA, E

Manages schema evolution for streaming data.

Why this answer

AWS Glue Schema Registry (A) is correct because it enables schema discovery, validation, and evolution for streaming data, allowing the pipeline to handle schema changes from the Oracle CDC source. It integrates with Apache Kafka and Amazon Kinesis Data Streams to enforce schema compatibility rules (e.g., backward, forward, full) as data arrives, ensuring downstream consumers can adapt to evolving schemas without breaking.

Exam trap

The DEA-C01 exam often tests the misconception that Amazon Kinesis Data Streams alone can perform CDC from a database, but Kinesis requires a separate agent or connector (like Debezium or DMS) to read database logs, making DMS the correct CDC service.

Practice this question →

547

Multi-Selecteasy

Which TWO AWS services can be used to ingest data from an on-premise relational database into Amazon S3 on a one-time basis?

Select 2 answers

A.AWS Data Pipeline

B.AWS Database Migration Service (DMS)

C.AWS Glue

D.Amazon Simple Queue Service (SQS)

E.Amazon Kinesis Data Streams

AnswersB, C

AWS Database Migration Service (DMS) is correct. It can perform a one-time full-load migration from an on-premise relational database directly to Amazon S3, supporting multiple output formats.

Why this answer

AWS DMS can perform one-time full-load migrations from on-premise relational databases to Amazon S3 by connecting to the source and writing data directly to S3 in formats like CSV or Parquet. AWS Glue can also be used for one-time data ingestion by creating an ETL job that reads from the source database via JDBC and writes to S3, with the option to run the job on demand. Both services support one-time transfers without ongoing replication.

Exam trap

The trap here is that candidates might think Glue is only for scheduled or recurring jobs, but Glue ETL jobs can be triggered on-demand for a one-time migration. Also, some might confuse Kinesis or SQS as suitable for one-time batch ingestion, but those are designed for streaming data.

Practice this question →

548

MCQmedium

A data pipeline uses Kinesis Data Firehose to deliver streaming data to an S3 bucket. The data volume spikes occasionally, causing the Firehose buffer to fill up and leading to increased delivery latency. The latency must remain under 60 seconds. What should be done to minimize latency?

A.Enable GZIP compression on the Firehose delivery stream.

B.Increase the buffer size to 128 MB to accommodate larger batches.

C.Switch to Kinesis Data Streams with a Lambda consumer.

D.Reduce the buffer interval to 60 seconds.

AnswerD

This forces delivery every 60 seconds, meeting the latency requirement.

Why this answer

Reducing the buffer interval to 60 seconds ensures that Firehose delivers data to S3 at most every 60 seconds, directly capping latency even if the buffer size is not full. This aligns with the requirement to keep latency under 60 seconds, as Firehose delivers data when either the buffer interval or buffer size threshold is met first.

Exam trap

AWS often tests the misconception that increasing buffer size or enabling compression reduces latency, when in fact these options increase latency by allowing more data to accumulate before delivery.

How to eliminate wrong answers

Option A is wrong because enabling GZIP compression reduces data size but does not affect the buffer interval or delivery frequency; it may even increase latency due to compression overhead. Option B is wrong because increasing the buffer size to 128 MB would allow more data to accumulate before delivery, which would increase latency during spikes, not decrease it. Option C is wrong because switching to Kinesis Data Streams with a Lambda consumer introduces additional complexity and potential for increased latency due to Lambda invocation overhead and scaling limitations, and does not directly guarantee sub-60-second delivery to S3.

Practice this question →

549

MCQmedium

A company uses S3 as a data lake. They want to ingest on-premises relational database data daily with full-load snapshots. The data volume is 500 GB per day. The database is accessible over the internet. Which service should they use for this ingestion?

A.Kinesis Data Firehose

B.AWS Glue ETL job reading from JDBC

C.AWS Database Migration Service (DMS)

D.AWS Transfer Family

AnswerC

DMS supports full-load migration from on-premises databases to S3.

Why this answer

AWS Database Migration Service (DMS) can perform full-load migrations from on-premises databases to S3, making it ideal for daily snapshots of 500 GB. Option A is wrong because Kinesis Data Firehose is designed for streaming data, not for periodic database snapshots. Option B is wrong because while AWS Glue ETL can read from JDBC, it is not optimized for large full-load snapshots and lacks built-in replication capabilities.

Option D is wrong because AWS Transfer Family is for file transfers over SFTP, FTPS, or FTP, not for direct database connections.

Practice this question →

550

Multi-Selecthard

A company runs a real-time analytics platform using Amazon Kinesis Data Streams. The data is consumed by multiple consumers: one for real-time dashboard (using Lambda) and one for long-term storage (using Firehose to S3). The Kinesis stream has 10 shards. Each record is 1 KB, and the total incoming data rate is 5 MB/s. The Lambda consumer is falling behind and processing latency exceeds 10 seconds. Which TWO actions should be taken to resolve the issue?

Select 2 answers

A.Increase the Lambda function's memory allocation

B.Increase the number of shards to 20

C.Enable enhanced fan-out for the Lambda consumer

D.Switch to using Kinesis Client Library (KCL) instead of Lambda

E.Decrease the batch size in the Lambda event source mapping

AnswersB, C

More shards increase the total throughput of the stream, allowing Lambda to process more data in parallel.

Why this answer

Increasing the number of shards from 10 to 20 doubles the stream's read capacity, allowing the Lambda consumer to poll more data per second and reduce backlog. Option C is correct because enabling enhanced fan-out provides each consumer with a dedicated 2 MB/s read throughput per shard, eliminating contention between the Lambda consumer and the Firehose consumer, which is critical when multiple consumers read from the same stream.

Exam trap

The trap here is that candidates often assume increasing Lambda resources (memory) or reducing batch size will fix processing lag, when the root cause is a shared read throughput bottleneck between multiple consumers on the same Kinesis stream.

Practice this question →

551

MCQmedium

Refer to the exhibit. An IAM policy is attached to a role used by an AWS Glue job. The job fails with an 'AccessDenied' error when trying to write to 's3://my-bucket/output/'. What is the most likely cause?

A.The resource ARN for S3 should include the bucket itself.

B.The glue:StartJobRun action is not allowed.

C.The policy does not grant s3:ListBucket permission.

D.The s3:GetObject action is missing.

AnswerA

This option indicates that the resource ARN for S3 should include the bucket itself. If the policy has s3:PutObject allowed on the bucket ARN (e.g., 'arn:aws:s3:::my-bucket') rather than on the object ARN (e.g., 'arn:aws:s3:::my-bucket/*'), the PutObject action will fail. Therefore, this is the most likely cause of the AccessDenied error.

Why this answer

The job fails with 'AccessDenied' when writing to S3. This is most likely because the IAM policy attached to the Glue role does not grant the s3:PutObject permission on the object ARN (e.g., 'arn:aws:s3:::my-bucket/output/*'). Option A points out that the resource ARN should include the bucket itself, which is a common error: if the policy uses the bucket ARN (without the /* suffix) for the PutObject action, it will not allow the write operation.

Option C is incorrect because s3:ListBucket is not required for the PutObject action; it is needed for listing contents, not for writing. Option B is irrelevant to the S3 write failure. Option D (s3:GetObject) is for reading, not writing.

Practice this question →

552

MCQeasy

A company has CSV files in an S3 bucket that need to be converted to Parquet and loaded into a Redshift table daily. The transformation is a simple schema mapping without joins. Which AWS Glue feature is BEST suited for this task?

A.AWS Glue ETL job

B.AWS Glue DataBrew

C.AWS Glue Workflow

D.AWS Glue Crawler

AnswerA

Glue ETL jobs can read CSV, convert to Parquet, and write to Redshift.

Why this answer

(AWS Glue ETL job) is the best suited because it can convert CSV to Parquet and load into Redshift daily. Option B (DataBrew) is a visual data preparation tool, not ideal for automated daily jobs. Option C (Workflow) orchestrates multiple jobs but does not perform transformation.

Option D (Crawler) only discovers schema and catalogs data, not transform.

Practice this question →

553

MCQmedium

A company uses AWS Glue to process data in Amazon S3. The Glue job fails with an error indicating that the partition keys in the catalog do not match the actual S3 partition structure. What is the most likely cause?

A.The IAM role does not have permissions to read the S3 data

B.The data files are encrypted with SSE-KMS

C.The table name in the catalog is different from the one used in the job

D.The Glue Data Catalog partition metadata is outdated after the S3 structure changed

AnswerD

The catalog must be refreshed by running a crawler to reflect S3 changes.

Why this answer

The Glue Data Catalog stores partition metadata separately from the actual S3 partition layout. When the S3 partition structure changes (e.g., new partitions are added or existing ones are renamed) without updating the catalog, the Glue job reads stale partition metadata, leading to a mismatch error. The job fails because it expects partitions based on the catalog, not the live S3 structure.

Exam trap

The trap here is that candidates confuse a partition metadata mismatch with other common Glue errors like IAM permissions or encryption issues, but the error message explicitly references partition keys, not access or decryption problems.

How to eliminate wrong answers

Option A is wrong because an IAM permissions issue would typically cause an Access Denied error, not a partition key mismatch error. Option B is wrong because SSE-KMS encryption affects data decryption, not the structure or metadata of partitions in the catalog. Option C is wrong because a table name mismatch would cause a 'table not found' error, not a partition key mismatch; the error specifically points to partition keys, not table identifiers.

Practice this question →

554

MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver data to an S3 bucket. The data must be delivered within 60 seconds of ingestion. Currently, the delivery takes 3 minutes due to large buffer sizes. How should the engineer adjust the Firehose configuration?

A.Decrease the buffer interval to 60 seconds.

B.Increase the buffer interval to 120 seconds.

C.Increase the buffer size to 128 MB.

D.Decrease the buffer size to 1 MB.

AnswerA

Lowering the buffer interval triggers delivery sooner, meeting the latency requirement.

Why this answer

Amazon Kinesis Data Firehose delivers data to S3 based on either a buffer size threshold or a buffer interval (in seconds), whichever is reached first. To ensure delivery within 60 seconds, you must decrease the buffer interval to 60 seconds, which forces Firehose to flush data to S3 every 60 seconds regardless of buffer size. The current 3-minute delay is caused by the buffer interval being larger than 60 seconds, so reducing it directly meets the requirement.

Exam trap

The trap here is that candidates mistakenly think decreasing the buffer size alone will speed up delivery, but without adjusting the buffer interval, Firehose may still wait up to the default interval (e.g., 300 seconds) before flushing, so both parameters must be considered to meet a time-based requirement.

How to eliminate wrong answers

Option B is wrong because increasing the buffer interval to 120 seconds would make the delivery delay even longer (up to 2 minutes), not shorter, and fails to meet the 60-second requirement. Option C is wrong because increasing the buffer size to 128 MB does not reduce delivery time; it may actually increase latency since Firehose waits for more data to accumulate before flushing, and the buffer interval is the primary control for time-based delivery. Option D is wrong because decreasing the buffer size to 1 MB could cause more frequent flushes but does not guarantee delivery within 60 seconds if the buffer interval remains larger than 60 seconds; the buffer interval must be explicitly set to 60 seconds to enforce the time constraint.

Practice this question →

555

Multi-Selectmedium

A data engineer is designing a data pipeline that uses AWS Glue to transform data stored in Amazon S3. The transformation logic must be written in Python and should handle schema evolution automatically. Which THREE features or configurations should the engineer use? (Select THREE.)

Select 3 answers

A.Schedule a Glue crawler to update the schema

B.Use `applyMapping` transformations

C.Use Spark SQL for transformations

D.Enable schema detection in the Glue job

E.Use DynamicFrames instead of DataFrames

AnswersB, D, E

Facilitates schema manipulation.

Why this answer

Correct options: B, D, E. AWS Glue DynamicFrames (E) handle schema evolution automatically by allowing schema on read and accommodating changes in data structure. Schema detection in the Glue job (D) enables the job to infer the schema from the data, which is essential for handling evolving schemas.

Using `applyMapping` (B) provides explicit control over schema transformations and can be combined with DynamicFrames to manage schema changes. Option A (scheduling a Glue crawler) is meant for updating the Data Catalog, not for within-job schema evolution. Option C (Spark SQL) does not inherently handle schema evolution; it relies on static schemas.

Practice this question →

556

MCQeasy

A logistics company uses AWS Glue to process GPS data from delivery trucks. The data is stored in Amazon S3 as JSON files. The Glue job reads the JSON files, converts them to Parquet, and writes them back to S3. The company notices that the Glue job takes too long to complete. The data engineer wants to improve the job's performance without changing the code. What should the data engineer do?

A.Increase the number of DPUs to 20.

B.Change the worker type to G.2X.

C.Change the worker type to G.1X.

D.Decrease the number of DPUs to 5 to reduce overhead.

AnswerB

G.2X workers have double the memory and compute, accelerating the transformation.

Why this answer

Changing the worker type to G.2X provides more memory and CPU per worker, which improves performance for memory-intensive tasks like converting JSON to Parquet. Option A is wrong because increasing the number of DPUs can help with parallelism but may still be limited by per-worker memory; upgrading worker type is more efficient. Option C is wrong because G.1X is the default and provides less resources than G.2X.

Option D is wrong because decreasing DPUs would reduce parallelism and worsen performance.

Practice this question →

557

MCQhard

A company is using AWS DMS to replicate data from an on-premises Oracle database to Amazon RDS for MySQL. The replication is working, but the target table has a different schema. Which DMS feature should be used to transform the source schema to match the target?

A.Use AWS Schema Conversion Tool (SCT)

B.Use AWS Glue ETL jobs

C.Use DMS transformation rules

D.Use AWS Lambda triggers

AnswerC

DMS transformation rules allow renaming tables, schemas, and columns during replication.

Why this answer

AWS DMS transformation rules allow you to modify the schema, table, or column names and data types during the migration process. This feature is specifically designed to handle schema transformations within the DMS task itself, enabling you to map the source Oracle schema to the target MySQL schema without external tools or services.

Exam trap

The trap here is that candidates often confuse AWS Schema Conversion Tool (SCT) with DMS transformation rules, assuming SCT handles runtime schema mapping, whereas SCT is a separate pre-migration assessment and conversion tool, not a DMS feature for ongoing replication transformations.

How to eliminate wrong answers

Option A is wrong because AWS Schema Conversion Tool (SCT) is used for heterogeneous database migrations to convert the entire database schema and code objects, but it is not a feature of DMS for runtime schema transformation during ongoing replication. Option B is wrong because AWS Glue ETL jobs are for batch data processing and transformation in a data lake or warehouse, not for real-time schema mapping within a DMS replication task. Option D is wrong because AWS Lambda triggers can be used for custom post-processing or validation, but they are not a built-in DMS feature for transforming source schemas to match target schemas during replication.

Practice this question →

558

MCQmedium

A company uses AWS Database Migration Service (DMS) to continuously replicate data from an Oracle RDS instance to S3. The data is used for analytics. The replication lags behind the source by several hours. Which change would most likely reduce the lag?

A.Change the target endpoint from S3 to Kinesis Data Firehose.

B.Increase the source RDS instance storage to improve I/O.

C.Use a larger DMS replication instance (e.g., dms.c5.large instead of dms.t3.medium).

D.Change the target data format from CSV to Parquet.

AnswerC

More compute resources reduce lag.

Why this answer

The replication lag is most likely caused by the DMS replication instance being undersized for the volume of change data capture (CDC) events. Upgrading from a burstable t3.medium to a compute-optimized c5.large instance provides more consistent CPU performance and higher network throughput, enabling faster processing of Oracle redo logs and reducing the lag between source and target.

Exam trap

The trap here is that candidates assume the lag is caused by the target (S3 write performance or format) or source database I/O, rather than recognizing that DMS replication instance sizing is the primary bottleneck for CDC throughput.

How to eliminate wrong answers

Option A is wrong because changing the target to Kinesis Data Firehose does not address the bottleneck in the DMS replication instance's ability to capture and apply changes; Firehose is a delivery stream that still requires DMS to push data, and the lag originates from DMS processing capacity, not the target endpoint type. Option B is wrong because increasing source RDS storage improves I/O for the database itself, but DMS reads Oracle redo logs via LogMiner or binary reader, which are not significantly throttled by source storage I/O in a CDC scenario; the lag is due to DMS processing speed, not source I/O. Option D is wrong because changing the data format from CSV to Parquet reduces the target storage size and can improve query performance, but it does not affect the rate at which DMS captures and replicates changes from the source; the lag is a replication throughput issue, not a format conversion issue.

Practice this question →

559

MCQhard

A company uses Amazon EMR to process large datasets stored in Amazon S3. The data is in Parquet format and partitioned by date. The EMR cluster uses Spark SQL for transformations. Recently, the job has been slow and some tasks are failing due to 'java.lang.OutOfMemoryError'. The cluster has 10 core nodes of type m5.xlarge. Which configuration change would MOST improve performance and stability?

A.Increase the number of Spark partitions using repartition(), but keep the same nodes.

B.Change the core node instance type to r5.xlarge (memory-optimized).

C.Increase the number of executor cores in the Spark configuration.

D.Enable Kryo serialization in the Spark configuration.

AnswerB

More memory per node helps OOM.

Why this answer

The error 'java.lang.OutOfMemoryError' indicates that the Spark executors are running out of memory during processing. The m5.xlarge instance type provides 16 GiB of memory, but the workload likely requires more memory per task. Switching to r5.xlarge (32 GiB of memory) doubles the available memory per node, reducing memory pressure and preventing task failures, which directly improves stability and performance for memory-intensive transformations.

Exam trap

The trap here is that candidates often focus on tuning Spark configurations (partitions, cores, serialization) to fix OutOfMemoryErrors, but the real issue is insufficient physical memory per node, which requires a change in instance family rather than software settings.

How to eliminate wrong answers

Option A is wrong because increasing the number of partitions with repartition() can actually increase memory overhead due to shuffle operations and does not address the root cause of insufficient memory per executor; it may even worsen the OutOfMemoryError by creating more tasks that compete for the same limited memory. Option C is wrong because increasing the number of executor cores without increasing memory per core will cause more concurrent tasks to share the same fixed heap, exacerbating memory contention and making OutOfMemoryErrors more likely. Option D is wrong because enabling Kryo serialization reduces the size of serialized objects and improves CPU efficiency, but it does not increase the available heap memory; it cannot prevent OutOfMemoryErrors caused by insufficient memory for data processing.

Practice this question →

560

MCQhard

A logistics company ingests GPS tracking data from thousands of vehicles into Amazon S3 via AWS Direct Connect. Each vehicle sends a message every 5 seconds, resulting in about 200,000 messages per second. Each message is about 200 bytes. The company uses AWS Glue to transform the data into a parquet format and load it into Amazon Redshift for real-time analytics. However, the Glue jobs are failing due to memory issues and the data is not being loaded into Redshift quickly enough. The company needs to reduce the latency of data availability in Redshift. Which action should the data engineer take?

A.Use Amazon Kinesis Data Analytics to process the data in real-time and write to Redshift directly.

B.Increase the size of the Redshift cluster to improve load performance.

C.Use Amazon Kinesis Data Firehose to ingest the data directly into S3 and then use Redshift Spectrum to query the data without loading.

D.Increase the number of DPUs and allocate more memory to the Glue job.

AnswerC

Firehose can handle high throughput and Redshift Spectrum reduces load time.

Why this answer

Amazon Kinesis Data Firehose can ingest high-throughput streaming data (200,000 messages/sec) and deliver it to S3 in near-real-time (typically under 60 seconds). By using Redshift Spectrum to query the data directly in S3, the company avoids the latency and memory issues associated with AWS Glue batch transformations and Redshift bulk loads. This approach reduces data availability latency significantly.

Option A is incorrect because Amazon Kinesis Data Analytics adds processing overhead and does not directly solve the Glue memory issue or reduce latency to Redshift; it is more suitable for real-time streaming analytics, not for minimizing data ingestion latency.

Option B is incorrect because increasing the Redshift cluster size improves query performance and load speed but does not address the root cause: the Glue jobs are failing due to memory issues, and the data is not being transformed quickly enough. The bottleneck is upstream of Redshift.

Option D is incorrect because increasing DPUs and memory for the Glue job might resolve memory issues but does not significantly reduce latency; Glue batch processing still incurs minutes of delay, whereas Firehose provides near-real-time delivery.

Practice this question →

561

MCQhard

A company runs a data pipeline using AWS Glue ETL jobs to process daily files from an S3 bucket. The files are in CSV format and range from 1 GB to 10 GB. The Glue job runs successfully for small files but fails with an 'Out of Memory' error for files larger than 5 GB. The job uses a single G.1X DPU (16 GB memory). The company needs to process these large files without changing the existing ETL script. Which solution should the company implement?

A.Convert the input files from CSV to Parquet format to reduce memory usage.

B.Use the Optimus format in AWS Glue to compress data.

C.Use Amazon EMR with Spark instead of AWS Glue.

D.Increase the number of DPUs and use the G.2X worker type to provide more memory per worker.

AnswerD

More DPUs and G.2X provide additional memory.

Why this answer

Increasing the number of DPUs and switching to the G.2X worker type allocates more memory per worker (32 GB instead of 16 GB), allowing the Glue job to process larger CSV files without modifying the ETL script. Option A is incorrect because converting to Parquet would require changing the script and may still encounter memory limits with very large files. Option B is incorrect because Optimus format is not a standard AWS Glue feature; the correct approach is to increase memory.

Option C is incorrect because moving to Amazon EMR with Spark would likely require rewriting the script, which the company wants to avoid.

Practice this question →

562

MCQhard

Refer to the exhibit. An IAM policy is attached to an AWS Glue ETL job. The job reads from the Kinesis stream 'input-stream' and writes to S3 bucket 'data-lake-bucket'. The job fails with an access denied error. Which missing permission is most likely the cause?

A.kinesis:PutRecord permission on a wildcard stream ARN

B.kinesis:DescribeStream permission

C.s3:PutObject permission on a specific prefix

D.s3:ListBucket permission on the bucket

AnswerD

Glue needs ListBucket to verify bucket existence and structure.

Why this answer

The AWS Glue ETL job fails with an access denied error because it lacks the s3:ListBucket permission on the 'data-lake-bucket'. When writing to S3, the job must first list the bucket to verify the target prefix exists and to handle multipart uploads; without this permission, the write operation fails even if s3:PutObject is granted.

Exam trap

The trap here is that candidates assume only s3:PutObject is needed for writing to S3, forgetting that AWS S3 operations like multipart uploads and prefix validation require the s3:ListBucket permission on the bucket resource.

How to eliminate wrong answers

Option A is wrong because the job reads from the Kinesis stream, so it needs kinesis:GetRecords or kinesis:SubscribeToShard, not kinesis:PutRecord, and a wildcard stream ARN would be overly permissive but not the missing permission. Option B is wrong because kinesis:DescribeStream is used for stream metadata retrieval, but the error occurs during the S3 write phase, not during Kinesis consumption. Option C is wrong because while s3:PutObject on a specific prefix is necessary for writing objects, the missing permission is the prerequisite s3:ListBucket action on the bucket itself, which is required to validate the target location before any PutObject call.

Practice this question →

563

MCQmedium

A data engineering team uses AWS Glue ETL jobs to process data daily. They notice that job run times are increasing as data volume grows. Which action will most effectively improve performance without changing the code?

A.Use a smaller instance type for the Glue job.

B.Enable job bookmark to skip previously processed data.

C.Split the data into more files in S3.

D.Increase the number of DPUs for the Glue job.

AnswerD

More DPUs increase parallelism and can significantly reduce run time.

Why this answer

Increasing the number of DPUs (Data Processing Units) allocated to the job provides more parallelism and memory, speeding up processing without code changes.

Practice this question →

564

MCQmedium

A company is streaming clickstream data from a website into Amazon Kinesis Data Streams. The data is then consumed by a Lambda function that transforms the records and writes them to an S3 bucket in Parquet format. Recently, the Lambda function has been timing out and the S3 bucket is not receiving all expected records. The Kinesis stream has a shard count of 10 and the Lambda function's reserved concurrency is set to the default. Which change would MOST likely resolve the issue?

A.Decrease the batch window from the default 300 seconds to 60 seconds.

B.Configure the Kinesis stream to directly write to S3 using a delivery stream.

C.Increase the Lambda function's reserved concurrency.

D.Increase the batch size from the default 100 to 500 records per invocation.

AnswerA

Correct: Decreasing the batch window reduces the number of records per invocation, which helps the Lambda function complete within its timeout.

Why this answer

The Lambda function is timing out because it cannot process the default batch of 100 records within the function timeout. Decreasing the batch window from 300 seconds to 60 seconds causes Lambda to invoke more frequently with smaller batches, reducing the number of records per invocation. This lowers the processing time per invocation, helping the function complete before the timeout.

Increasing the batch size (Option D) would worsen the issue by adding more records per invocation. Increasing reserved concurrency (Option C) does not reduce per-invocation processing time. Using a delivery stream (Option B) changes the architecture unnecessarily and may not preserve the transformation logic.

Exam trap

Candidates often think that increasing batch size or concurrency will improve throughput, but when functions are timing out, reducing the batch size (or batch window) is the correct fix. Increasing concurrency does not help per-invocation timeouts.

How to eliminate wrong answers

Option A is wrong because decreasing the batch window from 300 seconds to 60 seconds would cause more frequent invocations, increasing the likelihood of timeouts and not addressing the root cause. Option B is wrong because configuring a Kinesis Delivery Stream to write directly to S3 bypasses the Lambda transformation, which is required for converting records to Parquet format. Option C is wrong because increasing reserved concurrency would allow more concurrent invocations but does not reduce the processing load per invocation, so timeouts would persist.

Practice this question →

565

Multi-Selecthard

A company is running a critical application that generates millions of small JSON files every hour in an S3 bucket. A data engineer needs to process these files in near real-time using AWS Glue. The engineer wants to minimize the latency between file arrival and Glue job start. Which TWO actions should the engineer take?

Select 2 answers

A.Increase the Glue job's batch window to 600 seconds.

B.Increase the number of DPUs for the Glue job to accelerate processing.

C.Pre-process the files to consolidate them into larger files before the Glue job runs.

D.Use Amazon S3 event notifications to trigger an AWS Lambda function that starts the Glue job upon file arrival.

AnswersC, D

Fewer larger files reduce Glue job overhead and improve throughput.

Why this answer

Consolidating millions of small JSON files into larger files reduces the overhead of S3 LIST operations and minimizes the number of partitions Glue must scan. This directly lowers the latency between file arrival and job start, as Glue jobs are more efficient when processing fewer, larger files rather than many small files. Option D is correct because S3 event notifications can trigger a Lambda function that immediately starts the Glue job upon file arrival, enabling near real-time processing without polling or scheduled delays.

Exam trap

The trap here is confusing job startup latency with job execution speed — candidates often choose DPU increases (Option B) thinking they reduce latency, but DPUs only affect processing speed after the job starts, not the time to initiate the job.

Practice this question →

566

MCQeasy

A company receives streaming clickstream data from its website. The data must be ingested with low latency and transformed in real time before being stored in Amazon S3. Which AWS service combination is most suitable for this use case?

A.Amazon S3 with S3 Object Lambda

B.Amazon Kinesis Data Streams with Amazon Kinesis Data Analytics

C.Amazon Kinesis Data Firehose with AWS Lambda for transformation

D.AWS Glue jobs triggered by Amazon S3 events

AnswerB

Kinesis Data Streams provides low-latency ingestion and Kinesis Data Analytics enables real-time transformations.

Why this answer

Amazon Kinesis Data Streams ingests streaming clickstream data with low latency, and Amazon Kinesis Data Analytics performs real-time transformations using SQL or Apache Flink. The processed data can then be stored in Amazon S3 via a Kinesis Data Firehose delivery stream, meeting the requirement for low-latency ingestion and real-time transformation.

Exam trap

The trap here is that candidates confuse Kinesis Data Firehose (which is near-real-time with a 60-second minimum buffer) with true low-latency streaming, leading them to select Option C despite the explicit 'low latency' requirement in the question.

How to eliminate wrong answers

Option A is wrong because Amazon S3 with S3 Object Lambda applies transformations only when objects are retrieved, not during ingestion, and cannot handle real-time streaming data with low latency. Option C is wrong because Amazon Kinesis Data Firehose is a near-real-time service with a minimum buffer interval of 60 seconds, which does not meet the low-latency requirement for streaming ingestion. Option D is wrong because AWS Glue jobs triggered by Amazon S3 events are batch-oriented and incur significant startup latency (often minutes), making them unsuitable for real-time transformation of streaming data.

Practice this question →

567

MCQeasy

A company uses AWS Glue to run ETL jobs daily. The data is stored in S3 as Parquet files partitioned by date. Recently, jobs have failed with the error 'No such file or directory' for certain partitions. What is the MOST likely cause?

A.The schema has changed and Glue cannot parse the data.

B.A partition folder was deleted or not created by the upstream process.

C.The files are compressed with an unsupported codec.

D.The IAM role does not have s3:GetObject permission.

AnswerB

Missing partition leads to 'No such file or directory'.

Why this answer

The error 'No such file or directory' indicates that the Glue ETL job is attempting to read a specific S3 partition path that does not exist. Since the data is partitioned by date and the job runs daily, the most likely cause is that the upstream process failed to create or accidentally deleted the partition folder for that date. Glue's dynamic frame or Spark DataFrame will throw this error when it tries to list or read files from a missing prefix.

Exam trap

The trap here is that candidates confuse file-level permission errors (Option D) with missing directory errors, but S3 returns distinct HTTP status codes (403 vs 404) that map to different error messages in Spark/Glue.

How to eliminate wrong answers

Option A is wrong because a schema change would typically cause a parsing or schema mismatch error (e.g., 'Schema mismatch' or 'Cannot convert type'), not a 'No such file or directory' error. Option C is wrong because unsupported compression codecs (e.g., LZO without proper libraries) would cause a 'Codec not found' or 'Compression error', not a missing file error. Option D is wrong because missing s3:GetObject permission would result in an 'Access Denied' (403) error, not a 'No such file or directory' error.

Practice this question →

568

Multi-Selectmedium

A data engineer is designing a data ingestion pipeline for real-time clickstream data from a website. The data must be ingested with low latency (seconds) and made available for multiple consumer applications, including a dashboard that refreshes every minute and a machine learning model that processes data in near-real-time. The engineer needs to choose a streaming ingestion service. Which TWO services meet these requirements? (Select TWO.)

Select 2 answers

A.Amazon Kinesis Data Firehose

B.Amazon Managed Streaming for Apache Kafka (Amazon MSK)

C.Amazon Kinesis Data Streams

D.Amazon Simple Queue Service (SQS)

E.Amazon S3

AnswersB, C

MSK is a fully managed Kafka service that provides low-latency streaming and supports multiple consumer groups.

Why this answer

Amazon Kinesis Data Streams (C) provides sub-second ingestion latency and supports multiple consumer applications via its enhanced fan-out feature, enabling a dashboard and ML model to consume data concurrently with low latency. Amazon MSK (B) offers similar real-time capabilities with Apache Kafka's native pub/sub model, allowing multiple consumers to process the same stream independently and with low latency, meeting the near-real-time requirements.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Firehose with Kinesis Data Streams, assuming Firehose provides real-time ingestion, but Firehose buffers data for at least 60 seconds before delivery, making it unsuitable for sub-second latency requirements.

Practice this question →

569

MCQhard

A data engineer is designing a streaming pipeline that ingests data from an Amazon Kinesis Data Stream (with 5 shards) into Amazon S3. The data must be transformed using a complex stateful operation that cannot be done in a Lambda function (limited to 15 minutes). The engineer needs a solution that can maintain state across multiple records. Which service should be used?

A.Amazon EMR running Spark Structured Streaming

B.Amazon Kinesis Data Firehose with Lambda transformation

C.AWS Glue streaming ETL job

D.Amazon Kinesis Data Analytics for Apache Flink

AnswerD

Flink supports stateful stream processing, exactly what is needed.

Why this answer

Amazon Kinesis Data Analytics for Apache Flink is the correct choice because it supports stateful stream processing with exactly-once semantics, can maintain state across multiple records, and has no 15-minute execution limit like AWS Lambda. It natively integrates with Kinesis Data Streams and can sink transformed data to S3, meeting all requirements for complex stateful operations.

Exam trap

The trap here is that candidates often confuse AWS Glue streaming ETL (which is Spark-based and better for batch-oriented transformations) with a true stateful streaming engine, or they assume Kinesis Data Firehose can handle stateful logic via Lambda, not realizing Lambda's stateless nature and timeout limit.

How to eliminate wrong answers

Option A is wrong because Amazon EMR running Spark Structured Streaming, while capable of stateful processing, is overkill for a single streaming pipeline and requires managing a cluster, which adds operational overhead not needed when simpler managed services exist. Option B is wrong because Amazon Kinesis Data Firehose with Lambda transformation cannot handle stateful operations—Lambda has a 15-minute timeout and is stateless by design, making it unsuitable for maintaining state across multiple records. Option C is wrong because AWS Glue streaming ETL jobs are based on Spark and are designed for batch-oriented transformations, not for complex stateful operations that require persistent state across records in a low-latency streaming context.

Practice this question →

570

MCQhard

A financial services company ingests real-time stock trade data from multiple exchanges into Amazon Kinesis Data Streams. Each trade record is a JSON object with fields: trade_id, symbol, price, quantity, timestamp. The stream has 5 shards. The data is consumed by an AWS Lambda function that aggregates trades per symbol every minute and writes the results to an Amazon DynamoDB table for a real-time dashboard. Recently, the dashboard has been showing outdated data, and the Lambda function is experiencing high error rates. The CloudWatch logs show 'ProvisionedThroughputExceededException' errors from DynamoDB. The DynamoDB table has 10 read capacity units (RCU) and 10 write capacity units (WCU). The average trade volume is 5,000 trades per second across all symbols, and there are 100 symbols. The Lambda function is configured with a batch size of 100 and a 1-minute window. The data volume is expected to double in the next month. As a data engineer, what is the most appropriate course of action?

A.Switch the storage from DynamoDB to Amazon S3 and use Amazon Athena for the dashboard

B.Increase the number of Kinesis shards to 10 to increase Lambda concurrency

C.Increase the DynamoDB write capacity units to 100 and enable auto scaling

D.Use Amazon Kinesis Data Firehose to deliver data to S3 and use Amazon QuickSight for the dashboard

AnswerC

Correct. The DynamoDB table is throttling writes; increasing WCU to 100 and enabling auto scaling resolves the current issue and accommodates future growth.

Why this answer

The DynamoDB table is throttling due to insufficient write capacity. With 5,000 trades/s and updating per symbol per minute, the write rate is about 100 writes per minute (one per symbol), but the aggregation may cause bursts. However, the 'ProvisionedThroughputExceededException' indicates WCU is too low.

Increasing WCU to 100 resolves the immediate issue; auto scaling handles future growth. Option A (switch to S3 and Athena) changes the architecture and loses real-time capabilities. Option B (increase shards) addresses Lambda concurrency but not DynamoDB throttling.

Option D (use Firehose and QuickSight) is for delivery to S3, not real-time dashboard.

Practice this question →

571

MCQhard

A company ingests streaming data from social media APIs into Kinesis Data Streams. Each record is approximately 5 KB. The data must be enriched with geolocation information from a DynamoDB table before being stored in S3. The enrichment process takes about 200 ms per record. Which architecture minimizes latency and cost?

A.Use an EC2 instance running a custom application to consume from Kinesis, enrich, and write to S3

B.Use AWS Glue ETL jobs running continuously on the stream

C.Use Kinesis Data Analytics to perform enrichment with SQL

D.Use Kinesis Data Firehose with a Lambda function that queries DynamoDB

AnswerD

Firehose with Lambda can perform enrichment per record.

Why this answer

Kinesis Data Firehose can invoke a Lambda function for per-record enrichment, providing automatic scaling and low operational overhead. Option A is wrong because an EC2 instance adds significant operational overhead and requires manual scaling, increasing complexity and cost. Option B is wrong because AWS Glue ETL jobs are optimized for batch processing, not continuous streaming with low latency per record.

Option C is wrong because Kinesis Data Analytics (SQL) is designed for real-time analytics on streaming data, not for per-record enrichment involving external lookups like DynamoDB.

Practice this question →

572

MCQmedium

A company uses AWS Lambda to process records from an Amazon Kinesis Data Stream. Each record is about 50 KB. The Lambda function transforms the data and writes to Amazon DynamoDB. Recently, the Lambda function has been experiencing throttling and high error rates. The Kinesis stream has 10 shards. What is the most cost-effective solution to improve processing throughput?

A.Increase the number of shards in the Kinesis stream.

B.Increase the Parallelization Factor for the Lambda event source mapping.

C.Increase the memory allocated to the Lambda function.

D.Increase the Batch Window (MaximumBatchingWindowInSeconds) for the event source mapping.

AnswerD

Reduces invocation frequency.

Why this answer

Increasing the Batch Window (MaximumBatchingWindowInSeconds) allows the Lambda function to accumulate more records from the Kinesis stream before invoking the function, reducing the number of invocations and thus lowering the chance of throttling. This is the most cost-effective solution as it does not require additional shards, memory, or parallelization, and directly addresses the high error rates caused by excessive concurrent executions.

Exam trap

The trap here is that candidates often assume increasing shards or parallelization is the only way to improve throughput, but the question specifically asks for the most cost-effective solution, and increasing the batch window reduces invocation count without incurring additional costs.

How to eliminate wrong answers

Option A is wrong because increasing the number of shards would increase the number of concurrent Lambda invocations, potentially worsening throttling and increasing costs, not improving throughput cost-effectively. Option B is wrong because increasing the Parallelization Factor (which controls concurrent batches per shard) would also increase concurrency, leading to more throttling and higher costs, and is not a cost-effective fix. Option C is wrong because increasing memory allocated to the Lambda function does not directly address throttling or error rates caused by invocation frequency; it may improve per-invocation performance but at a higher cost without solving the root cause.

Practice this question →

573

MCQeasy

Refer to the exhibit. A data engineer is configuring a Kinesis Data Firehose delivery stream. The stream is expected to receive bursts of 10 MB of data every 2 minutes. What is the maximum time it will take for data to be delivered to S3 during a burst?

A.300 seconds

B.60 seconds

C.1 second

D.600 seconds

AnswerA

The buffer interval is 300 seconds; even with bursts, the maximum time is the interval.

Why this answer

The Kinesis Data Firehose delivery stream is configured with a buffer interval of 300 seconds (5 minutes) and a buffer size of 5 MB. During a burst of 10 MB every 2 minutes, the buffer will fill to 5 MB in 1 minute, so the buffer will be flushed every minute due to reaching the size threshold. However, the question asks for the maximum time it will take for data to be delivered to S3 during a burst.

The maximum time is determined by the buffer interval, which is 300 seconds. Even though the size trigger may cause earlier flushes, the maximum possible time before delivery is the interval setting of 300 seconds. Therefore, the correct answer is 300 seconds (option A).

Practice this question →

574

MCQeasy

A data engineer is responsible for ingesting daily CSV files from an external partner into an Amazon S3 bucket. The partner uploads files to an AWS Transfer Family (SFTP) endpoint. Once a file is uploaded, an AWS Lambda function triggers an AWS Glue ETL job to transform the data and load it into an Amazon RDS database. Recently, some files have failed to trigger the Glue job because the Lambda function timed out while waiting for the Glue job to complete. The engineer needs to ensure that all files are processed reliably without manual intervention. What should the data engineer do?

A.Modify the Lambda function to send a message to an Amazon SQS queue after uploading, and create a separate Lambda function that reads from the queue and triggers the Glue job asynchronously.

B.Increase the Lambda function timeout to 15 minutes to accommodate longer Glue jobs.

C.Increase the Lambda function's reserved concurrency to allow multiple invocations.

D.Configure S3 event notifications to trigger the Glue job directly without Lambda.

AnswerA

Decoupling prevents timeout and ensures retries.

Why this answer

Decoupling the Lambda function from the Glue job by using SQS allows the Lambda function to submit the job and exit, while a second Lambda function monitors completion. Option B is wrong because increasing Lambda timeout still ties it to job duration. Option C is wrong because the issue is not concurrency.

Option D is wrong because increasing S3 events does not address the timeout.

Practice this question →

575

MCQeasy

A data engineer needs to ingest data from an Amazon S3 bucket into Amazon Redshift for analytics. The data is in CSV format and the Redshift table already exists. Which service can be used to perform this ingestion with minimal configuration?

A.AWS Glue

B.Amazon Kinesis Data Firehose

C.Amazon Redshift COPY command

D.AWS Database Migration Service (DMS)

AnswerC

The COPY command loads data from S3 into Redshift efficiently.

Why this answer

The Amazon Redshift COPY command is the most direct and minimal-configuration method to load data from an S3 bucket into an existing Redshift table. It is purpose-built for bulk data ingestion from S3, supports CSV format natively, and requires only the table name, S3 path, IAM role, and format options — no additional services or pipelines needed.

Exam trap

The trap here is that candidates often overthink and choose AWS Glue or Kinesis Firehose because they assume a managed service is always required, but the COPY command is the simplest and most efficient native tool for batch loading from S3 into an existing Redshift table.

How to eliminate wrong answers

Option A is wrong because AWS Glue is an ETL service that requires creating crawlers, jobs, and triggers, which adds unnecessary complexity for a simple S3-to-Redshift load; it is overkill when the COPY command suffices. Option B is wrong because Amazon Kinesis Data Firehose is designed for streaming data ingestion into Redshift, not for batch loading from static CSV files in S3, and it requires configuring a delivery stream, buffering, and transformation. Option D is wrong because AWS DMS is used for continuous database migration and replication between databases, not for one-time bulk loading of CSV files from S3 into an existing Redshift table.

Practice this question →

576

MCQeasy

A data engineer has an IAM policy attached to an IAM role used by an AWS Glue job. The Glue job needs to read from S3 bucket 'data-bucket' and write to the same bucket. The job fails with an access denied error when trying to write to S3. What is the issue?

A.The Glue job cannot assume the IAM role because of trust policy.

B.The policy does not include s3:ListBucket permission, which Glue may need.

C.The resource ARN for S3 is missing the bucket-level permission.

D.The actions for S3 are incorrect; s3:PutObject is not sufficient.

AnswerB

Glue may require ListBucket to navigate the bucket.

Why this answer

AWS Glue jobs require the `s3:ListBucket` permission on the bucket to perform operations like listing objects, even when reading and writing specific keys. Without this permission, the job fails with an access denied error when trying to write, as Glue internally uses `ListBucket` to verify bucket existence and access patterns. The policy attached to the IAM role likely includes `s3:GetObject` and `s3:PutObject` but omits the bucket-level `s3:ListBucket` action, which is necessary for the Glue job to interact with S3 successfully.

Exam trap

The trap here is that candidates often assume only object-level permissions (GetObject, PutObject) are needed for read/write operations, overlooking the bucket-level ListBucket permission that AWS Glue implicitly requires for its internal operations.

How to eliminate wrong answers

Option A is wrong because the trust policy controls which entities can assume the IAM role, not the permissions within S3; if the Glue job could assume the role (which it does, as it runs), the trust policy is not the issue. Option C is wrong because the resource ARN for S3 must include both bucket-level and object-level ARNs (e.g., `arn:aws:s3:::data-bucket` and `arn:aws:s3:::data-bucket/*`), but the question states the job fails when writing, implying the bucket-level permission (like `s3:ListBucket`) is missing, not that the ARN format is incorrect. Option D is wrong because `s3:PutObject` is sufficient for writing objects to S3 when combined with the necessary bucket-level permissions; the issue is not that `s3:PutObject` is insufficient, but that the required `s3:ListBucket` action is missing from the policy.

Practice this question →

577

MCQeasy

A startup is ingesting event data from a mobile app into an Amazon Kinesis Data Streams stream with 2 shards. Each shard can ingest up to 1 MB/s or 1000 records/s. The app sends about 800 records per second with an average record size of 1.5 KB. The data engineer notices that the stream is throttling some records, resulting in data loss. The engineer needs to ensure that all records are ingested without changing the application code. What should the data engineer do?

A.Reduce the average record size to below 1 KB by compressing data on the client side.

B.Switch from Kinesis Data Streams to Kinesis Data Firehose for ingestion.

C.Increase the number of shards in the Kinesis stream to 3.

D.Enable enhanced fan-out on the stream to provide dedicated throughput to each consumer.

AnswerC

Adding shards increases total ingestion capacity.

Why this answer

The current throughput is 800 records/s * 1.5 KB = 1.2 MB/s, which exceeds the per-shard limit of 1 MB/s. Adding a third shard increases the total write capacity to 3 MB/s, accommodating the current traffic. Option A is incorrect because reducing record size would require modifying the application code.

Option B is incorrect because Kinesis Data Firehose is not designed for real-time ingestion and does not address the shard capacity issue. Option D is incorrect because enhanced fan-out improves consumer read throughput, not producer write throughput.

Practice this question →

578

MCQmedium

A data engineer is building a data pipeline that ingests data from Amazon S3 into Amazon Redshift. The data is in CSV format and includes a timestamp column. The pipeline should load only new data incrementally. Which approach is most efficient?

A.Use the COPY command to load the entire bucket and rely on Redshift to deduplicate

B.Use the COPY command with a manifest file that lists only the new S3 objects

C.Use Amazon Redshift Spectrum to query the S3 data directly without loading

D.Use INSERT statements within a loop to load each new file

AnswerB

A manifest file allows incremental loading by specifying only new files.

Why this answer

Using the COPY command with a manifest file allows you to explicitly list only the new S3 objects to be loaded, enabling incremental loading without scanning or loading the entire bucket. This approach is efficient as it avoids the overhead of deduplication or full-bucket scans, and it leverages Redshift's native high-speed parallel ingestion from S3.

Exam trap

The trap here is that candidates may think Redshift Spectrum is a valid alternative for loading data, but Spectrum is designed for external querying, not for persistent loading into Redshift tables, which is the explicit requirement in the question.

How to eliminate wrong answers

Option A is wrong because loading the entire bucket and relying on Redshift to deduplicate is inefficient; Redshift does not have built-in deduplication logic for COPY commands, and loading all data repeatedly would waste storage and compute resources. Option C is wrong because Redshift Spectrum queries data directly in S3 without loading it into Redshift tables, which does not meet the requirement of loading data into Redshift for persistent storage and incremental processing. Option D is wrong because using INSERT statements within a loop to load each new file is far less efficient than the COPY command, as it lacks parallelization and high-throughput optimization, leading to poor performance for large datasets.

Practice this question →

579

MCQmedium

A data engineer is ingesting data from an Amazon RDS for PostgreSQL database into Amazon S3 using AWS Glue. The Glue job reads the entire table each time it runs, which takes several hours. The team wants to reduce the job duration by reading only new or updated records. Which approach should the engineer adopt?

A.Enable job bookmarks in AWS Glue and use a column with timestamps as the bookmark key to read only incremental data.

B.Partition the table in the source database by date and read only the latest partition.

C.Increase the number of Glue workers to improve parallel reads.

D.Use Amazon Kinesis Data Streams to capture changes from PostgreSQL.

AnswerA

Glue bookmarks track processed records; using a timestamp column allows incremental reads.

Why this answer

AWS Glue job bookmarks track previously processed data using a specified column (e.g., a timestamp column) as the bookmark key. When enabled, the job reads only new or updated records since the last run, significantly reducing job duration by avoiding full table scans. This directly addresses the requirement to read incremental data from the PostgreSQL source.

Exam trap

The trap here is that candidates may confuse increasing parallelism (Option C) with reducing data volume, or assume that source-side partitioning (Option B) automatically translates to incremental reads in Glue, when in fact Glue job bookmarks are the native mechanism for incremental processing in batch jobs.

How to eliminate wrong answers

Option B is wrong because partitioning the source PostgreSQL table by date does not inherently enable incremental reads in AWS Glue; Glue would still need to scan the entire table unless combined with a bookmark or filter, and partitioning alone does not reduce the data read by the Glue job. Option C is wrong because increasing the number of Glue workers improves parallelism but does not reduce the volume of data read; the job would still process the entire table, just faster, which does not address the core issue of reading only new/updated records. Option D is wrong because Amazon Kinesis Data Streams is designed for real-time streaming ingestion, not for batch incremental reads from a static PostgreSQL table; it would require additional infrastructure (e.g., AWS DMS or Debezium) to capture CDC events, which is overkill and not a direct solution for reducing batch job duration.

Practice this question →

580

MCQeasy

A company has a nightly batch job that processes 100 GB of data from an Amazon S3 bucket and loads it into an Amazon Redshift table. The job currently runs on an Amazon EMR cluster. Which service would reduce operational overhead while providing similar functionality?

A.AWS Database Migration Service

B.AWS Glue

C.Amazon Redshift Spectrum

D.Amazon Athena

AnswerB

Glue can run serverless ETL jobs on a schedule, reducing overhead.

Why this answer

AWS Glue is a serverless ETL service that can process 100 GB of data from S3 and load it into Redshift without managing any infrastructure. It provides built-in job scheduling, automatic retries, and a Spark-based engine that handles large-scale data transformations, directly replacing the EMR cluster's functionality while eliminating operational overhead.

Exam trap

The DEA-C01 exam often tests the distinction between query engines (Athena, Redshift Spectrum) and ETL services (Glue), where candidates mistakenly choose Athena or Spectrum because they can read from S3, but they lack the batch processing and data loading capabilities required for this use case.

How to eliminate wrong answers

Option A is wrong because AWS Database Migration Service (DMS) is designed for continuous database replication or one-time migrations between databases, not for batch processing and transforming large datasets from S3 into Redshift. Option C is wrong because Amazon Redshift Spectrum allows querying data directly in S3 without loading it into Redshift, but it does not perform ETL transformations or replace the batch job's processing logic. Option D is wrong because Amazon Athena is an interactive query service for ad-hoc analysis on S3 data, not a managed ETL service for scheduled batch processing and loading into Redshift.

Practice this question →

581

MCQeasy

A data engineer needs to ingest data from an on-premises Apache Kafka cluster into Amazon S3. The data engineer wants to minimize operational overhead and avoid managing any servers. Which AWS service should the data engineer use?

A.Amazon Managed Streaming for Apache Kafka (Amazon MSK)

B.AWS Glue

C.Amazon Kinesis Data Analytics

D.Amazon Kinesis Data Streams

AnswerA

Amazon MSK is a fully managed Apache Kafka service that can ingest data from on-premises Kafka via MirrorMaker, then sink to S3.

Why this answer

Amazon MSK is correct. It is a fully managed Apache Kafka service that can ingest data from on-premises Kafka via MirrorMaker or replication, then sink to S3, minimizing operational overhead. AWS Glue (B) is for ETL jobs, not real-time Kafka ingestion.

Kinesis Data Analytics (C) is for analyzing streaming data, not ingestion. Amazon Kinesis Data Streams (D) is a different service not directly compatible with Kafka without custom connectors.

Practice this question →

582

Multi-Selecteasy

Which TWO AWS services can be used to ingest streaming data into Amazon S3? (Choose two.)

Select 2 answers

A.Amazon S3 Transfer Acceleration

B.Amazon Managed Streaming for Apache Kafka (Amazon MSK)

C.Amazon Kinesis Data Firehose

D.Amazon Elastic Block Store (Amazon EBS)

E.AWS Snowball

AnswersB, C

MSK can stream data to S3 via Kafka Connect S3 sink.

Why this answer

Amazon Kinesis Data Firehose is the easiest way to reliably load streaming data into Amazon S3. It can capture, transform, and deliver streaming data to S3 destinations in near real-time with no code required. Amazon MSK (Managed Streaming for Apache Kafka) can also ingest streaming data into S3 by using Kafka Connect with an S3 sink connector, which writes data from Kafka topics directly to S3.

Exam trap

The trap here is that candidates often confuse Amazon S3 Transfer Acceleration (a speed optimization for existing uploads) with a streaming ingestion service, or they mistakenly think EBS or Snowball can handle real-time streaming data when they are designed for persistent block storage and offline bulk transfer, respectively.

Practice this question →

583

Multi-Selecteasy

A data engineer is designing a real-time streaming pipeline to ingest clickstream data from a website into Amazon S3. The data must be transformed before storage. Which TWO AWS services can be used together to build this pipeline? (Choose TWO.)

Select 2 answers

A.Amazon Kinesis Data Firehose

B.AWS Glue

C.Amazon Kinesis Data Streams

D.Amazon S3 Transfer Acceleration

E.AWS Database Migration Service (DMS)

AnswersA, C

Delivers streaming data to S3 with transformation capabilities.

Why this answer

The correct combination is Amazon Kinesis Data Streams to ingest the streaming clickstream data, and Amazon Kinesis Data Firehose to read from the stream, perform transformations (e.g., via Lambda), and deliver the transformed data to Amazon S3. Kinesis Data Streams provides the durable ingestion layer, while Kinesis Data Firehose handles the delivery and transformation without managing servers.

Exam trap

The trap here is that candidates often confuse Kinesis Data Streams (which requires custom consumers and does not natively write to S3) with Kinesis Data Firehose (which directly delivers to S3 and supports built-in transformations), leading them to select only Data Streams or miss the need for a transformation service.

Practice this question →

584

MCQmedium

A company uses AWS Glue ETL jobs to transform data from Amazon S3 to Amazon Redshift. The job reads JSON files, applies schema mapping, and writes to a Redshift table. Recently, the job started failing with memory errors. The data volume has increased tenfold. Which approach should a data engineer take to resolve this issue with minimal code changes?

A.Switch from Spark to Python Shell job type.

B.Implement batch processing with smaller file sizes.

C.Increase the number of DPUs allocated to the Glue job.

D.Use Redshift Spectrum to query data directly from S3.

AnswerC

Provides more resources for processing.

Why this answer

Increasing the number of DPUs (Data Processing Units) allocated to the AWS Glue job directly addresses the memory constraint caused by a tenfold increase in data volume. Glue ETL jobs run on Apache Spark, which distributes data processing across executors; more DPUs provide more memory and compute capacity, allowing the job to handle larger datasets without code changes.

Exam trap

The trap here is that candidates may assume memory errors always require code optimization (e.g., batching or partitioning), but the question explicitly asks for minimal code changes, making resource scaling the correct answer.

How to eliminate wrong answers

Option A is wrong because switching from Spark to Python Shell job type would reduce parallelism and memory capacity, as Python Shell runs on a single node with limited resources, making it unsuitable for large-scale data transformations. Option B is wrong because implementing batch processing with smaller file sizes would require significant code changes to split and manage files, contradicting the 'minimal code changes' requirement, and does not address the root cause of insufficient memory allocation. Option D is wrong because using Redshift Spectrum to query data directly from S3 bypasses the Glue ETL job entirely, which is a different architectural approach that does not resolve the memory error in the existing Glue job and may introduce new costs and complexity.

Practice this question →

585

Multi-Selecthard

A data engineer is building a data ingestion pipeline using AWS Glue. The source is an Amazon DynamoDB table, and the target is an Amazon S3 data lake in Parquet format. The pipeline must handle large volumes and ensure exactly-once processing. Which THREE features should the engineer use together to achieve this? (Choose THREE.)

Select 3 answers

A.Use Amazon Kinesis Data Streams to capture DynamoDB Streams changes.

B.Configure the Glue job to convert data to Parquet format.

C.Use Amazon S3 Object Lambda to transform data on the fly.

D.Enable job bookmarks in the Glue job to track processed items.

E.Use DynamoDB's export to S3 feature to get a full snapshot.

AnswersB, D, E

Parquet is columnar and efficient for analytics.

Why this answer

Converting data to Parquet format is a core requirement for an S3 data lake, as Parquet offers columnar storage, compression, and efficient querying via services like Amazon Athena and Amazon Redshift Spectrum. AWS Glue natively supports Parquet as an output format, enabling the engineer to specify it in the job's output schema or transformation logic.

Exam trap

The trap here is that candidates often confuse streaming services like Kinesis Data Streams with batch processing, assuming they are required for exactly-once guarantees, when in fact AWS Glue job bookmarks combined with DynamoDB Streams or export to S3 provide a simpler and more reliable solution.

Practice this question →

586

MCQmedium

A data engineer uses AWS Glue to process data from S3. The Glue job frequently fails with 'Out of Memory' errors. The job reads several large compressed files. What is the MOST effective way to resolve this issue without changing the code?

A.Increase the number of G.1X workers or use G.2X workers

B.Convert the compressed files to uncompressed format before processing

C.Repartition the data to fewer partitions

D.Increase the job timeout setting

AnswerA

More workers or higher memory workers provide more heap space for processing.

Why this answer

Increasing the number of G.1X workers or switching to G.2X workers directly addresses the 'Out of Memory' errors by allocating more memory per Spark executor. G.1X provides 16 GB of memory per worker, while G.2X provides 32 GB, which is critical when processing large compressed files because decompression and transformation require additional heap space. This approach resolves the issue without modifying the job code, as it only changes the resource configuration.

Exam trap

The trap here is that candidates often confuse 'Out of Memory' errors with performance issues and choose to reduce parallelism (Option C) or increase timeout (Option D), not realizing that memory exhaustion requires more memory per executor, not fewer tasks or longer runtime.

How to eliminate wrong answers

Option B is wrong because converting compressed files to uncompressed format would increase the data volume read from S3, potentially worsening memory pressure and increasing I/O costs, and it requires code changes to handle the new format. Option C is wrong because repartitioning to fewer partitions reduces parallelism, causing each Spark task to process more data, which would exacerbate memory issues rather than resolve them. Option D is wrong because increasing the job timeout setting only extends the maximum runtime before the job is killed; it does not address the underlying memory exhaustion, so the job will still fail with 'Out of Memory' errors.

Practice this question →

587

MCQeasy

A data engineer is setting up a data ingestion pipeline using Amazon Kinesis Data Firehose to deliver web server logs to Amazon S3. The logs are in JSON format and the engineer wants to convert them to Parquet format. The engineer has configured a Glue table for the schema. However, when testing, the Firehose delivery stream fails with 'Error converting to Parquet'. The engineer checks the Glue table schema and notices that it includes a column 'timestamp' of type 'string' in the format 'yyyy-MM-dd HH:mm:ss'. The logs have a 'timestamp' field in the same format. What is the MOST likely cause of the failure?

A.Firehose does not support Parquet conversion.

B.The Glue table schema does not match the data schema exactly.

C.The S3 bucket lacks write permissions for Firehose.

D.The 'timestamp' column must be of type 'timestamp' instead of 'string'.

AnswerB

Mismatch between schema and data causes conversion failure.

Why this answer

The failure is most likely due to a schema mismatch between the Glue table and the actual data. Firehose uses the Glue table schema to convert JSON to Parquet, and if the schema does not exactly match (e.g., column order, data types, or extra columns), the conversion fails. Option A is incorrect because Firehose does support Parquet conversion when a Glue table is provided.

Option C is incorrect because S3 permissions would cause a different error (e.g., AccessDenied). Option D is incorrect because a string type for timestamp is acceptable as long as it matches; the issue is not the type but the overall schema mismatch.

Practice this question →

588

MCQeasy

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed in real-time and then stored in Amazon S3. Which AWS service should be used to perform the transformation?

A.AWS Glue

B.Amazon EMR

C.Amazon Kinesis Data Analytics

D.Amazon Athena

AnswerC

Kinesis Data Analytics provides real-time stream processing capabilities.

Why this answer

Amazon Kinesis Data Analytics (now part of Amazon Managed Service for Apache Flink) is the correct choice because it can consume streaming data from Kinesis Data Streams, apply real-time transformations using SQL or Apache Flink, and then output the transformed data to destinations like Amazon S3. This directly meets the requirement for real-time transformation of streaming IoT data before storage.

Exam trap

The trap here is that candidates often confuse AWS Glue's batch ETL capabilities with real-time streaming, leading them to select Glue despite its lack of native support for live Kinesis Data Streams processing.

How to eliminate wrong answers

Option A is wrong because AWS Glue is a serverless ETL service designed for batch processing and cataloging data in data lakes, not for real-time stream transformations; it cannot directly process live Kinesis Data Streams with sub-second latency. Option B is wrong because Amazon EMR is a big data platform for running frameworks like Apache Spark or Hadoop on clusters, which is overkill and not optimized for lightweight, continuous real-time transformations on a single Kinesis stream; it introduces cluster management overhead and higher latency. Option D is wrong because Amazon Athena is an interactive query service for analyzing data already stored in S3 using SQL, not for transforming streaming data in motion; it cannot ingest or process live Kinesis Data Streams.

Practice this question →

589

MCQmedium

A company uses AWS DMS to migrate an on-premises PostgreSQL database to Amazon RDS for PostgreSQL. After initial load, ongoing replication is set up. The replication task shows 'Task status: failed with error: The specified LSN is not available in the source database logs.' What is the most likely cause?

A.DMS does not support PostgreSQL as a source for ongoing replication.

B.The source database's network security group blocks outbound traffic to DMS.

C.The full load was incomplete, preventing CDC from starting.

D.The source database's WAL retention period is too short, and required logs have been purged.

AnswerD

DMS uses WAL logs for CDC; if logs are purged, replication fails.

Why this answer

The error 'The specified LSN is not available in the source database logs' indicates that AWS DMS is trying to read a Write-Ahead Log (WAL) position that has already been purged. PostgreSQL sources require sufficient WAL retention to allow DMS to catch up during ongoing replication (CDC). If the WAL segments are recycled or removed before DMS reads them, the replication task fails with this specific LSN error.

Exam trap

The trap here is that candidates confuse connectivity issues (Option B) or general CDC support (Option A) with the specific LSN error, which is a WAL retention problem unique to PostgreSQL logical replication.

How to eliminate wrong answers

Option A is wrong because AWS DMS fully supports PostgreSQL as a source for ongoing replication using logical replication slots and WAL-based CDC. Option B is wrong because network security group blocks would cause connection timeout or 'Unable to connect' errors, not an LSN availability error. Option C is wrong because an incomplete full load would prevent the task from starting CDC at all, but the error message specifically refers to missing LSN in logs, which occurs after CDC has begun and the source WAL has been purged.

Practice this question →

590

MCQeasy

Refer to the exhibit. A data engineer runs the above CLI command to find files smaller than 1000 bytes in a bucket. The command returns an empty array, but the engineer knows there are small files. What is the issue?

A.The prefix is incorrect; it should be 'logs/2023/01/01/'.

B.The bucket policy does not allow listing objects.

C.The query syntax is invalid; use a filter instead.

D.The Size is compared as a string, not an integer; remove quotes around '1000'.

AnswerD

JMESPath comparison requires numeric types.

Why this answer

In the AWS CLI `list-objects-v2` command with `--query`, the `Size` field is a numeric value, but the query string `Size < '1000'` compares it as a string. This causes a lexicographic comparison, so files with sizes like '900' would be correctly matched, but any size with more digits (e.g., '1000' itself or '999') may fail due to string ordering. Removing the quotes around `1000` treats it as an integer, enabling proper numeric comparison.

Exam trap

The DEA-C01 exam often tests the subtle distinction between string and numeric comparisons in JMESPath queries, where candidates assume quoted values are automatically coerced to numbers, but in reality, quotes force string comparison.

How to eliminate wrong answers

Option A is wrong because the prefix 'logs/2023/01/01/' is not necessarily incorrect; the engineer knows small files exist, and the prefix is just a filter—if files are under that prefix, the issue is not the prefix. Option B is wrong because if the bucket policy did not allow listing objects, the CLI command would return an access denied error, not an empty array. Option C is wrong because the query syntax using `--query` with JMESPath expressions is valid; a filter is not required for this comparison, and the syntax `Size < '1000'` is syntactically correct but semantically wrong due to type coercion.

Practice this question →

591

MCQhard

A company uses Amazon Kinesis Data Analytics for real-time anomaly detection on clickstream data. The application uses a sliding window of 1 minute. The data engineer notices that the application is producing incorrect results because late-arriving records are not being handled properly. What should the data engineer do to ensure late records are included in the window calculations?

A.Use a Kinesis Data Firehose to buffer the data and then send to Kinesis Data Analytics.

B.Increase the watermark delay in the Kinesis Data Analytics application to allow more time for late records.

C.Increase the window size from 1 minute to 2 minutes.

D.Increase the retention period of the Kinesis stream to 7 days.

AnswerB

Watermark delay controls how long the application waits for late data.

Why this answer

Kinesis Data Analytics uses watermarks to track event time progress and determine when to finalize window calculations. Increasing the watermark delay allows the application to wait longer for late-arriving records before closing the window, ensuring they are included in the aggregation.

Exam trap

The trap here is that candidates confuse stream retention (how long data is stored) with watermark delay (how long the application waits for late events), leading them to incorrectly choose option D.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Firehose is a delivery stream that buffers data for loading into destinations like S3 or Redshift; it does not provide late-record handling logic for Kinesis Data Analytics windows. Option C is wrong because increasing the window size from 1 minute to 2 minutes does not address late arrivals—it simply aggregates over a longer period, which can still miss records that arrive after the window's end time. Option D is wrong because increasing the retention period of the Kinesis stream to 7 days only affects how long data is stored in the stream, not how the analytics application handles late-arriving records within its window computations.

Practice this question →

← PreviousPage 8 of 8 · 591 questions total

Ready to test yourself?

Try a timed practice session using only Data Ingestion and Transformation questions.

Start 20-question session