Knowledge + Practice

CCNA Data Ingestion and Transformation Questions

75 of 591 questions · Page 4/8 · Data Ingestion and Transformation · Answers revealed

Practice these questions Domain overview All questions

226

MCQhard

A social media company ingests user activity data from multiple sources into Amazon S3. The data is in JSON format and includes fields: user_id, activity_type, timestamp, and metadata. The company wants to transform this data into a columnar format (Parquet) partitioned by date and activity_type for efficient querying with Amazon Athena. The pipeline must handle data that arrives up to 3 days late. Currently, a daily AWS Glue ETL job scans the entire S3 bucket for new files, transforms them, and writes to a separate output bucket. The job is taking longer as data volume grows, and the team wants to reduce processing time and cost. What should the engineer do?

A.Increase the number of DPUs for the Glue job to process data faster.

B.Use AWS Glue partition projection and schema inference to reduce scan time.

C.Replace AWS Glue with Amazon EMR and use Spark to process data in parallel.

D.Set up S3 event notifications to invoke an AWS Lambda function that triggers a Glue job for each new object, passing the object key so the job processes only that file.

AnswerD

This enables incremental processing, reduces scan time, and is cost-effective.

Why this answer

Using S3 event notifications with Lambda to trigger a Glue job for each new file allows incremental processing, reducing the time and cost of scanning the entire S3 bucket. Option A (increasing DPUs) does not address the root cause of scanning all files. Option B (partition projection) helps with query performance but not with the transformation process.

Option C (replacing Glue with EMR) adds operational overhead and is not necessary for this use case.

Practice this question →

227

MCQeasy

A company needs to ingest data from an on-premises SQL Server database into Amazon Redshift. The data volume is less than 1 TB and the network bandwidth is limited. Which AWS service should be used for the initial full load?

A.AWS Snowball Edge

B.AWS Database Migration Service (DMS)

C.Amazon S3 Transfer Acceleration

D.AWS Direct Connect

AnswerB

Designed for database migration with limited bandwidth.

Why this answer

AWS DMS is designed for migrating databases to AWS, including to Redshift. Option A (AWS Snowball) is for large data volumes (petabytes) and not efficient for <1 TB. Option C (Amazon S3 Transfer Acceleration) speeds up uploads to S3 but not directly to Redshift.

Option D (AWS Direct Connect) is a network connection, not a migration service.

Practice this question →

228

MCQeasy

A company wants to import data from an external FTP server into Amazon S3 on a daily basis. The data volumes are moderate. Which AWS service is MOST suitable for this task?

A.Amazon S3 Transfer Acceleration

B.AWS Transfer Family

C.AWS DataSync

D.AWS Glue with a JDBC connection

AnswerB

Transfer Family supports FTP/SFTP/FTPS and directly writes to S3.

Why this answer

AWS Transfer Family is the most suitable service because it provides fully managed support for SFTP, FTPS, and FTP protocols, enabling direct, secure file transfers from an external FTP server to Amazon S3 without needing to manage any infrastructure. It integrates natively with S3, so files are automatically stored in the specified bucket upon transfer completion, making it ideal for daily imports of moderate data volumes.

Exam trap

The trap here is that candidates often confuse AWS DataSync with a general-purpose file transfer tool, but DataSync requires an agent on the source and does not natively support FTP protocols, whereas AWS Transfer Family is purpose-built for FTP-based transfers to S3.

How to eliminate wrong answers

Option A is wrong because Amazon S3 Transfer Acceleration is a feature that speeds up uploads to S3 over the internet by using AWS edge locations, but it does not support FTP protocols or act as a server-side endpoint to receive files from an external FTP server. Option C is wrong because AWS DataSync is designed for moving large volumes of data between on-premises storage and AWS services, but it requires installing an agent on the source environment and does not natively support FTP as a source protocol. Option D is wrong because AWS Glue with a JDBC connection is intended for extracting data from databases using JDBC drivers, not for handling file transfers over FTP/SFTP/FTPS protocols.

Practice this question →

229

MCQmedium

A data engineer is building a real-time data pipeline to ingest sensor data from IoT devices. The data is sent to AWS IoT Core, which publishes messages to a Kinesis Data Stream. Each message is about 1 KB in size. The data must be transformed (add a device location field) and then stored in Amazon S3 for long-term analytics. The engineer has set up a Lambda function to transform the records and write to S3. However, the engineer notices that the Lambda function is invoked thousands of times per second, causing high costs and occasional throttling. The Lambda function processes only one record at a time. The engineer wants to reduce the number of Lambda invocations and improve throughput. What should the engineer do?

A.Reduce the number of shards in the Kinesis stream to limit concurrency.

B.Increase the Lambda function's memory allocation to improve performance.

C.Replace the Lambda function with Amazon Kinesis Data Firehose and use its built-in transformation.

D.Configure the event source mapping to use a larger batch size and set a batch window.

AnswerD

Correct. Configuring the event source mapping to use a larger batch size and set a batch window allows Lambda to process multiple records in a single invocation, drastically reducing invocation count and improving throughput.

Why this answer

Configuring the event source mapping with a larger batch size and a batch window allows Lambda to process multiple records per invocation, reducing the number of invocations and costs. This improves throughput and reduces throttling. Option A is incorrect because reducing shards reduces the stream capacity, causing backpressure and potential data loss.

Option B is incorrect because increasing memory does not reduce the number of invocations; it only speeds up processing per invocation, but still processes one record at a time. Option C is incorrect because Kinesis Data Firehose can batch records, but it still uses per-record Lambda transformation if you use a Lambda function, or it can use built-in transformations but not the flexible logic described. The most direct solution is to batch records in the existing Lambda function via event source mapping parameters.

Exam trap

A candidate might think that reducing the number of shards will reduce invocations, but that actually reduces the stream's ability to handle the data volume and can cause throttling or data loss.

Practice this question →

230

MCQmedium

A data engineer notices that an AWS Glue ETL job is running slower than expected. The job reads from Amazon S3, joins two datasets, and writes the result back to S3. The job uses the default worker type (G.1X) and 10 DPUs. Which action is most likely to improve performance?

A.Increase the number of DPUs to 20

B.Repartition the data before the join operation

C.Use coalesce to reduce the number of output files

D.Change the worker type to G.2X

AnswerB

Optimizes parallelism and reduces shuffling.

Why this answer

The default G.1X worker type provides 16 GB of memory and 4 vCPUs per DPU. With 10 DPUs, the job likely has sufficient compute but suffers from data skew or inefficient partitioning during the join. Repartitioning the data before the join ensures that keys are evenly distributed across partitions, reducing shuffle overhead and preventing straggler tasks, which directly improves performance.

Exam trap

The trap here is that candidates often assume more DPUs or a larger worker type always speeds up a job, but the DEA-C01 exam tests understanding that shuffle optimization (like repartitioning) is the most impactful fix for join performance issues.

How to eliminate wrong answers

Option A is wrong because increasing DPUs to 20 adds more parallelism but does not address the root cause of poor join performance, which is data skew or uneven partitioning; it may even increase shuffle overhead. Option C is wrong because coalesce reduces the number of output files by merging partitions, which is useful for downstream S3 reads but does not improve join performance and can actually cause data movement that slows the job. Option D is wrong because changing to G.2X (which doubles memory and vCPUs per DPU) may help memory-intensive operations but does not fix the partitioning issue; the job is likely I/O or shuffle-bound, not memory-bound.

Practice this question →

231

MCQmedium

Refer to the exhibit. A data engineer is configuring an IAM policy for an AWS Glue ETL job that reads data from the 'my-data-bucket' S3 bucket, transforms it, and writes the output back to the same bucket. The engineer wants to prevent accidental deletion of objects. Based on the policy, which statement is true about the Glue job's permissions?

A.The job can write objects but cannot read objects.

B.The job can read objects but cannot write objects.

C.The job can read and write, but may also delete objects.

D.The job can read and write objects, but cannot delete objects.

AnswerD

Get and Put allowed; Delete denied.

Why this answer

The IAM policy explicitly denies the `s3:DeleteObject` action, which prevents the Glue job from deleting objects in the 'my-data-bucket' S3 bucket. The policy allows `s3:GetObject` and `s3:PutObject` actions, enabling the job to read and write objects as required for the ETL process. This ensures the job can perform its transformation tasks without the risk of accidental deletion.

Exam trap

The trap here is that candidates may overlook the explicit deny statement and assume the job has full S3 access based on the allow actions, forgetting that an explicit deny overrides all allows.

How to eliminate wrong answers

Option A is wrong because the policy includes `s3:GetObject` permission, allowing the job to read objects, not just write. Option B is wrong because the policy includes `s3:PutObject` permission, allowing the job to write objects, not just read. Option C is wrong because the policy explicitly denies `s3:DeleteObject`, so the job cannot delete objects, contradicting the claim that it may also delete.

Practice this question →

232

MCQeasy

A data engineer is ingesting streaming data from an IoT fleet into Amazon S3 using Amazon Kinesis Data Firehose. The data arrives as JSON, but the downstream analytics require Parquet format. Which Firehose transformation should the engineer configure?

A.Use an S3 lifecycle policy to convert JSON to Parquet.

B.Configure a Lambda function as a data transformation in Firehose to convert JSON to Parquet.

C.Use S3 Batch Operations to convert existing JSON objects to Parquet.

D.Use Kinesis Data Analytics to convert the stream to Parquet before writing to S3.

AnswerB

Lambda can transform data format during delivery.

Why this answer

Amazon Kinesis Data Firehose can invoke an AWS Lambda function as a data transformation step to convert incoming JSON records to Parquet format before delivery to S3. This is the native, serverless way to perform record-level format conversion within the Firehose delivery stream, ensuring downstream analytics tools can directly query the Parquet data without additional processing.

Exam trap

The trap here is that candidates may confuse S3 lifecycle policies or Batch Operations as viable transformation tools, overlooking that Firehose's Lambda integration is the only option that performs real-time, record-level format conversion within the streaming pipeline.

How to eliminate wrong answers

Option A is wrong because S3 lifecycle policies manage object lifecycle transitions (e.g., to Glacier) or expiration, not format conversion; they cannot change JSON to Parquet. Option C is wrong because S3 Batch Operations are designed for bulk actions on existing objects (e.g., tagging, copying) and are not suitable for real-time streaming data conversion. Option D is wrong because Kinesis Data Analytics processes streaming data with SQL or Flink but does not natively output Parquet to S3; it would require a custom sink or additional transformation, making it an overly complex and indirect solution compared to Firehose's built-in Lambda transformation.

Practice this question →

233

MCQeasy

A company wants to ingest data from an on-premises Oracle database into Amazon S3 on a daily basis. The data volume is 500 GB per transfer. Which AWS service is most appropriate for this batch ingestion?

A.AWS Database Migration Service (DMS)

B.AWS Data Pipeline

C.Amazon Kinesis Data Firehose

D.AWS Glue

AnswerD

Glue can run scheduled crawlers and ETL jobs for batch ingestion.

Why this answer

AWS Glue is the most appropriate service for this batch ingestion because it is purpose-built for ETL (Extract, Transform, Load) workflows, including connecting to on-premises databases via JDBC, extracting large volumes of data (500 GB per day), and writing it to Amazon S3 in a scheduled, serverless manner. Glue's built-in crawlers and job scheduler handle daily batch runs efficiently without requiring manual infrastructure management, making it ideal for this use case.

Exam trap

The trap here is that candidates often confuse AWS Glue (batch ETL) with Amazon Kinesis Data Firehose (streaming), assuming both can handle any data ingestion, but Firehose cannot pull from a relational database and is not designed for large daily batch loads.

How to eliminate wrong answers

Option A is wrong because AWS DMS is designed for continuous, real-time database replication and migration, not for scheduled daily batch ingestion of 500 GB into S3; it focuses on keeping source and target in sync rather than periodic bulk loads. Option B is wrong because AWS Data Pipeline is a legacy service that requires managing EC2 instances and has been largely superseded by AWS Glue for ETL workloads; it lacks the serverless simplicity and native integration with Glue's catalog and crawlers. Option C is wrong because Amazon Kinesis Data Firehose is built for streaming data ingestion (near real-time, small records) and cannot handle 500 GB batch transfers from an on-premises Oracle database via JDBC; it expects data to be pushed via HTTP, Kinesis Streams, or SDK, not pulled from a relational database.

Practice this question →

234

MCQmedium

A data engineer is building a data ingestion pipeline that reads JSON files from Amazon S3 and loads them into an Amazon Redshift table using COPY commands. The files are gzip compressed and contain nested JSON. The engineer wants to minimize transformation steps. Which approach should the engineer use?

A.Use Amazon Athena to query the JSON and INSERT INTO Redshift.

B.Use Kinesis Data Firehose to transform and load into Redshift.

C.Use the COPY command with the 'auto' option to ingest JSON directly.

D.Use AWS Glue ETL to flatten the JSON and write to S3 as CSV, then COPY from CSV.

AnswerC

COPY with 'auto' automatically parses JSON.

Why this answer

The COPY command with the 'auto' option can directly ingest gzip-compressed JSON files from S3 into Redshift, automatically inferring the schema and handling nested structures without requiring intermediate transformation steps. This minimizes transformation steps by leveraging Redshift's native JSON parsing capability, which supports both 'auto' and 'jsonpaths' options for nested data.

Exam trap

The trap here is that candidates often overcomplicate the solution by assuming nested JSON requires an ETL tool like Glue or Athena, when Redshift's COPY command with 'auto' or 'jsonpaths' can handle nested structures natively, minimizing transformation steps as explicitly requested.

How to eliminate wrong answers

Option A is wrong because using Athena to query JSON and then INSERT INTO Redshift adds unnecessary transformation steps and latency, as Athena is an interactive query service not designed for high-throughput ingestion pipelines, and the INSERT approach lacks the parallelism and compression handling of COPY. Option B is wrong because Kinesis Data Firehose is optimized for streaming data, not batch ingestion from S3, and it would require additional configuration to read from S3 and transform JSON, introducing extra complexity and cost. Option D is wrong because using AWS Glue ETL to flatten JSON to CSV adds an unnecessary transformation step, contradicting the requirement to minimize transformation steps; the COPY command can directly handle nested JSON without flattening.

Practice this question →

235

MCQhard

Refer to the exhibit. A data engineer runs this AWS CLI command to create a Glue job. The job processes JSON files in an S3 bucket and writes Parquet files to another bucket. After the first successful run, the job re-processes all input files instead of only new files. What is the most likely cause?

A.The ScriptLocation points to an incorrect S3 path.

B.The --max-retries parameter is set to 0.

C.The job script does not implement job bookmark support.

D.The IAM role lacks permissions to read bookmark state.

AnswerC

Bookmarks require explicit implementation in the script.

Why this answer

The command sets '--job-bookmark-enable' but if the job script does not use the bookmark APIs or implement bookmark support, Glue will not track processed files and will reprocess all input on each run. Option A is incorrect because the ScriptLocation is valid and does not affect bookmark behavior. Option B is incorrect because max-retries does not control reprocessing.

Option D is incorrect because the IAM role is specified and permissions for bookmark state are not explicitly shown, but the lack of bookmark support in the script is the issue.

Practice this question →

236

Multi-Selecthard

A data engineering team is building a data lake on Amazon S3. They need to ingest data from multiple sources: (1) streaming IoT data, (2) daily CSV exports from an on-premises system via SFTP, and (3) change data capture (CDC) from an Amazon Aurora database. Which THREE services should the team use to ingest these data sources?

Select 3 answers

A.Amazon Kinesis Data Streams for IoT data ingestion.

B.AWS Database Migration Service (DMS) for CDC from Aurora.

C.AWS Transfer Family for SFTP-based file ingestion.

D.AWS Glue ETL for CDC from Aurora.

E.Amazon EMR for daily CSV ingestion.

AnswersA, B, C

Kinesis is ideal for real-time streaming data from devices.

Why this answer

Amazon Kinesis Data Streams is purpose-built for real-time streaming data ingestion, making it ideal for IoT data that arrives continuously. It can capture and store data streams for processing by consumers like Kinesis Data Analytics or Lambda, providing low-latency ingestion and durable storage.

Exam trap

The trap here is confusing AWS Glue ETL (a batch ETL tool) with AWS DMS (a database migration and CDC service), and assuming Amazon EMR is an ingestion service rather than a processing framework for large-scale data transformations.

Practice this question →

237

MCQmedium

Refer to the exhibit. A data engineer has attached this IAM policy to an AWS Glue job role. The Glue job fails when trying to write transformed data to an S3 bucket located in a different AWS account. What is the most likely reason?

A.The policy does not allow lambda:InvokeAsync

B.The Glue job role does not have permissions to write to S3

C.The policy does not grant s3:ListBucket, and the bucket policy may not allow cross-account access

D.The policy does not include kinesis:DescribeStream

AnswerC

Cross-account S3 access requires both bucket policy and IAM permissions, including ListBucket.

Why this answer

The IAM policy shown does not include the s3:ListBucket permission, which is required for the Glue job to list objects in the S3 bucket before writing. Additionally, cross-account access requires both the source account's IAM policy (this one) to grant write permissions and the target account's S3 bucket policy to explicitly allow the source account's role, which may not be configured. Without s3:ListBucket, the Glue job cannot verify the bucket's existence or structure, causing the write operation to fail.

Exam trap

The trap here is that candidates assume s3:PutObject alone is sufficient for writing to S3, but AWS requires s3:ListBucket for bucket-level operations like listing and validation, especially in cross-account scenarios where the bucket's existence must be confirmed.

How to eliminate wrong answers

Option A is wrong because lambda:InvokeAsync is a permission for invoking AWS Lambda functions asynchronously, which is irrelevant to writing data to S3 from a Glue job. Option B is wrong because the policy does include s3:PutObject and s3:PutObjectAcl, which are write permissions; the failure is due to missing s3:ListBucket, not a lack of write permissions entirely. Option D is wrong because kinesis:DescribeStream is a permission for Amazon Kinesis streams, which is unrelated to S3 write operations in this cross-account scenario.

Practice this question →

238

MCQhard

A data engineer is reviewing the S3 Lifecycle policy for a data lake bucket. The goal is to archive log data after 30 days and delete it after 365 days, and delete temporary data after 1 day. What is wrong with the current configuration?

A.The prefix filter for the first rule does not include a wildcard, so it may not match all log files.

B.The rule for temp data has no transition, so it will not expire objects.

C.The expiration for the first rule will not delete objects in GLACIER storage class unless they are restored first.

D.The transition to GLACIER should be after 30 days, but the expiration should be after 365 days from the transition, not from creation.

AnswerD

The Days in lifecycle rules are always from the object creation date, not from the transition.

Why this answer

The current lifecycle configuration is correct: it transitions objects to GLACIER after 30 days and expires them 365 days after creation. However, a common mistake is to configure expiration relative to the transition date. Option D identifies this error by stating that expiration should be based on creation date, not transition.

The other options are incorrect: A - prefix filters do not require wildcards; B - expiration actions do not need a prior transition; C - S3 Lifecycle can expire GLACIER objects without restoration.

Exam trap

Many candidates think that expiration for GLACIER objects requires prior restoration, but AWS documentation states that expiration can delete objects directly. Also, expiration is always based on creation date, not transition date.

Practice this question →

239

MCQmedium

A company is ingesting streaming data from a fleet of weather sensors. Each sensor sends a JSON payload every second. The data is used for real-time dashboarding and also archived to S3. The pipeline should handle sudden bursts of data without data loss. Which architecture meets these requirements?

A.Amazon EC2 with Apache Kafka -> S3

B.Amazon Kinesis Data Streams -> AWS Lambda for dashboard -> Amazon Kinesis Data Firehose -> S3

C.Amazon Kinesis Data Firehose directly with no buffer

D.Amazon SQS -> AWS Lambda -> S3

AnswerB

Streams provide buffer, Firehose delivers to S3, Lambda processes for dashboard.

Why this answer

Kinesis Data Streams provides durable, scalable ingestion that can handle sudden bursts of data without loss, while Lambda processes records for real-time dashboarding and Kinesis Data Firehose reliably buffers and archives data to S3. This decoupled architecture ensures no data is lost even during traffic spikes, as Kinesis Data Streams retains data for up to 365 days and Firehose can buffer incoming records before writing to S3.

Exam trap

The DEA-C01 exam often tests the misconception that Kinesis Data Firehose can be used as a standalone ingestion service without a buffer, but the trap here is that Firehose requires a buffer (minimum 60 seconds or 1 MB) to function, and without it, data would be lost during bursts, making Option C an incorrect choice.

How to eliminate wrong answers

Option A is wrong because Apache Kafka on EC2 introduces operational overhead for managing brokers, partitions, and replication, and does not natively integrate with S3 without additional tooling like Kafka Connect or a custom consumer, making it less reliable for a fully managed serverless pipeline. Option C is wrong because Kinesis Data Firehose with no buffer cannot handle sudden bursts of data; it requires a buffer interval (minimum 60 seconds) or buffer size to accumulate records before delivery, and without buffering it would fail to absorb spikes, leading to data loss or throttling. Option D is wrong because Amazon SQS does not guarantee order preservation for streaming data (unless using FIFO, which limits throughput) and Lambda's 6-minute timeout and lack of native S3 archiving make it unsuitable for continuous, high-frequency ingestion and archival without additional components like Firehose.

Practice this question →

240

MCQeasy

A company needs to ingest data from an external API that returns CSV files daily. The files range from 100 MB to 2 GB. The data should be landed in Amazon S3 and then transformed using AWS Glue. Which ingestion method is most cost-effective and requires the least operational overhead?

A.Set up AWS DataSync to transfer the file from the API endpoint to S3

B.Use Amazon Kinesis Data Firehose with a direct PUT

C.Deploy an AWS Direct Connect connection to the external API for faster transfer

D.Schedule an AWS Lambda function to download the CSV file and upload it to Amazon S3

AnswerD

Simple, cost-effective, and serverless for daily files.

Why this answer

Scheduling an AWS Lambda function to download the CSV file from the external API and upload it to Amazon S3 is the most cost-effective and operationally lightweight approach. Lambda can handle files up to 2 GB (with appropriate memory and timeout settings) and runs on a serverless, pay-per-execution model, eliminating the need for infrastructure management. This method directly addresses the daily, batch-oriented nature of the data ingestion without requiring additional services or complex configurations.

Exam trap

The trap here is that candidates often over-engineer the solution by choosing streaming or dedicated network services (like Kinesis Firehose or Direct Connect) for a simple batch ingestion task, failing to recognize that a serverless, scheduled Lambda function is the most cost-effective and low-overhead approach for daily CSV file transfers from an external API.

How to eliminate wrong answers

Option A is wrong because AWS DataSync is designed for transferring data between on-premises storage and AWS, not for pulling data from an external API endpoint; it cannot directly interface with an HTTP-based API that returns CSV files. Option B is wrong because Amazon Kinesis Data Firehose with a direct PUT is optimized for streaming, real-time data ingestion, not for handling large, daily batch CSV files up to 2 GB; it would introduce unnecessary complexity and cost for a simple scheduled batch transfer. Option C is wrong because AWS Direct Connect provides a dedicated network connection from on-premises to AWS, but it does not connect to an external API; it is intended for hybrid cloud architectures and would be overkill, costly, and operationally heavy for this use case.

Practice this question →

241

MCQmedium

A company wants to ingest data from an on-premises SQL Server database into Amazon Redshift. They need to transform the data during ingestion, such as masking PII columns. Which approach meets these requirements with minimal operational overhead?

A.Use AWS Glue ETL jobs to extract data from SQL Server, transform it, and load into Redshift

B.Use a custom application on EC2 to extract, transform, and load

C.Use Kinesis Data Firehose to stream data from SQL Server to Redshift

D.Use AWS Database Migration Service (DMS) with transformation rules

AnswerD

DMS supports data transformation and loads into Redshift.

Why this answer

AWS DMS can perform transformations, such as data masking, during the migration process and can load data directly into Amazon Redshift, minimizing operational overhead. Option A (AWS Glue ETL) adds an extra step and overhead compared to DMS. Option B (custom application on EC2) introduces high operational overhead for management and scaling.

Option C (Kinesis Data Firehose) is designed for streaming data and is not suitable for batch ingestion from an on-premises SQL Server database.

Practice this question →

242

MCQhard

A data engineer is designing a data ingestion pipeline for a social media company. The pipeline ingests user posts from a REST API into Amazon S3. The API returns JSON data with an array of posts. The engineer needs to transform the data into individual JSON objects per post and store them in S3 with a partition structure of year/month/day/hour. The data should be available in S3 within 15 minutes of ingestion. The engineer decides to use AWS Lambda for transformation. Which combination of services should the engineer use to meet these requirements with minimal operational overhead?

A.Use AWS Step Functions to orchestrate an API call and data transformation with Lambda, running every 15 minutes.

B.Use Amazon CloudWatch Events to trigger an AWS Lambda function every 15 minutes. The Lambda function calls the API, transforms the data, and writes individual JSON objects to S3 with the required partition structure.

C.Use AWS Glue ETL jobs scheduled with AWS Glue triggers to run every 15 minutes.

D.Use Amazon Kinesis Data Firehose with a Lambda function for transformation. Configure Firehose to pull from the API every 15 minutes.

AnswerB

Simple and cost-effective for periodic API polling.

Why this answer

Using Amazon CloudWatch Events (or EventBridge) to trigger an AWS Lambda function every 15 minutes provides a simple, serverless solution with minimal operational overhead. The Lambda function can call the REST API, transform the array of posts into individual JSON objects, and write them to S3 with the partition structure year/month/day/hour. This meets the 15-minute latency requirement without managing infrastructure.

Option A (AWS Step Functions) adds unnecessary orchestration complexity for a simple scheduled task. Option C (AWS Glue ETL) is too heavyweight for this lightweight transformation and incurs additional cost and setup. Option D (Amazon Kinesis Data Firehose) is designed for streaming data, not periodic batch API calls; it cannot pull from an API on a schedule without custom logic, making it less suitable.

Practice this question →

243

MCQmedium

A data engineer is troubleshooting an AWS Glue ETL job that fails with an OutOfMemory error when processing large JSON files from Amazon S3. The files contain deeply nested structures. Which approach should the engineer take to resolve this issue?

A.Use the `recurse` option with `getResolvedOptions` to limit recursion

B.Increase the number of workers in the Glue job configuration

C.Increase the DPU (Data Processing Unit) per worker

D.Decrease the number of partitions while reading the data

AnswerC

More DPU per worker allocates more memory, resolving OOM.

Why this answer

Increasing the DPU per worker allocates more memory per worker, directly addressing the OutOfMemory error when processing large files. Option A is incorrect because the `recurse` option is not a valid argument for `getResolvedOptions`, which is used for retrieving job parameters. Option B is incorrect because increasing the number of workers adds parallelism but does not increase memory per worker; OOM occurs per worker.

Option D is incorrect because decreasing partitions reduces parallelism, potentially causing each partition to be larger, worsening the memory issue.

Practice this question →

244

Multi-Selectmedium

A company is designing a data ingestion pipeline for clickstream data from a website. The data must be ingested in near real-time. Which TWO services can be used together to build this pipeline?

Select 2 answers

A.Amazon Kinesis Data Streams

B.Amazon Simple Queue Service (SQS)

C.Amazon S3

D.Amazon Kinesis Data Firehose

E.Amazon DynamoDB

AnswersA, D

Amazon Kinesis Data Streams can ingest clickstream data in near real-time, making it suitable for the pipeline.

Why this answer

Amazon Kinesis Data Streams can ingest clickstream data in near real-time. Option D is correct because Amazon Kinesis Data Firehose can deliver that streaming data to destinations like S3. Option B is wrong because SQS is a message queuing service, not a streaming ingestion service.

Option C is wrong because S3 is a storage service, not a real-time ingestion service. Option E is wrong because DynamoDB is a NoSQL database, not a streaming ingestion service.

Practice this question →

245

MCQmedium

Refer to the exhibit. An AWS Glue ETL job is failing with an OutOfMemoryError. The job reads from Amazon S3 and performs a GROUP BY on a large dataset. Which change should the data engineer make to resolve this error?

A.Use coalesce to reduce the number of partitions.

B.Increase the number of DPUs allocated to the Glue job.

C.Increase the number of partitions in the DataFrame.

D.Use repartition to increase the number of partitions.

AnswerB

More DPUs increase total memory available.

Why this answer

The OutOfMemoryError in an AWS Glue ETL job performing a GROUP BY on a large dataset indicates that the executors do not have enough memory to handle the shuffle operations required for aggregation. Increasing the number of DPUs (Data Processing Units) allocated to the Glue job increases the total memory and compute resources available, allowing the job to process larger partitions without running out of memory.

Exam trap

The trap here is that candidates often confuse partition tuning (coalesce/repartition) with resource allocation, mistakenly thinking that adjusting partitions alone can fix memory errors without increasing the underlying compute and memory capacity.

How to eliminate wrong answers

Option A is wrong because using coalesce to reduce the number of partitions would decrease parallelism and concentrate data into fewer partitions, potentially worsening memory pressure and making the OutOfMemoryError more likely. Option C is wrong because increasing the number of partitions in the DataFrame without adding more resources (DPUs) would spread data across more tasks but still rely on the same total memory, which does not resolve the underlying memory shortage. Option D is wrong because repartitioning to increase the number of partitions similarly does not add memory; it only redistributes data, which can even increase shuffle overhead and exacerbate memory issues.

Practice this question →

246

Multi-Selecteasy

A data engineer is designing a data ingestion pipeline to load JSON files from Amazon S3 into Amazon Redshift. Which TWO methods can be used to load the data efficiently?

Select 2 answers

A.Use Amazon Kinesis Data Firehose to directly load into Redshift.

B.Use AWS DMS to replicate from S3 to Redshift.

C.Use the Redshift COPY command to load from S3.

D.Use a staging table in S3 and then COPY into Redshift.

E.Use individual INSERT statements in a loop.

AnswersC, D

COPY is the fastest way to bulk load from S3.

Why this answer

The Redshift COPY command is specifically designed to efficiently load large datasets from Amazon S3 by automatically parallelizing the data across cluster nodes, leveraging the cluster's compute resources for high-throughput ingestion. It supports JSON data natively via the 'json' option, making it ideal for loading JSON files directly from S3 without intermediate transformations.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Firehose's ability to 'deliver' to Redshift with the actual loading mechanism, not realizing that Firehose only writes to S3 and then triggers a COPY command, making Option A a distractor for a direct load method.

Practice this question →

247

MCQhard

A data engineer needs to design a data ingestion pipeline that captures change data capture (CDC) events from an on-premises SQL Server database to Amazon S3 with low latency. The pipeline must handle schema changes and ensure exactly-once delivery semantics. Which combination of AWS services should the engineer use?

A.AWS Database Migration Service (DMS) with Amazon Kinesis Data Firehose to Amazon S3

B.AWS AppFlow with SQL Server connector to Amazon S3

C.AWS Glue ETL job with JDBC connection to SQL Server and writing to Amazon S3

D.Amazon Kinesis Data Streams with AWS Lambda consumer writing to Amazon S3

AnswerA

DMS captures CDC, Firehose delivers to S3 with low latency and supports partitioning.

Why this answer

AWS DMS can capture ongoing changes from SQL Server using its CDC capability and stream them directly into Amazon Kinesis Data Firehose, which buffers and delivers data to Amazon S3 with low latency. DMS handles schema changes by propagating them to the target, and Kinesis Data Firehose, combined with DMS's transactional integrity, supports exactly-once delivery semantics when configured with a primary key and appropriate error handling.

Exam trap

The trap here is that candidates often assume AWS Glue or Kinesis Data Streams are the default for real-time CDC, but they overlook DMS's native CDC support for on-premises databases and its seamless integration with Firehose for exactly-once delivery.

How to eliminate wrong answers

Option B is wrong because AWS AppFlow does not support CDC from on-premises SQL Server; it is designed for SaaS applications and requires a public endpoint, not on-premises databases. Option C is wrong because AWS Glue ETL jobs with JDBC are batch-oriented, not real-time, and cannot provide low-latency CDC or exactly-once delivery without complex custom checkpointing. Option D is wrong because Amazon Kinesis Data Streams with a Lambda consumer does not natively integrate with SQL Server CDC; it would require custom code to capture changes, and Lambda does not guarantee exactly-once delivery to S3 due to potential retries and lack of idempotency handling.

Practice this question →

248

MCQhard

A company runs an e-commerce platform that generates clickstream data from user interactions on their website. The data is sent as JSON objects via HTTP POST to an API Gateway endpoint, which triggers a Lambda function that writes each record to a Kinesis Data Stream (100 shards). A second Lambda function consumes the stream, transforms the data (enriches with geolocation from a DynamoDB table), and writes to a Kinesis Data Firehose delivery stream that delivers Parquet files to an S3 data lake every 5 minutes. The system has been working for months, but recently the Firehose delivery stream started showing 'DeliveryFailed' errors for a subset of records. The errors point to 'InvalidData' from the Lambda transformation. The engineer reviews the Lambda transformation code and notices that the geolocation lookup occasionally fails because the DynamoDB table has a throttling issue. The engineer needs to handle these failures gracefully so that records that fail enrichment are still delivered to S3 with a null geolocation field, without blocking other records. Which course of action should the engineer take?

A.Configure the Kinesis Data Firehose delivery stream to send failed records to a dead-letter queue (DLQ) for later reprocessing.

B.Modify the Lambda function to send failed records to a separate Kinesis Data Stream for manual processing.

C.Modify the Lambda function to catch exceptions during the geolocation lookup, set the geolocation field to null, and continue processing the record.

D.Increase the read capacity units (RCUs) on the DynamoDB table to eliminate throttling.

AnswerC

This ensures all records are delivered with a default value, maintaining pipeline throughput.

Why this answer

It modifies the Lambda function to catch exceptions during the geolocation lookup, set the geolocation field to null, and continue processing. This ensures that records that fail enrichment are still delivered to S3 with a null geolocation field, without blocking other records, and without requiring additional infrastructure. Option A is incorrect because Kinesis Data Firehose does not natively support a dead-letter queue (DLQ); failed records can be sent to an S3 bucket for failed data, but that would not include the transformed data with null geolocation.

Option B is incorrect because sending failed records to a separate Kinesis Data Stream adds complexity and does not ensure they are delivered to S3 with the desired null geolocation field. Option D is incorrect because increasing RCUs may reduce throttling but does not eliminate the possibility of failures, and it increases cost; the requirement is to handle failures gracefully, not prevent them entirely.

Practice this question →

249

MCQeasy

A data engineer needs to ingest daily CSV files from an external FTP server into Amazon S3. The files are 5 GB each. Which service is MOST suitable to automate this ingestion?

A.AWS AppSync

B.AWS DataSync

C.Amazon S3 Transfer Acceleration

D.AWS Glue

AnswerB

DataSync supports scheduled transfers from FTP servers to S3 with built-in monitoring.

Why this answer

AWS DataSync is the most suitable service for automated, scheduled transfers of large files from an external FTP server to Amazon S3. It supports both one-time and recurring transfers and can handle high throughput for files up to 5 GB. S3 Transfer Acceleration only speeds up uploads to S3, not transfers from FTP servers.

AWS Glue is an ETL service, not a file transfer tool. AWS AppSync is for real-time APIs, not batch file ingestion.

Practice this question →

250

MCQmedium

Refer to the exhibit. A data engineer is using a Kinesis Data Stream with one shard. The application writes 2000 records per second, each 1 KB. The put record calls are frequently throttled. What is the most likely cause?

A.The stream has only one shard, which limits writes to 1000 records per second

B.The retention period of 24 hours is too short

C.The stream uses KMS encryption, causing additional latency

D.Enhanced monitoring is not enabled, causing performance issues

AnswerA

Each shard supports 1000 records/sec write.

Why this answer

A Kinesis Data Stream shard has a write throughput limit of 1,000 records per second (or 1 MB per second). Since the application is writing 2,000 records per second (each 1 KB) to a single shard, it exceeds the shard's record-per-second quota, causing the PutRecord calls to be throttled. The solution is to increase the number of shards to at least two to distribute the load.

Exam trap

The DEA-C01 exam often tests the misconception that throttling is caused by encryption latency or monitoring settings, but the real trap is forgetting that each shard has a hard limit of 1,000 records per second, regardless of other configurations.

How to eliminate wrong answers

Option B is wrong because the retention period (default 24 hours, max 365 days) controls how long records are stored, not the write throughput; throttling is unrelated to retention. Option C is wrong because KMS encryption adds latency to encrypt/decrypt operations but does not reduce the shard-level write limit of 1,000 records per second; throttling is a capacity issue, not a latency issue. Option D is wrong because enhanced monitoring provides detailed metrics (e.g., user, request, stream-level) but does not affect the shard's write throughput limits; throttling occurs regardless of monitoring settings.

Practice this question →

251

MCQmedium

A company uses AWS Lambda to process messages from an Amazon SQS queue. The messages contain JSON payloads that need to be transformed and written to an Amazon DynamoDB table. Recently, the Lambda function has been timing out and messages are being sent to the dead-letter queue (DLQ). What is the BEST way to troubleshoot and resolve this issue?

A.Use a standard SQS queue instead of a DLQ to reprocess failed messages automatically.

B.Increase the Lambda function timeout and monitor DynamoDB write capacity to ensure it is not throttling.

C.Switch the SQS queue to a FIFO queue to ensure exactly-once processing.

D.Increase the visibility timeout of the SQS queue to 30 minutes.

AnswerB

Increasing timeout allows longer processing; DynamoDB throttling could cause delays.

Why this answer

The Lambda function is timing out, which suggests the function's execution duration is exceeding its configured timeout. Increasing the timeout gives the function more time to process messages. Additionally, if DynamoDB write capacity is insufficient, throttling can cause retries that further delay processing, so monitoring and possibly increasing write capacity units addresses the root cause of timeouts.

Exam trap

The trap here is that candidates often focus on SQS queue configuration (visibility timeout, queue type) rather than addressing the actual performance bottleneck in the Lambda function or downstream DynamoDB service.

How to eliminate wrong answers

Option A is wrong because using a standard SQS queue instead of a DLQ does not automatically reprocess failed messages; it only changes the queue type and does not address the underlying timeout or throttling issue. Option C is wrong because switching to a FIFO queue enforces exactly-once processing and message ordering, but it does not resolve timeouts or DynamoDB throttling; it could even introduce additional latency. Option D is wrong because increasing the visibility timeout to 30 minutes only delays when a message becomes visible again after a failure, but it does not fix the root cause of timeouts or throttling; it may mask the problem.

Practice this question →

252

MCQeasy

The command returns an empty result, but you know there are objects in the 'logs/' prefix larger than 1000 bytes. What is the MOST likely reason?

A.The prefix 'logs/' is incorrect; the objects are in a different prefix.

B.The comparison 'Size > '1000'' uses a string instead of a number, so it never matches.

C.The command does not paginate, so it only checks the first 1000 objects.

D.The output format is set to text, but the query requires JSON.

AnswerB

Size is a numeric field; comparing to a string causes the filter to be false.

Why this answer

The `Size > '1000'` comparison treats `'1000'` as a string literal rather than a numeric value. In AWS CLI commands like `list-objects-v2` combined with JMESPath queries, numeric comparisons require unquoted numbers; a quoted string will never match a numeric field, resulting in an empty result even when objects larger than 1000 bytes exist.

Exam trap

The DEA-C01 exam often tests the subtle distinction between string and numeric comparisons in JMESPath queries, where candidates mistakenly assume that quoted numbers are automatically coerced to integers.

How to eliminate wrong answers

Option A is wrong because the question states you know objects exist in the 'logs/' prefix, so an incorrect prefix would contradict that given knowledge. Option C is wrong because the AWS S3 `list-objects-v2` command paginates by default (up to 1000 objects per page) and will continue to fetch all objects across pages unless `--max-items` is explicitly set; the empty result is not due to pagination limits. Option D is wrong because the output format (text vs.

JSON) does not affect the query logic; the JMESPath filter operates on the JSON response internally, and text output simply formats the result differently.

Practice this question →

253

Multi-Selecteasy

A company uses AWS Glue to run ETL jobs daily. The data engineer wants to reduce costs by optimizing the job configuration. Which two actions will help reduce costs? (Choose TWO.)

Select 2 answers

A.Use G.1X worker type instead of G.2X

B.Increase the job timeout to 48 hours

C.Enable Spark UI logging for debugging

D.Reduce the number of DPUs allocated to the job if the data volume is small

E.Increase the number of job retries to handle transient failures

AnswersA, D

G.1X is half the cost of G.2X.

Why this answer

G.1X workers provide 16 GB of memory and 4 vCPUs, while G.2X workers provide 32 GB and 8 vCPUs. For many ETL jobs, especially those with smaller data volumes or less complex transformations, G.1X workers are sufficient, and using them instead of G.2X directly reduces the cost per DPU-hour since AWS Glue pricing is based on DPU capacity. This optimization lowers costs without sacrificing performance if the job is not memory- or CPU-bound.

Exam trap

The trap here is that candidates often confuse cost optimization with reliability improvements, such as retries or timeouts, and may overlook that reducing worker size or DPU count directly lowers resource consumption and cost.

Practice this question →

254

MCQeasy

A company needs to ingest CSV files from an FTP server into Amazon S3 daily. The files are typically 50 MB each, and the process should be fully managed with minimal operational overhead. Which AWS service should be used?

A.AWS Lambda with FTP library

B.AWS DataSync

C.AWS Transfer Family

D.Amazon AppFlow

AnswerC

Managed FTP/SFTP service that writes directly to S3.

Why this answer

AWS Transfer Family is the correct choice because it provides a fully managed, serverless solution for transferring files to and from Amazon S3 using FTP, FTPS, or SFTP protocols. It eliminates the need to manage any FTP infrastructure, directly integrates with S3 as a destination, and handles the daily ingestion of 50 MB CSV files with minimal operational overhead, aligning perfectly with the requirement for a fully managed service.

Exam trap

The trap here is that candidates often confuse AWS DataSync as a general-purpose file transfer service, but it does not support FTP protocol natively and requires an agent, making it unsuitable for a fully managed FTP-to-S3 ingestion without infrastructure management.

How to eliminate wrong answers

Option A is wrong because AWS Lambda with an FTP library would require custom code, management of execution timeouts (Lambda has a 15-minute maximum), and handling of stateful FTP connections, which introduces significant operational overhead and is not fully managed. Option B is wrong because AWS DataSync is designed for high-speed, large-scale data transfers between on-premises storage and AWS, but it does not natively support the FTP protocol; it requires an agent installed on-premises and is optimized for bulk transfers, not simple daily FTP pulls. Option D is wrong because Amazon AppFlow supports data ingestion from SaaS applications (e.g., Salesforce, Slack) and AWS services, but it does not support FTP servers as a source, making it incompatible with the requirement.

Practice this question →

255

MCQhard

A data engineer is designing a data ingestion pipeline for IoT sensor data. The sensors send JSON messages every second. The data must be available in Amazon S3 within 5 minutes and must be transformed (JSON to Parquet) before storage. Which combination of services meets these requirements?

A.Amazon Kinesis Data Streams with AWS Glue streaming ETL

B.Amazon Kinesis Data Firehose with data transformation and Parquet conversion

C.Amazon Kinesis Data Analytics with output to S3

D.Amazon S3 with S3 Event Notifications to AWS Lambda for transformation

AnswerB

Firehose can transform and convert to Parquet before delivery.

Why this answer

Amazon Kinesis Data Firehose can ingest streaming data, apply a transformation (e.g., convert JSON to Parquet), and deliver the transformed data to Amazon S3 with a buffer interval of up to 60 seconds, easily meeting the 5-minute latency requirement. Option A is incorrect because AWS Glue streaming ETL adds complexity and is not necessary for simple JSON-to-Parquet conversion; Kinesis Data Firehose handles this natively. Option C is incorrect because Kinesis Data Analytics is designed for real-time analytics and does not directly output to S3 in a transformed format without additional components.

Option D is incorrect because S3 Event Notifications to Lambda would incur impractically high invocation costs and latency for per-second sensor data, and transforming on write to S3 would exceed the 5-minute window.

Practice this question →

256

MCQhard

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams and AWS Lambda. The Lambda function processes records and writes to Amazon DynamoDB. The engineer notices that the Lambda function is throttled during high traffic. Which action should the engineer take to reduce throttling?

A.Increase the Lambda function timeout

B.Disable retries on the Lambda function

C.Increase the number of shards in the Kinesis data stream

D.Use an Amazon SQS queue as an intermediate buffer

AnswerC

More shards allow more Lambda concurrent executions, reducing throttling.

Why this answer

Increasing the number of shards in the Kinesis data stream increases the overall throughput of the stream, which allows the Lambda event source mapping to poll more shards concurrently. Each shard is processed by one Lambda invocation at a time, so more shards mean more concurrent Lambda executions, reducing the per-invocation load and the likelihood of throttling.

Exam trap

The DEA-C01 exam often tests the misconception that throttling is caused by Lambda function performance (timeout or retries) rather than the stream's shard count, leading candidates to choose options that affect execution duration or error handling instead of scaling the source.

How to eliminate wrong answers

Option A is wrong because increasing the Lambda function timeout does not reduce throttling; it only allows the function to run longer, which does not affect the rate at which Lambda invokes the function. Option B is wrong because disabling retries on the Lambda function would cause records to be dropped or sent to a dead-letter queue, but it does not prevent throttling; throttling occurs when the concurrent execution limit is reached, not from retries. Option D is wrong because using an Amazon SQS queue as an intermediate buffer would decouple the stream from Lambda but does not directly address the root cause of throttling, which is insufficient shard count to handle the incoming data volume; SQS would add latency and complexity without increasing concurrency.

Practice this question →

257

MCQeasy

A company needs to ingest streaming data from multiple sources and store it in Amazon S3. The data volume is up to 5 GB per hour. What is the MOST cost-effective ingestion service?

A.AWS Glue

B.Amazon Kinesis Data Streams

C.Amazon Kinesis Data Firehose

D.AWS Lambda

AnswerC

Amazon Kinesis Data Firehose is fully managed, scales automatically, and charges based on data volume, making it cost-effective.

Why this answer

Amazon Kinesis Data Firehose is the most cost-effective service for ingesting streaming data into Amazon S3 at 5 GB/hour. It is fully managed, automatically scales, and charges only for data ingested (per GB), with no upfront provisioning. While Amazon Kinesis Data Streams (KDS) may have lower throughput cost for steady loads, it requires manual shard management and typically needs additional components (e.g., Lambda functions) to deliver data to S3, increasing operational overhead and total cost.

AWS Glue is a batch ETL service, not designed for streaming. AWS Lambda is a compute service and would require custom code and scaling logic, making it more expensive and complex for this use case. Therefore, Kinesis Data Firehose provides the simplest and most cost-effective solution for streaming data ingestion directly to S3.

Practice this question →

258

MCQmedium

A company uses AWS Glue to process CSV files stored in Amazon S3. The data pipeline runs daily, but recently some jobs have failed with a 'MemoryError'. The data volume has grown from 1 GB to 10 GB per day. What is the MOST cost-effective solution to resolve this issue?

A.Change the Glue worker type from Standard to G.2X.

B.Increase the number of DPUs (Data Processing Units) allocated to the Glue job.

C.Convert the CSV files to Parquet format using an S3 batch operation.

D.Migrate the job to Amazon EMR with a larger cluster.

AnswerB

More DPUs provide more memory and processing power.

Why this answer

Increasing the number of DPUs allocates more memory and processing capacity to the Glue job, directly addressing the MemoryError caused by data growth from 1 GB to 10 GB. Option A is wrong because changing the worker type to G.2X provides more memory per worker, but it is a more expensive option compared to simply increasing DPUs, which scales horizontally. Option C is wrong because converting CSV to Parquet improves performance and reduces storage but does not add memory to the Glue job; the job still runs with the same DPU allocation.

Option D is wrong because migrating to Amazon EMR introduces additional operational complexity and cost; increasing DPUs in Glue is simpler and more cost-effective for scaling an existing job.

Practice this question →

259

MCQeasy

A company is designing a data ingestion pipeline to load CSV files from an SFTP server into Amazon S3. The files are generated hourly and range from 10 MB to 500 MB. Which AWS service should be used to orchestrate the transfer with minimal operational overhead?

A.AWS Glue

B.Amazon AppFlow

C.AWS Transfer Family

D.AWS DataSync

AnswerC

AWS Transfer Family provides managed SFTP with automatic uploads to S3.

Why this answer

AWS Transfer Family is the correct choice because it provides a fully managed, serverless SFTP endpoint that can directly receive files from an SFTP server and automatically store them in Amazon S3. This eliminates the need to manage any compute infrastructure or write custom code for the transfer, minimizing operational overhead for hourly CSV file ingestion.

Exam trap

The trap here is that candidates often confuse AWS DataSync (which requires an on-premises agent and does not support SFTP) with Transfer Family, or they mistakenly think AWS Glue can handle SFTP ingestion because it supports custom connectors, but Glue is not designed for real-time file transfer orchestration.

How to eliminate wrong answers

Option A is wrong because AWS Glue is a serverless data integration service primarily for ETL (extract, transform, load) jobs, not for orchestrating file transfers from an SFTP server; it lacks native SFTP connectors and would require custom scripts or additional services to handle the transfer. Option B is wrong because Amazon AppFlow supports data ingestion from SaaS applications (e.g., Salesforce, Slack) and does not support SFTP as a source, so it cannot be used to pull files from an SFTP server. Option D is wrong because AWS DataSync is designed for large-scale, recurring data transfers between on-premises storage and AWS, but it requires installing an agent on the on-premises network and does not natively support SFTP as a source protocol; it is optimized for NFS/SMB, not SFTP.

Practice this question →

260

MCQeasy

A data pipeline uses AWS Glue to process data from an S3 data lake. The pipeline fails intermittently with a 'ThrottlingException' when writing to a DynamoDB table. What is the MOST likely cause?

A.The DynamoDB table's write capacity is insufficient for the workload.

B.The network connection between Glue and DynamoDB is unstable.

C.The Glue job's timeout setting is too low.

D.The Glue job does not have sufficient IAM permissions to write to DynamoDB.

AnswerA

ThrottlingException indicates the write capacity is exceeded; increasing capacity or using auto-scaling resolves it.

Why this answer

A ThrottlingException from DynamoDB indicates that the request rate to the table has exceeded the provisioned write capacity. AWS Glue jobs can generate high-throughput writes, and if the DynamoDB table's write capacity units (WCUs) are not sufficient to handle the burst, DynamoDB will throttle the requests. This is the most direct cause of the intermittent failure described.

Exam trap

The trap here is that candidates may confuse ThrottlingException with permission errors (Option D) or network issues (Option B), but AWS specifically tests the understanding that DynamoDB throttling is a capacity management mechanism, not a connectivity or authorization problem.

How to eliminate wrong answers

Option B is wrong because network instability between Glue and DynamoDB would typically result in connection timeouts or retryable network errors, not a specific ThrottlingException which is an application-level error from DynamoDB's API. Option C is wrong because a Glue job's timeout setting controls how long the job can run before being terminated, not how it handles individual API throttling errors; a timeout would cause a different error (e.g., 'Timeout exceeded'). Option D is wrong because insufficient IAM permissions would result in an AccessDeniedException, not a ThrottlingException; the error message directly indicates capacity limits, not authorization failures.

Practice this question →

261

Multi-Selectmedium

A company is building a data lake on Amazon S3 and needs to ingest data from multiple sources. The ingestion must be automated and handle schema changes. Which THREE services can be used together to achieve this? (Choose THREE.)

Select 3 answers

A.Amazon Redshift

B.AWS Glue Crawler

C.Amazon Kinesis Data Firehose

D.Amazon EMR

E.AWS Lambda

AnswersB, C, E

Glue Crawler can discover schema and update the Data Catalog.

Why this answer

AWS Glue Crawler (B) automatically discovers and catalogs schemas from data sources. Amazon Kinesis Data Firehose (C) ingests streaming data into S3. AWS Lambda (E) can transform data on the fly.

Together, they automate ingestion and handle schema changes. Amazon Redshift (A) is a data warehouse, not a data lake ingestion service. Amazon EMR (D) is a big data processing framework, not primarily for automated schema change handling.

Practice this question →

262

MCQeasy

A company stores IoT sensor data in S3 as JSON files. They need to convert the data to Parquet format for efficient querying with Amazon Athena. Which AWS service can perform this transformation with minimal effort?

A.Kinesis Data Firehose

B.Amazon Athena

C.AWS Glue ETL job

D.AWS Lambda

AnswerC

Glue ETL can convert JSON to Parquet.

Why this answer

AWS Glue ETL jobs can easily convert JSON to Parquet with built-in transforms. Option A is wrong because Kinesis Data Firehose is for streaming data ingestion, not batch transformations. Option B is wrong because Amazon Athena is a query engine, not a transformation service.

Option D is wrong because AWS Lambda is for small, event-driven transformations and is not ideal for large-scale batch conversion.

Practice this question →

263

Multi-Selecteasy

A data engineer needs to ingest JSON files from an Amazon S3 bucket into an Amazon DynamoDB table. The files are uploaded every hour. Which THREE services can be used together to build this ingestion pipeline?

Select 3 answers

A.AWS Step Functions

B.Amazon SQS

C.Amazon DynamoDB Streams

D.Amazon S3 Event Notifications

E.AWS Lambda

AnswersB, D, E

SQS can decouple S3 events from Lambda for reliability.

Why this answer

Amazon SQS is correct because it decouples the ingestion pipeline, allowing S3 Event Notifications to send messages to an SQS queue when new JSON files arrive. AWS Lambda can then poll the SQS queue to process the files and write to DynamoDB, ensuring reliable, asynchronous ingestion without data loss.

Exam trap

The trap here is that candidates often confuse DynamoDB Streams (for capturing table changes) with the ingestion pipeline itself, or incorrectly assume Step Functions is needed for simple event-driven workflows, when SQS+Lambda is the standard serverless pattern for this use case.

Practice this question →

264

MCQeasy

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data is then consumed by a custom application for real-time analytics. Recently, the application has been experiencing high latency. The operations team suspects the shard count is insufficient. How should the team increase the shard count of the existing stream?

A.Use the UpdateShardCount API to increase the shard count for the stream.

B.Delete the existing stream and create a new one with a higher shard count.

C.Manually split a shard using the SplitShard API on each existing shard.

D.Modify the PutRecord calls to include a new shard key that distributes data across more shards.

AnswerA

UpdateShardCount correctly increases shards.

Why this answer

The UpdateShardCount API is the correct method to increase the shard count of an existing Kinesis Data Stream without data loss or downtime. It allows you to specify a target shard count, and Kinesis automatically splits shards to achieve that count, redistributing the hash key range across the new shards. This directly addresses the high latency caused by insufficient shard count by increasing the stream's throughput capacity.

Exam trap

The trap here is that candidates might think manually splitting shards (Option C) is the only way to increase shard count, but the UpdateShardCount API is the designed, automated method that avoids the complexity and risk of manual splits.

How to eliminate wrong answers

Option B is wrong because deleting and recreating the stream would cause data loss and downtime, which is unnecessary when the UpdateShardCount API can dynamically scale the existing stream. Option C is wrong because manually splitting each shard using the SplitShard API is not the recommended approach for increasing the overall shard count; it requires careful planning of hash key ranges and is error-prone, whereas UpdateShardCount automates the process. Option D is wrong because modifying PutRecord calls to include a new shard key does not increase the shard count; it only changes how data is distributed among existing shards, which does not solve the throughput bottleneck.

Practice this question →

265

MCQmedium

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data must be transformed (e.g., enrich with user location) before being stored in Amazon S3. Which architecture is MOST efficient for this transformation?

A.Use AWS Glue to run a streaming ETL job.

B.Use Amazon EMR to consume the stream using Spark Streaming.

C.Use AWS Lambda to process each record from the stream and write to S3.

D.Use Amazon Kinesis Data Analytics to transform the stream and output to Amazon Kinesis Data Firehose, which writes to S3.

AnswerD

Kinesis Data Analytics can run SQL on the stream, and Firehose delivers to S3 in batches.

Why this answer

Amazon Kinesis Data Analytics (KDA) can perform real-time transformations (e.g., enriching clickstream data with user location via SQL or Flink) on the stream, then output the transformed data to Kinesis Data Firehose, which can batch and compress records before writing to S3. This architecture minimizes operational overhead and is purpose-built for streaming transformations, avoiding the latency and complexity of Lambda cold starts or the provisioning overhead of Glue/EMR.

Exam trap

The trap here is that candidates often choose AWS Lambda (Option C) because it seems serverless and simple, but they overlook Lambda's lack of native batching to S3 and its 15-minute timeout, which makes it inefficient for continuous, high-volume streaming transformations compared to KDA + Firehose.

How to eliminate wrong answers

Option A is wrong because AWS Glue streaming ETL jobs are designed for batch-oriented transformations and incur higher startup latency and cost compared to KDA for simple per-record enrichments. Option B is wrong because Amazon EMR with Spark Streaming requires managing a persistent cluster, which adds operational complexity and cost for a continuous, low-latency transformation that could be handled serverlessly. Option C is wrong because AWS Lambda has a maximum invocation duration of 15 minutes and is not ideal for high-throughput, sustained streaming transformations; it also lacks native integration with Kinesis Data Firehose for batching and compression to S3, leading to excessive S3 PUT requests and higher costs.

Practice this question →

266

MCQmedium

A company is using AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job fails intermittently with a 'MemoryError' when the stream has a sudden spike in data volume. Which configuration change would best prevent this error?

A.Increase the number of DPUs (Data Processing Units) for the Glue job.

B.Store intermediate results in Amazon RDS.

C.Use a batch transformation instead of streaming.

D.Increase the number of shards in the Kinesis data stream.

AnswerA

Increasing DPUs adds more memory and compute capacity to the Glue job, directly addressing the MemoryError during data spikes.

Why this answer

Increasing the number of DPUs in the AWS Glue job provides more memory and compute capacity to handle data spikes. Option B is wrong because storing intermediate results in Amazon RDS does not prevent memory errors in Glue; it introduces a database dependency and does not increase Glue's memory. Option C is wrong because switching to batch transformation is not a solution for a streaming job; the job is designed for streaming and batch does not address the memory issue.

Option D is wrong because increasing the number of shards in Kinesis increases throughput but does not directly solve memory errors in Glue; it may even increase the data volume per unit time and worsen the problem.

Practice this question →

267

MCQeasy

A company uses AWS Glue ETL jobs to transform data in Amazon S3. The data arrives in JSON format but needs to be converted to Parquet for efficient querying. Which AWS Glue feature should be used to infer the schema and generate transformation code?

A.Amazon S3 Select

B.Amazon Athena

C.Amazon Kinesis Data Analytics

D.AWS Glue crawlers

AnswerD

Crawlers populate the Data Catalog with schema information used by Glue ETL jobs.

Why this answer

AWS Glue crawlers are the correct feature because they automatically connect to data stores (like S3), infer the schema of JSON data by sampling it, and populate the AWS Glue Data Catalog with table definitions. This catalog schema can then be used by AWS Glue ETL jobs to generate transformation code (e.g., converting JSON to Parquet) without manual schema definition.

Exam trap

The trap here is that candidates confuse AWS Glue crawlers with Amazon Athena or S3 Select, assuming any query or analysis tool can infer schemas for ETL, but only crawlers are designed to automatically discover and catalog schemas for Glue ETL jobs.

How to eliminate wrong answers

Option A is wrong because Amazon S3 Select is a query-in-place service that retrieves subsets of data from S3 objects using SQL, but it does not infer schemas or generate ETL transformation code. Option B is wrong because Amazon Athena is an interactive query service that uses SQL to analyze data directly in S3, but it does not generate ETL transformation code or automatically infer schemas for Glue ETL jobs (though it can query Glue Data Catalog tables). Option C is wrong because Amazon Kinesis Data Analytics processes streaming data in real time using SQL or Apache Flink, not batch transformation of JSON to Parquet in S3, and it does not infer schemas for Glue ETL jobs.

Practice this question →

268

MCQeasy

A company wants to transform data in Amazon S3 using SQL queries without provisioning servers. The transformations are ad-hoc and run occasionally. Which service should be used?

A.AWS Glue

B.Amazon Redshift Spectrum

C.Amazon EMR

D.Amazon Athena

AnswerD

Athena is serverless and supports SQL queries directly on S3 data.

Why this answer

Amazon Athena is the correct choice because it enables serverless, ad-hoc SQL querying directly on data stored in Amazon S3 without requiring any infrastructure provisioning. Since the transformations are occasional and ad-hoc, Athena's pay-per-query model and zero setup overhead align perfectly with the requirement.

Exam trap

The trap here is that candidates often confuse AWS Glue's ability to run SQL via Spark SQL or Athena as a transformation engine, but Glue requires provisioning resources and is not designed for ad-hoc serverless SQL queries.

How to eliminate wrong answers

Option A is wrong because AWS Glue is primarily an ETL service that requires provisioning and managing crawlers, jobs, and triggers, and is designed for scheduled or event-driven batch transformations, not lightweight ad-hoc SQL queries. Option B is wrong because Amazon Redshift Spectrum extends Redshift's query capabilities to S3 but still requires a provisioned Redshift cluster to run, which violates the 'without provisioning servers' constraint. Option C is wrong because Amazon EMR is a managed big data platform that requires provisioning EC2 instances and configuring clusters, making it unsuitable for occasional, serverless SQL queries.

Practice this question →

269

MCQmedium

A company is using AWS DMS to migrate a 5 TB SQL Server database to Amazon Aurora PostgreSQL. The migration is using full load plus CDC. After the full load completes, the ongoing replication task is failing with errors related to large transactions on the source. The team needs to ensure that CDC continues without falling behind. What should the team do?

A.Use Amazon Kinesis Data Streams as an intermediate target for CDC.

B.Increase the DMS replication instance size to provide more memory and CPU.

C.Modify the DMS task settings to increase MaxFileSize and decrease the CommitRate.

D.Disable foreign key constraints on the target Aurora database.

AnswerB

Increasing the DMS replication instance size provides more memory and CPU, allowing the instance to process large transactions more efficiently and keep up with CDC changes without falling behind.

Why this answer

Increasing the DMS replication instance size provides more memory and CPU, enabling the instance to process large transactions more efficiently and keep up with CDC changes without falling behind. Option C (modifying MaxFileSize and decreasing CommitRate) is not the best solution because decreasing CommitRate means less frequent commits, which could worsen the problem by accumulating more data per commit. The more direct approach is to scale the instance vertically to handle the load.

Practice this question →

270

MCQhard

A data engineering team uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. They notice that the application's checkpointing is failing intermittently, causing data reprocessing. The application uses a large state. Which configuration change should the team make to improve checkpoint reliability?

A.Disable checkpointing to avoid failures.

B.Switch the state backend from in-memory to RocksDB.

C.Increase the parallelism of the application.

D.Increase the checkpointing interval.

AnswerD

Longer intervals reduce checkpoint frequency and associated failures.

Why this answer

Increasing the checkpointing interval reduces the frequency of checkpoint operations, giving the system more time to complete each checkpoint before the next one starts. This alleviates backpressure and resource contention, which is critical when dealing with large state, as checkpointing large state is I/O and CPU intensive and can fail if intervals are too tight.

Exam trap

The trap here is that candidates often confuse improving state backend performance (RocksDB) with fixing checkpoint reliability, when the root cause is checkpoint timing pressure, not state storage efficiency.

How to eliminate wrong answers

Option A is wrong because disabling checkpointing eliminates fault tolerance entirely, which would cause data loss on failure and is not a valid reliability improvement. Option B is wrong because switching to RocksDB improves state storage efficiency and reduces memory pressure, but it does not directly address checkpoint failures caused by overly frequent checkpointing; RocksDB can even increase checkpoint duration due to disk I/O. Option C is wrong because increasing parallelism distributes workload but also increases the number of concurrent checkpoint operations and network overhead, potentially worsening checkpoint failures when state is large.

Practice this question →

271

MCQeasy

An e-commerce company wants to capture clickstream data from its website and store it in Amazon S3 for analytics. The data arrives continuously and the company needs near-real-time processing. Which solution is most appropriate?

A.AWS Data Pipeline

B.AWS Snowball Edge

C.Amazon Kinesis Data Firehose

D.Amazon S3 Transfer Acceleration

AnswerC

Firehose captures streaming data and delivers to S3 with low latency.

Why this answer

Amazon Kinesis Data Firehose is the most appropriate solution because it is a fully managed service designed to ingest streaming data and deliver it to destinations like Amazon S3 with near-real-time latency. The company needs continuous clickstream capture and near-real-time processing, which Firehose provides. Option A (AWS Data Pipeline) is for batch processing, not streaming.

Option B (AWS Snowball Edge) is for offline data transfer, not real-time. Option D (S3 Transfer Acceleration) improves upload speed but is not a streaming ingestion service.

Practice this question →

272

MCQhard

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is compressed with GZIP and partitioned by year, month, day, and hour. The delivery stream is configured to buffer up to 5 MB or 60 seconds. Some records are missing from S3. What is the most likely cause?

A.The S3 bucket does not have sufficient permissions

B.The data compression format is incompatible with S3

C.The Lambda transformation function timed out and records were skipped

D.The partition key configuration is incorrect

AnswerC

Firehose drops records if the transformation Lambda exceeds the timeout.

Why this answer

When a Lambda transformation function times out, Kinesis Data Firehose will skip the affected records by default. The delivery stream configuration (5 MB or 60 seconds) only controls buffering, not Lambda invocation failures. If the Lambda function exceeds its timeout limit, Firehose treats the invocation as failed and, depending on the error handling configuration, may drop the records entirely, leading to missing data in S3.

Exam trap

The trap here is that candidates often assume buffering settings (5 MB or 60 seconds) guarantee delivery, but they overlook that Lambda transformation failures can silently drop records without explicit error logging unless CloudWatch monitoring is set up.

How to eliminate wrong answers

Option A is wrong because insufficient S3 bucket permissions would cause delivery failures or error logs, not selective missing records; Firehose would report a permission error in CloudWatch Logs. Option B is wrong because GZIP compression is fully compatible with S3 and is a standard compression format supported by Firehose for S3 delivery. Option D is wrong because the partition key configuration (year/month/day/hour) is correctly defined and does not cause record loss; incorrect partitioning would only affect the folder structure, not the presence of records.

Practice this question →

273

MCQhard

Refer to the exhibit. A data engineer is configuring an IAM policy for a Lambda function that writes transformed data to S3. The function writes to both 'example-bucket/data/' and 'example-bucket/public/'. The policy is intended to enforce server-side encryption with SSE-S3 for all objects written to the 'public/' prefix, while allowing all operations on other prefixes. However, the Lambda function is failing with an AccessDenied error when writing to 'example-bucket/public/'. What is the most likely cause?

A.The policy denies DeleteObject on 'public/'.

B.The policy denies PutObject on 'public/' unconditionally.

C.The policy does not allow GetObject for 'public/'.

D.The Lambda function is not setting the 'x-amz-server-side-encryption' header to 'AES256' when writing to 'public/'.

AnswerD

The Deny condition requires AES256 encryption.

Why this answer

The policy enforces SSE-S3 encryption for objects written to the 'public/' prefix. When a Lambda function writes to S3 without setting the 'x-amz-server-side-encryption' header to 'AES256', the request fails with an AccessDenied error if the bucket policy requires SSE-S3. The policy explicitly denies PutObject unless the encryption header is present, so the function must include this header to succeed.

Exam trap

The DEA-C01 exam often tests the nuance that bucket policies can conditionally deny operations based on request headers, and candidates mistakenly think the error is due to missing IAM permissions rather than a missing encryption header.

How to eliminate wrong answers

Option A is wrong because the error is about writing (PutObject), not deleting (DeleteObject), and the policy focuses on encryption enforcement, not delete permissions. Option B is wrong because the policy does not unconditionally deny PutObject; it denies PutObject only when the required SSE-S3 encryption header is missing, which is a conditional denial. Option C is wrong because GetObject is not relevant to the write operation failing; the error occurs during PutObject, and the policy does not restrict read access for 'public/'.

Practice this question →

274

Multi-Selecteasy

A data engineer is designing a data ingestion pipeline for real-time clickstream data. Which TWO services can be used to ingest the data into Amazon Kinesis Data Streams?

Select 2 answers

A.Amazon S3

B.Kinesis Producer Library (KPL)

C.Kinesis Data Firehose

D.AWS SDK

E.AWS Glue

AnswersB, D

KPL is designed to send data to Kinesis Data Streams efficiently.

Why this answer

Options B and D are correct. The Kinesis Producer Library (KPL) is a library for producers to send data to Kinesis Data Streams. AWS SDK can also be used directly.

Option A is wrong because Amazon S3 is a storage service, not a producer. Option C is wrong because Kinesis Data Firehose is a downstream consumer or delivery service, not a producer. Option E is wrong because AWS Glue is an ETL service, not a producer.

Practice this question →

275

MCQeasy

A company needs to ingest data from an on-premises MySQL database into Amazon S3 for analytics. The database is 2 TB in size. The company has a low-bandwidth internet connection (10 Mbps). They need to perform an initial full load and then incremental updates every hour. Which approach should they use?

A.Use Kinesis Data Firehose to stream data from MySQL to S3.

B.Use AWS Database Migration Service (DMS) to perform the full load and ongoing replication.

C.Use AWS Glue ETL jobs to extract data and load into S3.

D.Use AWS Snowball Edge to transfer the initial full load, then use AWS DataSync for incremental updates.

AnswerB

AWS DMS can perform a full 2 TB load from MySQL to S3 even over a 10 Mbps link because it uses change data capture (CDC) to track incremental changes after the initial load, enabling hourly updates without re-scanning the entire source. This satisfies the low-bandwidth constraint by minimising repeated data transfer.

Why this answer

AWS Database Migration Service (DMS) supports full load and ongoing replication, and can be used with limited bandwidth. Option A is wrong because Kinesis Data Firehose is for streaming data, not database replication. Option C is wrong because Glue ETL is not optimized for continuous replication.

Option D is wrong because Snowball Edge is for offline transfer, not ongoing replication.

Practice this question →

276

MCQmedium

Refer to the exhibit. A data engineer is troubleshooting a Kinesis Data Streams consumer that is falling behind. The stream has 2 shards and is receiving data at a rate of 2 MB/s. The consumer is an AWS Lambda function with a batch size of 100 records. What should the engineer do to improve consumer throughput?

A.Decrease the Lambda batch size to 10 records

B.Increase the retention period of the stream to 168 hours

C.Increase the number of shards in the stream to 4

D.Increase the memory allocation of the Lambda function

AnswerC

More shards increase parallelism and throughput for both producers and consumers.

Why this answer

Increasing the number of shards to 4 doubles the stream's total ingestion capacity to 4 MB/s, which directly increases the number of concurrent Lambda invocations and thus consumer throughput. With 2 shards, each shard can support up to 1 MB/s input and 2 MB/s output, so the current 2 MB/s load is at the shard-level output limit, causing the consumer to fall behind.

Exam trap

The DEA-C01 exam often tests the misconception that Lambda memory or batch size adjustments are the primary levers for throughput, when in fact the shard count directly controls the parallelism and read capacity of a Kinesis stream consumer.

How to eliminate wrong answers

Option A is wrong because decreasing the batch size reduces the number of records processed per invocation, which increases the number of Lambda invocations and overhead, potentially worsening throughput rather than improving it. Option B is wrong because increasing the retention period (up to 365 days) only affects how long records are stored in the stream, not the rate at which the consumer can read or process data. Option D is wrong because while increasing Lambda memory can improve CPU performance, the bottleneck here is the shard-level read throughput limit (2 MB/s per shard for the consumer), not Lambda compute capacity.

Practice this question →

277

MCQhard

A company is ingesting streaming data from multiple sources using Amazon Kinesis Data Streams. The data is then processed by an AWS Lambda function that transforms the records and writes them to an Amazon S3 bucket. The Lambda function is failing intermittently with timeout errors. The average record size is 5 KB, and the shard count is 2. What is the MOST likely cause of the timeout errors?

A.The Lambda function timeout is set too low for the processing time required.

B.The Kinesis data retention period is too short, causing data to be lost before processing.

C.The Lambda function's reserved concurrency is set too low, causing throttling.

D.The Lambda function is receiving too many records per invocation, exceeding the 6 MB payload limit.

AnswerA

The default Lambda timeout is 3 seconds, which may not be sufficient for processing each batch of records and writing to S3.

Why this answer

The Lambda function is timing out, indicating that the configured timeout is insufficient for the actual processing time. Lambda has a default timeout of 3 seconds, but it can be set from 1 second to 15 minutes. If the transformation logic or S3 write operation takes longer than the configured timeout, the function will fail with a timeout error.

Option B is incorrect because the Kinesis data retention period (default 24 hours) affects data availability, not Lambda execution time. Option C is incorrect because reserved concurrency controls the number of concurrent invocations and can cause throttling, not timeouts. Option D is incorrect because with an average record size of 5 KB, even the default batch size of 100 records results in only 500 KB per invocation, well below the 6 MB payload limit.

Practice this question →

278

Multi-Selecthard

A company uses Amazon RDS for MySQL as a source for AWS DMS to replicate data to S3. The replication task is failing with 'OutOfMemory' errors on the DMS instance. The source table has 10 million rows with large BLOB columns. Which THREE changes would most likely resolve the issue?

Select 3 answers

A.Set the LOB column settings to 'Limited LOB mode' and specify a max LOB size.

B.Disable logging for the DMS task to free memory.

C.Enable Full LOB mode to handle LOBs more efficiently.

D.Increase the DMS replication instance size to a compute-optimized class.

E.Increase the number of parallel threads in the task settings.

AnswersA, D, E

Limited LOB mode avoids loading entire LOBs into memory.

Why this answer

Setting LOB columns to 'Limited LOB mode' with a specified max LOB size prevents DMS from loading entire LOBs into memory. Instead, DMS truncates LOBs to the specified size, reducing memory consumption and avoiding OutOfMemory errors when replicating large BLOB columns from MySQL to S3.

Exam trap

The trap here is that candidates often assume Full LOB mode is always the safest choice for large objects, but it actually increases memory usage and can cause OutOfMemory errors, whereas Limited LOB mode with a max size is the correct memory-saving approach.

Practice this question →

279

MCQeasy

Refer to the exhibit. A data engineer runs this AWS Glue Data Catalog DDL statement to create a table. The CSV files in 's3://my-bucket/sales/' use a pipe delimiter (|) instead of a comma. What change is needed to correctly read the data?

A.Change the 'field.delim' property to '|'.

B.Change the LOCATION to read from a subfolder.

C.Add a partition projection configuration.

D.Run a crawler to detect the schema automatically.

AnswerA

The delimiter must match the actual file format.

Why this answer

The AWS Glue Data Catalog DDL statement uses the default 'field.delim' property, which expects comma-separated values. Since the CSV files use a pipe delimiter (|), the table will not parse rows correctly. Setting 'field.delim' to '|' in the SerDe properties tells the Hive-compatible SerDe to split on pipes instead of commas, enabling correct data ingestion.

Exam trap

The DEA-C01 exam often tests the misconception that changing the LOCATION or adding partition projection will fix parsing issues, when in fact the core problem is the SerDe delimiter property not matching the actual file format.

How to eliminate wrong answers

Option B is wrong because changing the LOCATION to a subfolder does not alter the delimiter interpretation; it only changes the source path, leaving the parsing issue unresolved. Option C is wrong because partition projection configuration optimizes partition pruning for partitioned tables, but it does not affect how individual records are parsed within files. Option D is wrong because running a crawler would detect the schema and delimiter automatically, but the question explicitly asks what change is needed to the given DDL statement, and a crawler is an alternative approach, not a modification to the existing DDL.

Practice this question →

280

MCQeasy

A data engineer needs to transform JSON data into CSV format using AWS Glue. The transformation is simple and must be executed on a schedule. Which Glue component is MOST suitable?

A.Glue Crawler

B.Glue Data Catalog

C.Glue Development Endpoint

D.Glue ETL job

AnswerD

Glue ETL jobs run transformations and can be scheduled.

Why this answer

A Glue ETL job is the most suitable component because it can execute a script (Python/Scala) to transform JSON data into CSV format and can be scheduled to run on a recurring basis. Glue Crawlers only discover and catalog metadata, not transform data. The Glue Data Catalog is a metadata repository, not a transformation tool.

A Glue Development Endpoint is used for interactive development and testing, not for scheduled production jobs.

Practice this question →

281

Multi-Selectmedium

An e-commerce company is building a near-real-time dashboard to monitor customer clickstream data. The data is ingested via Amazon Kinesis Data Streams, transformed using AWS Lambda, and stored in Amazon S3. The team needs to query the data using Amazon Athena. Which THREE steps should be taken to optimize cost and performance? (Choose three.)

Select 3 answers

A.Use AWS Glue Data Catalog to store the table metadata.

B.Store the data in JSON format for flexibility.

C.Convert the data to Apache Parquet or ORC format.

D.Compress the data using gzip or snappy.

E.Partition the data by date in S3 (e.g., year/month/day).

AnswersC, D, E

Columnar formats reduce data scanned and improve compression.

Why this answer

To optimize cost and performance when querying data with Athena, use columnar formats like Parquet or ORC (C) to reduce data scanned and improve compression. Compress data with gzip or Snappy (D) to reduce storage costs and data transferred during queries. Partition data by date (E) to limit the amount of data scanned per query.

Option A (Glue Data Catalog) is a prerequisite, not an optimization step. Option B (JSON) is less efficient than columnar formats for analytical queries.

Exam trap

A common trap is to consider AWS Glue Data Catalog as an optimization step, but it is merely a requirement; the actual optimizations are compression, partitioning, and columnar formats.

Practice this question →

282

MCQeasy

A small startup is building a data pipeline to ingest customer orders from a web application into Amazon Redshift for analytics. The orders are written to an Amazon RDS MySQL database. The startup wants to replicate the orders to Redshift in near-real time (within 5 minutes) with minimal operational overhead. The data volume is low, averaging 100 new orders per minute. The startup has a single data engineer who is also responsible for other tasks. What is the simplest solution?

A.Use AWS Glue with a scheduled job every 5 minutes to copy data from MySQL to Redshift

B.Use Amazon EMR with Spark streaming to read from MySQL and write to Redshift

C.Use an AWS Lambda function to query MySQL every minute and insert into Redshift

D.Use AWS Database Migration Service (DMS) with continuous replication

AnswerD

DMS is purpose-built for database replication and easy to set up.

Why this answer

AWS DMS can continuously replicate from MySQL to Redshift with minimal setup and low overhead. Option A (AWS Glue) is batch-oriented and may not meet the 5-minute latency. Option B (Amazon EMR) is overkill for low data volumes.

Option C (AWS Lambda) requires custom code and may not efficiently handle the replication.

Practice this question →

283

MCQmedium

A data engineer is troubleshooting a Kinesis Data Firehose delivery stream that is failing to deliver data to an Amazon S3 bucket. The stream is configured with a Lambda transformation function. The CloudWatch logs show that the Lambda function is timing out. Which action should the engineer take to resolve the issue?

A.Reduce the Firehose buffer interval.

B.Increase the Lambda function timeout setting.

C.Decrease the Lambda function's batch size in Firehose.

D.Increase the memory allocated to the Lambda function.

AnswerB

Extending timeout allows more time for processing.

Why this answer

The CloudWatch logs indicate the Lambda function is timing out. The default Lambda timeout for a Firehose transformation is 60 seconds, and if the function's processing exceeds this limit, it will fail. Increasing the Lambda function timeout setting (Option B) directly addresses this by allowing the function more time to complete its execution before being terminated.

Exam trap

The trap here is that candidates often confuse a Lambda timeout with a performance issue and immediately choose to increase memory (Option D) or reduce batch size (Option C), when the correct first step is to increase the timeout setting as indicated by the specific CloudWatch error.

How to eliminate wrong answers

Option A is wrong because reducing the Firehose buffer interval does not affect the Lambda function's execution time; it only causes Firehose to send smaller batches more frequently, which could increase the number of invocations but not resolve a timeout. Option C is wrong because decreasing the Lambda function's batch size reduces the number of records per invocation, which might reduce processing time per invocation, but the root cause is the function's timeout setting, not the batch size; a smaller batch size may not fix the timeout if the function itself is slow. Option D is wrong because increasing memory allocated to the Lambda function can improve CPU performance and reduce execution time, but the immediate and direct fix for a timeout error is to increase the timeout setting; memory adjustments are a secondary optimization.

Practice this question →

284

Multi-Selectmedium

A company is using AWS Glue to process data from an Amazon S3 data lake. The Glue job runs daily and transforms data into multiple output formats. Which TWO actions can the company take to optimize the Glue job's performance and reduce costs? (Choose TWO.)

Select 2 answers

A.Increase the number of DPUs allocated to the job.

B.Reduce the number of DPUs (Data Processing Units) allocated to the job.

C.Disable job bookmarking to force full reprocessing every run.

D.Increase the job timeout to allow more time for processing.

E.Enable job bookmarking to process only new data.

AnswersA, E

More DPUs can speed up processing, reducing runtime and possibly cost.

Why this answer

Options A and E are correct. Increasing the number of DPUs (A) can parallelize processing and improve performance for data-intensive Glue jobs, potentially reducing runtime and cost if the job runs faster. Enabling job bookmarking (E) allows Glue to track processed data and process only new or changed data in incremental runs, reducing processing time and cost by avoiding full reprocessing.

Option B (reducing DPUs) would likely decrease performance. Option C (disabling bookmarking) would force full reprocessing, increasing cost and time. Option D (increasing job timeout) does not optimize performance or cost; it only allows the job to run longer, which could increase cost if the job is inefficient.

Practice this question →

285

MCQhard

A company ingests millions of small files (1-10 KB) into Amazon S3 every hour. These files are then processed by AWS Glue ETL jobs. The Glue jobs are slow because of the overhead of reading many small files. Which strategy will most effectively improve Glue job performance?

A.Enable Glue job bookmark.

B.Increase the number of DPUs for the Glue job.

C.Use S3 Select to filter data before Glue reads it.

D.Use a Lambda function to merge small files into larger ones before Glue processes them.

AnswerD

Merging files reduces the number of objects, speeding up Glue's list and read operations.

Why this answer

Grouping small files into larger ones (e.g., by merging in a preprocessing step) reduces the number of file read operations and improves Glue's efficiency. Using S3 Select or increasing DPUs helps but doesn't address the root cause.

Practice this question →

286

Multi-Selectmedium

A company is using Amazon Kinesis Data Streams to process real-time stock trade data. The data is consumed by a Lambda function that calculates moving averages and stores results in Amazon DynamoDB. The Lambda function is failing with 'ProvisionedThroughputExceededException' on the DynamoDB table. The table has on-demand capacity. Which TWO actions should the engineer take to resolve this issue?

Select 2 answers

A.Add a dead-letter queue and configure the Lambda function to retry on failure with exponential backoff.

B.Decrease the batch window to 0 seconds to process records immediately.

C.Increase the Lambda function's reserved concurrency to process more shards.

D.Increase the batch size of the Kinesis event source mapping for the Lambda function.

AnswersA, D

Retries with backoff help handle throttling gracefully.

Why this answer

Adding a dead-letter queue (DLQ) and configuring the Lambda function to retry on failure with exponential backoff allows the function to handle transient ProvisionedThroughputExceededExceptions from DynamoDB. Since the table uses on-demand capacity, the exception indicates a momentary throttle due to traffic spikes; exponential backoff with retries gives DynamoDB time to scale up, while the DLQ captures records that persistently fail for later analysis.

Exam trap

The trap here is that candidates may think increasing concurrency (Option C) helps with DynamoDB throttling, but it actually increases write pressure, while the correct approach is to reduce the request rate via batching (Option D) and handle retries with exponential backoff (Option A).

Practice this question →

287

MCQmedium

A data engineering team is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data is then consumed by an AWS Lambda function that transforms and loads it into Amazon S3. Recently, the team noticed that the Lambda function is failing with throttling errors (HTTP 429) from the Kinesis API. Which configuration change should the team make to resolve this issue?

A.Disable retries on the Lambda function and configure a dead-letter queue for failed records.

B.Replace Kinesis Data Streams with Amazon DynamoDB Streams for ingestion.

C.Reduce the batch size and increase the number of shards in the Kinesis stream to increase parallelism.

D.Increase the batch size in the Lambda event source mapping to reduce the number of invocations.

AnswerC

Reducing batch size lowers records per invocation, and more shards increase parallelism, reducing throttling.

Why this answer

Reducing the batch size and increasing the number of shards directly addresses the HTTP 429 throttling errors from the Kinesis API. Each shard supports up to 5 read transactions per second and a maximum read rate of 2 MB/s; by increasing shards, you increase the available read throughput, and reducing the batch size lowers the number of records per invocation, preventing the Lambda function from exceeding the per-shard read limits.

Exam trap

The trap here is that candidates often assume increasing the batch size reduces invocations and thus throttling, but in reality, larger batches increase the data volume per GetRecords call, making throttling worse; the correct approach is to reduce batch size and increase shards to distribute the read load.

How to eliminate wrong answers

Option A is wrong because disabling retries would cause data loss for failed records; a dead-letter queue captures failures but does not resolve the root cause of throttling from the Kinesis API. Option B is wrong because replacing Kinesis Data Streams with DynamoDB Streams would change the ingestion mechanism entirely and does not address the existing throttling issue; DynamoDB Streams have their own throughput limitations and are not a direct substitute for high-throughput IoT data ingestion. Option D is wrong because increasing the batch size would cause the Lambda function to request more records per invocation, increasing the read load on the Kinesis shards and exacerbating the throttling errors.

Practice this question →

288

MCQeasy

A data engineer is building a pipeline to ingest JSON files from Amazon S3 into Amazon Redshift. The files are 100 MB each and arrive every 5 minutes. Which service is BEST suited for this ingestion?

A.AWS Glue ETL job

B.Amazon Redshift COPY command

C.AWS Lambda with Redshift Data API

D.Amazon Kinesis Data Firehose with Redshift destination

AnswerB

COPY is optimized for loading large data from S3.

Why this answer

The Amazon Redshift COPY command is the most efficient and best-suited service for bulk loading 100 MB JSON files from S3 into Redshift at regular 5-minute intervals. It is optimized for high-throughput, parallel ingestion directly from S3, minimizing latency and resource overhead compared to other services.

Exam trap

The trap here is that candidates often overcomplicate the solution by choosing AWS Glue or Kinesis Firehose for batch ingestion, overlooking that the Redshift COPY command is the simplest, fastest, and most cost-effective option for bulk loading files from S3.

How to eliminate wrong answers

Option A is wrong because AWS Glue ETL jobs are designed for complex data transformation and schema conversion, not for simple, periodic bulk loading of JSON files into Redshift; using Glue adds unnecessary cost and complexity for a straightforward COPY operation. Option C is wrong because AWS Lambda with Redshift Data API is intended for small, transactional queries and has a 15-minute execution timeout and payload size limits, making it unsuitable for ingesting 100 MB files every 5 minutes. Option D is wrong because Amazon Kinesis Data Firehose with Redshift destination is built for streaming data ingestion with near-real-time delivery, not for batch loading of pre-existing files from S3; it would require additional setup to read from S3 and introduces unnecessary buffering and transformation overhead.

Practice this question →

289

MCQhard

A data engineer is troubleshooting a Kinesis Data Firehose delivery stream that sends data to an Amazon S3 bucket. The delivery stream has a buffer size of 5 MB and a buffer interval of 60 seconds. The data ingestion rate is 2 MB per second. The engineer notices that S3 objects are created every 60 seconds but each object is only about 2 MB. What should the engineer do to reduce the number of small S3 objects?

A.Increase the buffer size to 10 MB.

B.Decrease the buffer interval to 30 seconds.

C.Reduce the buffer size to 2 MB.

D.Switch to Kinesis Data Streams and use a Lambda function to write to S3.

AnswerA

Increasing the buffer size to 10 MB will allow the stream to buffer more data before writing to S3, resulting in larger objects.

Why this answer

Increasing the buffer size to 10 MB will allow the stream to buffer more data before writing to S3, resulting in larger objects. Option B is wrong because decreasing the buffer interval would create objects more frequently, making the problem worse. Option C is wrong because reducing the buffer size would create even smaller objects.

Option D is wrong because switching to Kinesis Data Streams does not solve the buffering issue.

Practice this question →

290

MCQhard

A company uses Amazon Kinesis Data Streams to ingest IoT sensor data. The data is processed by an AWS Lambda function that transforms the records and writes to an Amazon S3 bucket. Recently, the Lambda function has been failing with 'Rate exceeded' errors for the S3 PUT API calls. The data volume is 10 MB/s with average record size 2 KB. What should be done to resolve this issue?

A.Add a random prefix to the S3 object key to distribute writes across multiple prefixes

B.Switch to Amazon Kinesis Data Firehose to write to S3

C.Increase the Lambda function's reserved concurrency

D.Increase the number of Kinesis shards

AnswerA

Random prefixes increase the number of S3 partitions, raising the PUT request limit.

Why this answer

The 'Rate exceeded' error for S3 PUT API calls indicates that the Lambda function is hitting S3 request rate limits. S3 buckets have a default limit of 3,500 PUT requests per second per prefix. With a data volume of 10 MB/s and an average record size of 2 KB, the Lambda function is generating approximately 5,000 PUT requests per second (10 MB/s ÷ 2 KB), which exceeds the per-prefix limit.

Adding a random prefix to the S3 object key distributes writes across multiple prefixes, effectively increasing the aggregate request rate limit to 3,500 PUT requests per second per prefix, thereby resolving the throttling issue.

Exam trap

The trap here is that candidates often confuse Kinesis shard scaling (Option D) or Lambda concurrency (Option C) with S3 rate limits, not realizing that the bottleneck is the S3 API request rate per prefix, not the data ingestion pipeline throughput.

How to eliminate wrong answers

Option B is wrong because Amazon Kinesis Data Firehose writes to S3 in batches and can also encounter S3 rate limits if the underlying prefix is not partitioned; it does not inherently solve the per-prefix request rate limit issue. Option C is wrong because increasing the Lambda function's reserved concurrency would increase the number of concurrent invocations, which would generate even more S3 PUT requests per second, exacerbating the rate limiting problem. Option D is wrong because increasing the number of Kinesis shards increases the data ingestion parallelism but does not affect the S3 PUT request rate limit; the Lambda function would still write to the same S3 prefix at the same rate.

Practice this question →

291

Multi-Selectmedium

A company uses AWS Glue to process data from Amazon S3. The Glue job fails with a 'SchemaDetectionException'. The data engineer wants to ensure the schema is correctly inferred. Which TWO actions should the engineer take? (Choose two.)

Select 2 answers

A.Use the Glue Data Catalog as the source for schema definition.

B.Add a column with a default value to the data.

C.Increase the number of Glue DPUs to speed up processing.

D.Convert all input files to Parquet format.

E.Set the 'groupFiles' option to 'inPartition' to combine small files.

AnswersA, E

Using the Glue Data Catalog as the source for schema definition ensures the schema is predefined and consistent, preventing schema detection errors.

Why this answer

The correct answers are A and E. Option A uses the Glue Data Catalog as the source for schema definition, providing a consistent schema and avoiding schema detection failures. Option E sets the 'groupFiles' option to 'inPartition', which helps Glue combine small files within a partition for schema inference, reducing 'SchemaDetectionException' errors.

Option B is incorrect because adding a column with a default value does not affect schema detection. Option C is incorrect because increasing DPUs only improves processing speed, not schema inference. Option D is incorrect because converting to Parquet may change the schema but does not directly address schema detection issues.

Practice this question →

292

Multi-Selectmedium

A company is building a data lake on Amazon S3. The data sources include relational databases, streaming data, and log files. The data engineer needs to ensure that the data ingestion pipeline can handle schema evolution, support both batch and streaming, and provide a unified metadata catalog. Which THREE services should the engineer use? (Choose three.)

Select 3 answers

A.AWS Glue

B.Amazon DynamoDB

C.Amazon Athena

D.Amazon S3

E.Amazon Kinesis Data Firehose

AnswersA, D, E

Provides schema discovery, catalog, and batch ETL.

Why this answer

AWS Glue is correct because it provides a unified metadata catalog (the AWS Glue Data Catalog) that stores schema information for data stored in Amazon S3. It supports schema evolution by allowing you to update the catalog schema as data formats change, and it integrates with both batch (AWS Glue ETL jobs) and streaming (AWS Glue Streaming ETL) ingestion pipelines, making it the central service for metadata management in a data lake.

Exam trap

The trap here is that candidates often confuse Amazon Athena as a metadata catalog or ingestion service, but it is only a query engine that reads from S3 and relies on Glue for metadata, so it does not fulfill the ingestion or catalog requirements.

Practice this question →

293

MCQeasy

A data engineer needs to transform CSV files to Parquet format using AWS Glue. The source data contains sensitive columns that must be masked. Which Glue feature should be used?

A.AWS Glue DataBrew

B.AWS Glue Studio

C.AWS Glue Crawler

D.AWS Glue Schema Registry

AnswerA

DataBrew provides visual data preparation with built-in masking.

Why this answer

AWS Glue DataBrew is a visual data preparation tool that includes built-in transformations for masking sensitive data, such as hashing, tokenizing, or redacting columns. This makes it the correct choice for transforming CSV files to Parquet while applying column-level masking without writing custom code.

Exam trap

The trap here is confusing AWS Glue DataBrew's visual data preparation and masking capabilities with AWS Glue Studio's visual ETL job authoring, which lacks built-in masking transforms and requires custom code.

How to eliminate wrong answers

Option B (AWS Glue Studio) is wrong because it is a visual authoring tool for building ETL jobs, but it does not natively include data masking transformations; you would need to write custom PySpark or Spark SQL code to implement masking. Option C (AWS Glue Crawler) is wrong because it is used for schema discovery and populating the Data Catalog, not for transforming or masking data. Option D (AWS Glue Schema Registry) is wrong because it manages and validates schema evolution for streaming data, not for masking sensitive columns during batch transformations.

Practice this question →

294

Multi-Selecthard

A company uses AWS Glue to run ETL jobs that transform data from Amazon S3 (Parquet) into a denormalized format for Amazon Redshift. The Glue job uses the DynamicFrame API. The job is failing with a 'MemoryError' when performing a join operation. The data is skewed on the join key. Which THREE actions can reduce memory usage and improve job stability? (Choose THREE.)

Select 3 answers

A.Use a broadcast join if one of the tables is small enough.

B.Use a salted join key to distribute skewed keys across partitions.

C.Increase the number of DPUs for the Glue job.

D.Repartition the data on the join key before the join operation.

E.Split the transformation into multiple Glue job steps to reduce per-step memory.

AnswersA, B, E

Avoids shuffling small table.

Why this answer

A broadcast join (using `join` with `broadcast` hint or `DynamicFrame.join(..., transformation_ctx='...')` with broadcast enabled) avoids shuffling the larger table across the cluster by copying the small table to every executor. This eliminates the memory pressure from skewed key distribution during the shuffle phase, reducing the risk of a MemoryError.

Exam trap

The trap here is that candidates often assume increasing resources (DPUs) or repartitioning will fix memory issues, but they fail to recognize that data skew on the join key is the root cause, which requires skew-aware techniques like salting or broadcast joins.

Practice this question →

295

MCQhard

Refer to the exhibit. A data engineer is setting up an Amazon Kinesis Data Firehose delivery stream that writes to an S3 bucket named 'example-bucket'. The IAM role assumed by Firehose has the attached policy shown. When testing, the Firehose delivery stream fails with an access denied error. What is the most likely cause?

A.The S3 bucket has server-side encryption enabled that needs additional permissions.

B.The IAM role does not have permission to use AWS KMS keys.

C.The bucket policy denies access from the Firehose service principal.

D.The IAM policy is missing the s3:AbortMultipartUpload and s3:ListBucket actions.

AnswerD

Firehose uses multipart uploads and needs these permissions.

Why this answer

Kinesis Data Firehose requires the s3:AbortMultipartUpload and s3:ListBucket permissions in addition to s3:PutObject to successfully write data to an S3 bucket. The IAM policy shown only grants s3:PutObject and s3:GetObject, so without these missing actions, the delivery stream fails with access denied. Option A is incorrect because enabling server-side encryption on the S3 bucket does not inherently cause access denial if the IAM role has the necessary permissions; the issue here is the missing S3 actions.

Option B is incorrect because the policy does not involve KMS keys, and the error is not related to encryption key access. Option C is incorrect because there is no indication that the bucket policy explicitly denies the Firehose service principal; the problem is the IAM role's insufficient permissions.

Practice this question →

296

MCQeasy

A data engineer needs to transform JSON data from Amazon S3 into Parquet format using AWS Glue. The source files are in a bucket with thousands of small files. What is the best practice to optimize the Glue job performance?

A.Convert the JSON files to CSV before processing with Glue.

B.Enable 'Group small files' in the Glue job or use a DynamicFrame with coalesce.

C.Use an AWS Lambda function to pre-process the files.

D.Increase the number of DPUs to the maximum.

AnswerB

Grouping reduces the number of tasks and improves performance.

Why this answer

Enabling 'Group small files' in AWS Glue automatically coalesces thousands of small input files into larger partitions, reducing the number of tasks and minimizing overhead from task scheduling and S3 list operations. This is the recommended best practice for handling small files in Glue ETL jobs, as it optimizes read performance without requiring manual coalesce or repartitioning.

Exam trap

The trap here is that candidates assume more DPUs always improve performance, but for small files the bottleneck is metadata overhead, not compute capacity, so increasing DPUs without addressing file grouping leads to wasted resources and no speedup.

How to eliminate wrong answers

Option A is wrong because converting JSON to CSV adds an unnecessary preprocessing step and does not address the root cause of small file overhead; Glue can read JSON directly and convert to Parquet efficiently. Option C is wrong because using Lambda to pre-process files introduces additional cost, complexity, and potential timeout issues for large numbers of files, and does not leverage Glue's built-in optimization for small files. Option D is wrong because simply increasing DPUs does not solve the small file problem; it may even worsen performance by creating more task slots that compete for the same small files, leading to inefficient resource utilization.

Practice this question →

297

Multi-Selecthard

A company is migrating its data warehouse from on-premises to Amazon Redshift. The migration involves copying 50 TB of data from an S3 bucket to Redshift. The network bandwidth is limited to 1 Gbps. Which TWO approaches should the team use to complete the transfer within 7 days?

Select 2 answers

A.Use Amazon S3 Transfer Acceleration

B.Use AWS Direct Connect with 10 Gbps

C.Use AWS Snowball Edge to transfer the data to S3

D.Use AWS Lambda to copy data in parallel

E.Use Amazon Kinesis Data Firehose

AnswersA, C

S3 Transfer Acceleration can speed up uploads over the network.

Why this answer

Amazon S3 Transfer Acceleration (option A) uses AWS edge locations to accelerate uploads over the public internet by routing traffic through optimized paths, which can significantly improve transfer speeds for large datasets when bandwidth is limited. With 1 Gbps bandwidth, the theoretical maximum transfer for 50 TB over 7 days is approximately 75.6 TB (1 Gbps * 7 days * 86400 seconds/day / 8 bits/byte), so the raw bandwidth is sufficient, but Transfer Acceleration helps overcome latency and packet loss issues that can reduce effective throughput. This makes it a valid approach to ensure the transfer completes within the time window.

Exam trap

The trap here is that candidates assume 1 Gbps bandwidth is sufficient for 50 TB in 7 days based on raw calculations, but they overlook real-world network inefficiencies like TCP window scaling, packet loss, and latency, which can drastically reduce effective throughput, making S3 Transfer Acceleration or Snowball Edge necessary.

Practice this question →

298

MCQeasy

A company needs to ingest data from an on-premises database to Amazon S3 with minimal impact on the source database. The data volume is several TB. Which AWS service is best suited for this task?

A.AWS Direct Connect

B.AWS Snowball Edge

C.AWS Database Migration Service (DMS)

D.Amazon S3 Transfer Acceleration

AnswerC

DMS can migrate data from on-premises to S3 with minimal impact using CDC.

Why this answer

AWS Database Migration Service (DMS) is best suited because it can continuously replicate data from an on-premises database to Amazon S3 with minimal impact on the source. DMS uses change data capture (CDC) to capture only incremental changes after an initial full load, avoiding heavy read loads on the source database. This makes it ideal for migrating several TB of data while keeping the source operational.

Exam trap

The trap here is that candidates confuse network acceleration services (Direct Connect, Transfer Acceleration) or offline transfer devices (Snowball) with database-specific migration tools, overlooking that DMS is the only option that directly reads from a database with minimal impact via CDC.

How to eliminate wrong answers

Option A is wrong because AWS Direct Connect provides a dedicated network connection for consistent bandwidth, but it does not perform data ingestion or migration itself; it is a transport layer, not a service that reads from a database. Option B is wrong because AWS Snowball Edge is a physical device for offline data transfer, which is suitable for very large datasets (petabytes) but introduces significant latency and is not designed for minimal impact on a live database during continuous ingestion. Option D is wrong because Amazon S3 Transfer Acceleration speeds up uploads to S3 over the internet using optimized network paths, but it does not interact with the source database or handle database-specific data extraction and transformation.

Practice this question →

299

MCQmedium

A data engineer needs to transform CSV files arriving in S3 into Parquet format and partition them by date. The transformation should be event-driven and run immediately after each file is uploaded. Which approach is most efficient?

A.Use S3 event notification to trigger an AWS Glue job

B.Use an S3 event notification to invoke a Lambda function that converts the file

C.Use an Amazon EMR cluster running Spark to process files as they arrive

D.Use Amazon Athena CREATE TABLE AS SELECT (CTAS) on a schedule

AnswerA

Glue jobs can be triggered by S3 events and efficiently convert to Parquet with partitioning.

Why this answer

AWS Glue jobs can be triggered directly by S3 event notifications, enabling event-driven, serverless transformation of CSV to Parquet with partitioning by date. Glue is optimized for this batch ETL workload, handling schema inference and partitioning efficiently without managing infrastructure, making it the most efficient choice for immediate, per-file transformation.

Exam trap

The trap here is that candidates often choose Lambda for its simplicity and event-driven nature, failing to recognize its execution limits and lack of native support for complex transformations like Parquet conversion with partitioning, which Glue is specifically designed to handle.

How to eliminate wrong answers

Option B is wrong because Lambda functions have a maximum execution time of 15 minutes and a 10 GB memory limit, making them unsuitable for converting large CSV files to Parquet, especially with partitioning logic that may require significant compute and memory. Option C is wrong because an Amazon EMR cluster running Spark is overkill for per-file transformations; it incurs startup latency and ongoing cluster costs, and is designed for large-scale batch processing, not event-driven, single-file triggers. Option D is wrong because Athena CTAS is a query-based operation that runs on a schedule, not event-driven; it scans the entire source data each time, which is inefficient for incremental file arrivals and cannot be triggered immediately per file upload.

Practice this question →

300

MCQhard

A healthcare company processes patient records in near-real-time using Amazon Kinesis Data Streams. Each record contains sensitive personal health information (PHI). The data must be encrypted at rest and in transit. The company also needs to audit access to the data. The data engineer is designing the ingestion pipeline. Which combination of services and configurations meets these requirements?

A.Use Kinesis Data Firehose to deliver data to S3 with SSE-S3, and enable CloudTrail for S3.

B.Use Kinesis Data Streams with TLS and enable CloudTrail for auditing. Do not enable SSE.

C.Use Kinesis Data Streams with SSE-KMS and TLS, and enable CloudTrail for data events.

D.Use Kinesis Data Streams with SSE-KMS and TLS. Do not enable any auditing.

AnswerC

Provides encryption at rest and in transit, plus auditing.

Why this answer

Kinesis Data Streams supports server-side encryption (SSE) using AWS KMS for at-rest encryption, and TLS for in-transit. CloudTrail can log Kinesis API calls for auditing. Option A lacks encryption at rest.

Option B lacks auditing. Option D is wrong because S3 does not replace Kinesis for streaming.

Practice this question →

← PreviousPage 4 of 8 · 591 questions totalNext →

Ready to test yourself?

Try a timed practice session using only Data Ingestion and Transformation questions.

Start 20-question session