CCNA Ml Data Engineering Questions — Page 4 of 5

226

MCQhard

A data engineering team is designing a data lake on Amazon S3. The data is ingested from multiple sources in JSON, CSV, and Parquet formats. The team needs to make the data available for analysis using Amazon Athena and Amazon Redshift Spectrum. The team wants to minimize data transformation costs and storage overhead. Which data storage approach should the team use?

A.Load the data into Amazon Redshift cluster and then unload to S3 in Parquet

B.Store the data in its original format in S3 and use Athena to query directly

C.Store the data in its original format and use AWS Glue to convert to Parquet when queried

D.Convert all data to Apache Parquet before storing in S3

AnswerD

Parquet is columnar, reducing storage and improving query performance.

Why this answer

Converting all data to Apache Parquet before storing in S3 minimizes storage overhead and improves query performance. Parquet is a columnar format that provides efficient compression and encoding schemes, reducing storage costs. It is natively supported by Amazon Athena and Redshift Spectrum, enabling fast analytics without on-the-fly conversion.

Option B (storing in original format) increases storage costs and can degrade query performance, especially with JSON or CSV. Option C incurs transformation costs each time data is queried, negating any storage benefit. Option A adds unnecessary transformation steps and cluster costs.

Therefore, upfront conversion to Parquet is the most cost-effective strategy for this use case.

Practice this question →

227

MCQhard

A data engineer is building a data pipeline that uses AWS Lambda to process records from an SQS queue and write results to an S3 bucket. The Lambda function processes each record individually and writes a separate file to S3. The team notices high latency and wants to reduce the number of S3 PUT requests to improve performance and reduce cost. Which approach should the data engineer take?

A.Use S3 multipart upload for each record to improve throughput.

B.Increase the Lambda function's memory allocation to improve processing speed.

C.Use S3 Batch Operations to process the records in batches.

D.Aggregate multiple records into a single file in a DynamoDB table, then periodically write the aggregated data to S3.

AnswerD

Aggregation reduces the number of S3 PUT requests by writing larger files less frequently.

Why this answer

It reduces the number of S3 PUT requests by aggregating multiple records into a single file in DynamoDB and then periodically writing the aggregated data to S3. This approach directly addresses the high latency and cost issue caused by writing a separate S3 object per record, as S3 PUT requests are billed per operation and have overhead. By batching records before writing, the pipeline reduces the total number of PUT requests, improving throughput and lowering costs.

Exam trap

The trap here is that candidates often confuse 'multipart upload' (Option A) with batching, but multipart upload is for large files, not for reducing the count of small PUT requests, and they may overlook that S3 Batch Operations (Option C) is a post-ingestion tool, not a streaming aggregation mechanism.

How to eliminate wrong answers

Option A is wrong because S3 multipart upload is designed for large objects (over 100 MB) to improve upload throughput and resilience, not for reducing the number of PUT requests for many small records; using it per record would actually increase overhead and cost. Option B is wrong because increasing Lambda memory allocation improves CPU and network throughput for a single invocation, but it does not reduce the number of S3 PUT requests or address the fundamental issue of writing one file per record. Option C is wrong because S3 Batch Operations is used for bulk actions on existing S3 objects (e.g., copying, tagging, restoring), not for processing records from an SQS queue or writing aggregated data from a Lambda pipeline.

Practice this question →

228

MCQeasy

A data engineer needs to ingest streaming data from an on-premises Kafka cluster into Amazon S3 with minimal operational overhead. Which AWS service should be used to stream the data into S3 without managing servers?

A.Amazon Kinesis Data Streams

B.AWS Glue

C.Amazon Managed Streaming for Apache Kafka (Amazon MSK)

D.Amazon Kinesis Data Firehose

AnswerD

Kinesis Data Firehose can directly ingest streaming data and deliver to S3 without managing servers.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service that can directly ingest streaming data from an on-premises Kafka cluster (via a Kinesis Data Firehose HTTP endpoint or a custom producer) and deliver it to Amazon S3 without requiring any server management. It handles scaling, buffering, and compression automatically, minimizing operational overhead.

Exam trap

The trap here is that candidates often confuse Amazon MSK (a managed Kafka cluster) with a direct S3 ingestion service, but MSK still requires you to build and manage the pipeline to S3, whereas Kinesis Data Firehose is purpose-built for serverless streaming to destinations like S3.

How to eliminate wrong answers

Option A is wrong because Amazon Kinesis Data Streams is a real-time data streaming service that requires you to manage consumers and write custom code to load data into S3, which increases operational overhead. Option B is wrong because AWS Glue is a serverless ETL service for batch data processing and cataloging, not designed for real-time streaming ingestion into S3. Option C is wrong because Amazon MSK is a managed Kafka service that still requires you to manage Kafka producers, consumers, and configurations, and does not directly stream data into S3 without additional components like Kafka Connect or custom consumers.

Practice this question →

229

MCQeasy

A machine learning engineer is using Amazon SageMaker to train a model. The training dataset is 2 TB and is stored in Amazon S3. The engineer wants to reduce the training time by improving data loading performance. Which data ingestion mode should be used?

A.Pipe mode

B.Incremental mode

C.File mode

D.Fast file mode

AnswerA

Pipe mode streams data from S3 directly to the algorithm, reducing I/O wait time.

Why this answer

Pipe mode is the correct choice because it streams data directly from Amazon S3 to the training container via a Unix named pipe, bypassing disk writes and reducing I/O latency. For a 2 TB dataset, this eliminates the bottleneck of downloading data to the training instance's Amazon Elastic Block Store (EBS) volume, significantly improving data loading performance and reducing overall training time.

Exam trap

The trap here is that candidates may confuse 'Fast file mode' as a superior alternative to Pipe mode, but Fast file mode still requires writing data to a file system (e.g., FSx for Lustre), which introduces additional latency compared to Pipe mode's direct streaming, making Pipe mode the optimal choice for reducing training time with large datasets.

How to eliminate wrong answers

Option B (Incremental mode) is wrong because it is not a valid SageMaker data ingestion mode; SageMaker supports Pipe, File, and Fast File modes, but not Incremental mode. Option C (File mode) is wrong because it downloads the entire dataset from S3 to the EBS volume before training begins, which for a 2 TB dataset would incur high latency and storage overhead, negating the goal of reducing training time. Option D (Fast file mode) is wrong because it is a variant of File mode that uses a high-performance file system (e.g., Amazon FSx for Lustre) but still requires data to be written to a file system, adding overhead compared to the direct streaming approach of Pipe mode.

Practice this question →

230

MCQhard

A data scientist wants to run a one-time SQL query on a large dataset stored in Amazon S3 (CSV format, 2 TB) using Amazon Athena. The query involves joining this dataset with a smaller table stored in Amazon RDS. What is the MOST cost-effective and performant approach?

A.Export the RDS table to S3 in Parquet format, then use Athena to join the two S3 datasets

B.Use Amazon Redshift Spectrum to query both S3 and RDS

C.Use Athena Federated Query to query RDS directly

D.Use AWS Glue ETL to join the data and write results back to S3, then query with Athena

AnswerA

This keeps the query in Athena's environment, avoiding data movement and using columnar format for performance.

Why this answer

Exporting the RDS table to S3 as Parquet and running the join in Athena avoids data transfer costs and leverages Athena's fast query engine. Option B (federated query) adds complexity and may be slower. Option C (Redshift Spectrum) requires a Redshift cluster.

Option D (Glue ETL) is overkill for a one-time query.

Practice this question →

231

MCQhard

A research lab stores large genomic datasets in Amazon S3 Glacier Deep Archive. They need to run a one-time analysis on a subset of 10 PB of data. The analysis will use an Amazon EMR cluster with Amazon S3 as the data source. What is the MOST cost-effective and performant way to make the data available for the EMR cluster?

A.Restore the data to S3 Standard-IA and delete after the analysis

B.Configure the EMR cluster to read directly from Glacier Deep Archive using S3 Console

C.Initiate a Bulk retrieval request and restore the data to S3 Standard for the duration of the analysis

D.Initiate an Expedited retrieval request and use the temporary copy for the EMR cluster

AnswerC

Bulk retrieval is the lowest cost tier, and restoring to Standard avoids IA minimum charges.

Why this answer

Bulk retrieval is the most cost-effective retrieval tier for large, non-urgent data from S3 Glacier Deep Archive, completing within 48 hours. Restoring to S3 Standard provides direct, high-throughput access for the EMR cluster, and deleting the data after analysis avoids ongoing storage costs. This approach balances performance (EMR reads from S3 Standard) with minimal cost (Bulk retrieval is the cheapest retrieval option).

Exam trap

The trap here is that candidates assume Expedited retrieval is always the fastest and thus best for performance, ignoring the massive cost difference at petabyte scale and the fact that Bulk retrieval's 48-hour window is acceptable for a one-time analysis.

How to eliminate wrong answers

Option A is wrong because restoring to S3 Standard-IA still incurs retrieval costs and per-GB storage fees, and the data must first be restored from Glacier Deep Archive (which requires a retrieval request) before it can be transitioned to Standard-IA; it does not avoid the retrieval step. Option B is wrong because Amazon EMR cannot read directly from S3 Glacier Deep Archive; S3 Glacier Deep Archive is not a real-time data source and requires a restoration process to make objects readable. Option D is wrong because Expedited retrieval is designed for urgent, small-scale retrievals (typically 1–5 minutes for archives up to 250 MB) and is prohibitively expensive for 10 PB of data, making it cost-ineffective for a one-time analysis.

Practice this question →

232

MCQhard

A company is migrating its on-premises Apache Hadoop cluster to AWS. The cluster processes large datasets using Spark jobs. The company wants to minimize operational overhead and use native AWS services. Which combination of services should the company use?

A.Amazon EMR with Spark and Amazon S3

B.Amazon Redshift with Spectrum and Amazon S3

C.Amazon Athena and AWS Glue

D.Amazon EC2 instances with Apache Spark installed and Amazon S3

AnswerA

EMR is a managed service that runs Spark and integrates with S3.

Why this answer

Amazon EMR is a managed Hadoop framework that natively supports Spark jobs, and Amazon S3 provides scalable and durable object storage for the data. This combination minimizes operational overhead as EMR automatically handles cluster provisioning, scaling, and monitoring. Option B is incorrect because Amazon Redshift is a data warehouse, not a Hadoop cluster, and Spectrum is for querying data in S3, not for running Spark jobs.

Option C is incorrect because Amazon Athena is a serverless query service for SQL-based analytics, not for executing Spark jobs, and AWS Glue is an ETL service, not a compute engine for Spark. Option D is incorrect because running Apache Spark on EC2 instances requires manual setup, maintenance, and scaling of the cluster, increasing operational overhead compared to using a managed service like EMR.

Practice this question →

233

MCQeasy

A retail company uses Amazon Redshift for its data warehouse. The data engineering team runs ETL jobs that load data from multiple sources into Redshift daily. They notice that the load performance is slow and the cluster CPU utilization is high during the ETL window. The team wants to improve load performance without changing the cluster configuration. They currently load data using INSERT statements from a staging table. What should they do?

A.Run VACUUM and ANALYZE before loading

B.Use the COPY command to load data from S3 in parallel

C.Increase the number of nodes in the Redshift cluster

D.Apply compression encoding on the staging table

AnswerB

COPY is optimized for bulk loading.

Why this answer

The COPY command is the most efficient way to load large amounts of data into Amazon Redshift because it uses the cluster's nodes in parallel to read data from Amazon S3, maximizing throughput. Option A (VACUUM and ANALYZE) are maintenance operations that reclaim space and update statistics, but they do not improve load performance. Option C (increasing node count) contradicts the requirement to not change cluster configuration.

Option D (compression encoding) can reduce storage and improve scan performance but does not significantly speed up the initial load.

Practice this question →

234

MCQeasy

A machine learning engineer needs to process a large dataset that does not fit on a single Amazon SageMaker notebook instance's EBS volume. The data is stored in S3. What is the MOST efficient way to access the data from the notebook?

A.Increase the EBS volume size to 5 TB.

B.Mount the S3 bucket as a file system using s3fs.

C.Read the data directly from S3 using the boto3 library.

D.Use SageMaker File input mode in the notebook.

AnswerC

Reading directly from S3 avoids storage limitations and is efficient for large datasets.

Why this answer

Reading data directly from S3 using the boto3 library is the most efficient approach for a dataset that exceeds the notebook instance's EBS volume capacity. Boto3 allows you to stream data in chunks or use S3 Select for server-side filtering, avoiding the need to download the entire dataset to local storage. This method leverages S3's high-throughput API and eliminates the bottleneck of writing to a local EBS volume, which is limited in size and I/O performance.

Exam trap

The trap here is that candidates confuse SageMaker's File input mode (designed for training jobs) with a general-purpose data access method for notebooks, or they assume that mounting S3 as a filesystem (s3fs) is efficient for large-scale data processing, when in reality it introduces performance penalties due to FUSE overhead and lack of native parallel I/O.

How to eliminate wrong answers

Option A is wrong because increasing the EBS volume to 5 TB does not solve the fundamental issue of the dataset not fitting; it only postpones the problem and incurs unnecessary cost, and SageMaker notebook instances have a maximum EBS volume size of 5 TB, which may still be insufficient for extremely large datasets. Option B is wrong because mounting an S3 bucket as a file system using s3fs relies on FUSE (Filesystem in Userspace), which introduces significant latency and overhead due to metadata caching and POSIX translation, and is not designed for high-throughput data processing in a notebook environment. Option D is wrong because SageMaker File input mode is a training job feature that streams data from S3 to the training container, not a method for accessing data within a notebook instance; it cannot be used directly in a notebook's kernel.

Practice this question →

235

Multi-Selectmedium

Which TWO configurations are required to enable AWS Glue to access data stored in a VPC? (Choose two.)

Select 2 answers

A.A VPC endpoint for Amazon S3.

B.An AWS Glue connection object that specifies the VPC, subnet, and security group.

C.A NAT gateway in a public subnet.

D.An Internet gateway attached to the VPC.

E.An S3 bucket policy that allows access from the Glue service principal.

AnswersA, B

Correct: Allows Glue jobs in VPC to access S3 without Internet.

Why this answer

A VPC endpoint for Amazon S3 (Option A) is required because it allows AWS Glue to access S3 data privately over the AWS network without traversing the public internet, which is necessary when Glue runs inside a VPC. An AWS Glue connection object (Option B) is required because it defines the VPC, subnet, and security group that Glue will use to launch its resources within the VPC, enabling it to access data stores in that VPC.

Exam trap

The trap here is that candidates often think a NAT gateway or Internet gateway is needed for Glue to access S3 from within a VPC, but AWS Glue can use a VPC endpoint for S3 to keep traffic private and avoid internet routing.

Practice this question →

236

Multi-Selectmedium

Which TWO data formats are columnar and optimized for analytics queries in Amazon S3?

Select 2 answers

A.CSV

B.ORC

C.JSON

D.Avro

E.Parquet

AnswersB, E

ORC is columnar and optimized for analytics.

Why this answer

ORC (Optimized Row Columnar) is a columnar storage format that stores data in a column-oriented manner, enabling efficient compression and predicate pushdown for analytics queries on Amazon S3. It is designed for high-performance read operations in big data frameworks like Apache Hive and Spark, making it ideal for aggregation and filtering workloads.

Exam trap

The trap here is that candidates often confuse 'binary format' (Avro) or 'structured text' (JSON, CSV) with columnar optimization, failing to recognize that only columnar formats like ORC and Parquet provide the compression and predicate pushdown needed for analytics at scale.

Practice this question →

237

MCQhard

A company uses an Amazon SageMaker notebook to train a model using data from an S3 bucket. The IAM role attached to the notebook has the following policy. What is the MOST specific change needed to allow the notebook to read from the bucket 'ml-data-123'?

A.Add an Allow statement for 's3:GetObject' on 'ml-data-123' to the IAM policy.

B.Remove the Deny statement from the IAM policy.

C.Create an S3 access point and update the IAM policy to use the access point ARN.

D.Add a bucket policy on 'ml-data-123' that grants access to the notebook's IAM role.

AnswerB

An explicit deny overrides any allow; removing the deny allows the existing S3 actions to work.

Why this answer

The existing IAM policy includes an explicit Deny statement that blocks all s3:GetObject access to the bucket 'ml-data-123'. In IAM, an explicit Deny overrides any Allow, so even if other policies grant read access, the Deny prevents it. Removing the Deny statement is the most specific change because it eliminates the blocking condition without requiring additional permissions or resources.

Exam trap

The trap here is that candidates often focus on adding Allow permissions or alternative access methods (like access points or bucket policies) without recognizing that an explicit Deny in the IAM policy is the absolute blocker that must be removed first.

How to eliminate wrong answers

Option A is wrong because adding an Allow statement for 's3:GetObject' on 'ml-data-123' would still be overridden by the existing explicit Deny statement, so it would not resolve the issue. Option C is wrong because creating an S3 access point and updating the IAM policy does not address the root cause—the explicit Deny—and adds unnecessary complexity; the Deny would still block access through the access point. Option D is wrong because adding a bucket policy that grants access to the notebook's IAM role cannot override an explicit Deny in the IAM policy; the Deny takes precedence regardless of bucket policy.

Practice this question →

238

MCQmedium

An IAM policy is attached to a group. A user in the group tries to read the object s3://data-lake-bucket/sensitive/file.txt from an IP address 192.168.1.1. What will happen?

A.The request is allowed because the Allow statement grants s3:GetObject

B.The request is allowed because the Deny condition does not match

C.The request is denied because of the Deny statement

D.The request is denied because the policy has no explicit Allow for the sensitive prefix

AnswerC

Deny applies when condition is met.

Why this answer

The Deny statement explicitly denies any S3 action on the sensitive prefix when the source IP is not from 10.0.0.0/8. Since the IP 192.168.1.1 is not in that range, the Deny applies. Deny statements override Allow statements.

So the user is denied access.

Practice this question →

239

MCQhard

A company processes large streams of IoT sensor data using Amazon Kinesis Data Streams with 100 shards. Each sensor reading is about 1 KB. The data is consumed by an Amazon EMR cluster running Spark Streaming jobs. The team notices that the Spark Streaming job's processing time is gradually increasing, and the stream is falling behind. They suspect the issue is due to skewed data distribution across shards. Which approach should the team take to diagnose and resolve the issue?

A.Increase the number of shards to 200 to provide more parallelism.

B.Modify the producer to add a random prefix to the partition key, ensuring even distribution across all shards, and monitor the stream using CloudWatch.

C.Check Amazon CloudWatch metrics for Kinesis to identify hot shards, then manually redistribute the data by repartitioning in Spark.

D.Use the Kinesis Client Library (KCL) with a custom worker to rebalance the load across shards.

AnswerB

Adding a random prefix to partition keys uniformizes distribution, eliminating hot shards; CloudWatch helps confirm the fix.

Why this answer

Adding a random prefix to the partition key ensures that sensor data is evenly distributed across all 100 shards, eliminating hot shards that cause processing delays. This directly addresses the skewed data distribution issue without requiring infrastructure changes, and the team can monitor the improvement using CloudWatch metrics like IncomingBytes and ReadProvisionedThroughputExceeded.

Exam trap

The trap here is that candidates often confuse consumer-side rebalancing (KCL or Spark repartitioning) with producer-side data distribution, and incorrectly assume that increasing shards or using Spark repartitioning can fix a hot shard caused by a poor partition key.

How to eliminate wrong answers

Option A is wrong because simply increasing the number of shards to 200 does not fix the root cause of skewed distribution; it only adds more shards that may still be unevenly loaded if the partition key remains the same, potentially worsening the imbalance. Option C is wrong because while CloudWatch metrics can identify hot shards, manually redistributing data by repartitioning in Spark does not change how data is written to Kinesis shards; the producer-side partition key must be fixed to prevent future skew. Option D is wrong because the Kinesis Client Library (KCL) rebalances consumers across shards, but it cannot change how data is distributed across shards at the producer level; the skew originates from the producer's partition key selection.

Practice this question →

240

MCQeasy

A data engineer is tasked with building a data pipeline that moves data from an on-premises database to Amazon S3 for analytics. The database is a MySQL instance that is 2 TB in size. The company has a 1 Gbps dedicated network connection to AWS (AWS Direct Connect). The data must be transferred once daily. The engineer needs to choose the most efficient and reliable service for this task. Which service should they use?

A.AWS DataSync

B.AWS Database Migration Service (DMS)

C.AWS Glue

D.Amazon S3 Transfer Acceleration

AnswerB

DMS is designed for database migrations and supports S3 as a target.

Why this answer

AWS Database Migration Service (DMS) is the correct choice because it is purpose-built for migrating databases to AWS, supporting continuous replication and one-time migrations from MySQL to Amazon S3. It can handle the 2 TB dataset efficiently over a 1 Gbps Direct Connect link by using change data capture (CDC) for ongoing replication and parallel tasks for throughput, ensuring reliability with built-in monitoring and restart capabilities.

Exam trap

The trap here is that candidates often confuse AWS DataSync (a file-transfer service) with a database migration tool, overlooking that DMS is the only option that natively supports extracting data from a relational database like MySQL and writing it to Amazon S3 in a structured format.

How to eliminate wrong answers

Option A is wrong because AWS DataSync is designed for file and object storage transfers (e.g., NFS, SMB, S3), not for direct database-to-S3 migration; it cannot connect to a MySQL database natively. Option C is wrong because AWS Glue is an ETL service that requires a schema and transformation logic, not a direct database migration tool; it would need additional setup (e.g., JDBC connections and crawlers) and lacks the optimized CDC and bulk load capabilities of DMS for a 2 TB database. Option D is wrong because Amazon S3 Transfer Acceleration only speeds up uploads to S3 over the public internet by using edge locations, but it does not handle database extraction or schema conversion; it is irrelevant when a Direct Connect connection is already in place.

Practice this question →

241

MCQhard

A data engineer is building a data pipeline that uses Amazon S3 to store raw data, AWS Lambda for transformation, and Amazon DynamoDB for serving. The Lambda function experiences high latency when writing to DynamoDB. Which action will most effectively reduce the latency?

A.Enable DynamoDB Accelerator (DAX) for caching

B.Use Amazon S3 instead of DynamoDB

C.Configure a VPC gateway endpoint for DynamoDB

D.Increase the DynamoDB write capacity units

AnswerD

Increasing write capacity units directly addresses throttling and reduces write latency by providing more throughput.

Why this answer

Increasing write capacity units for DynamoDB directly reduces write latency by minimizing throttling. DAX (Option A) is a read cache and does not improve write latency. Option B is incorrect because using S3 would increase latency due to its different access pattern.

Option C is incorrect because VPC gateway endpoint improves network connectivity but does not reduce write latency.

Practice this question →

242

MCQhard

A company runs a critical data pipeline using Apache Spark on Amazon EMR. The pipeline reads data from Amazon S3, performs complex transformations, and writes results back to S3. The job runs every hour and must complete within 30 minutes. Recently, the job has been taking longer and occasionally failing due to executor losses. The team suspects memory pressure. Which action should the team take to improve stability and performance without increasing cost?

A.Increase the spark.executor.memory setting to allocate more memory per executor.

B.Increase the number of core nodes in the EMR cluster.

C.Decrease the number of shuffle partitions (spark.sql.shuffle.partitions) to reduce overhead.

D.Enable Spark dynamic allocation to adjust executors based on workload.

AnswerD

Dynamic allocation helps utilize resources efficiently and prevents over-allocation.

Why this answer

Enabling Spark dynamic allocation allows the cluster to automatically scale the number of executors up and down based on the workload. This helps alleviate memory pressure by releasing idle executors and requesting additional executors only when needed, improving resource utilization without increasing overall cluster cost. Option A is incorrect because simply increasing spark.executor.memory may cause YARN container failures if the instance memory is exceeded, and does not address the root cause of executor losses.

Option B is incorrect because adding core nodes increases cost and may not resolve memory pressure if the issue is inefficient resource allocation. Option C is incorrect because decreasing shuffle partitions reduces parallelism and can increase memory per task, potentially worsening memory pressure and prolonging job runtime.

Practice this question →

243

MCQmedium

A data engineer uses AWS Glue to run ETL jobs that transform data from JSON to Parquet. The job runs successfully but takes 30 minutes longer than expected. CloudWatch metrics show high memory utilization and disk spills. What is the most likely cause?

A.The number of DPUs is too low

B.The sink bucket has insufficient I/O throughput

C.The source data format is too large

D.The data is skewed and not evenly distributed across partitions

AnswerD

Data skew causes some tasks to take longer, leading to spills and increased runtime.

Why this answer

High memory utilization and disk spills in AWS Glue indicate that the data is not evenly distributed across partitions, causing some executors to handle a disproportionate amount of data. This data skew leads to excessive spilling to disk as memory is exhausted, which significantly slows down the job. Option D directly addresses this root cause, as skewed data prevents efficient parallel processing.

Exam trap

The trap here is that candidates often assume high memory usage means insufficient resources (DPUs) and choose Option A, but the real culprit is data skew causing inefficient resource utilization, not a lack of total compute capacity.

How to eliminate wrong answers

Option A is wrong because increasing DPUs would add more parallelism but does not fix the underlying data skew; it may even worsen memory pressure if the skewed partitions are not repartitioned. Option B is wrong because insufficient I/O throughput to the sink bucket would manifest as write throttling or retries, not as high memory utilization and disk spills during transformation. Option C is wrong because the source data format being large is not inherently a problem—Parquet is columnar and efficient; the issue is how the data is distributed across partitions, not its total size.

Practice this question →

244

MCQmedium

Refer to the exhibit. A data engineer is troubleshooting an AWS Glue job that fails with an 'AccessDenied' error when trying to write to the S3 bucket 'my-data-lake'. The IAM policy attached to the Glue service role is shown. What is the missing permission?

A.s3:ListBucket

B.s3:PutObjectAcl

C.s3:GetBucketLocation

D.s3:DeleteObject

AnswerA

Correct: Glue needs ListBucket to list objects in the bucket.

Why this answer

The policy allows s3:GetObject and s3:PutObject on the bucket's objects, but it does not allow s3:ListBucket on the bucket itself. Many Glue operations require ListBucket to discover objects. Option A (s3:ListBucket) is the missing permission.

Option B (s3:PutObjectAcl) is not needed. Option C (s3:GetBucketLocation) is not required. Option D (s3:DeleteObject) is not needed.

Practice this question →

245

MCQhard

An ML team is building a recommendation system. The training data includes user-item interactions stored in Amazon DynamoDB. The team wants to export this data to S3 in Parquet format for use with Amazon SageMaker. The export should be incremental (only new or changed records) and run daily. Which approach meets these requirements with MINIMAL operational overhead?

A.Use the DynamoDB Export to S3 feature and schedule it daily with AWS Glue.

B.Use DynamoDB Streams with AWS Lambda to write changes to S3 in Parquet format.

C.Use a script that scans the DynamoDB table and filters by last updated timestamp.

D.Set up an Amazon EMR cluster running Spark jobs to read DynamoDB and write to S3.

AnswerB

Streams capture changes in near-real-time, enabling incremental exports with minimal overhead.

Why this answer

DynamoDB Streams capture every change (insert, update, delete) in near real-time, and AWS Lambda can process these events to write only the changed records to S3 in Parquet format. This approach provides incremental, daily exports with minimal operational overhead, as it is fully serverless and requires no infrastructure management.

Exam trap

The trap here is that candidates often choose Option A because they assume 'Export to S3' is incremental, but it actually exports the entire table, not just changes, leading to higher costs and redundant data processing.

How to eliminate wrong answers

Option A is wrong because the DynamoDB Export to S3 feature exports the entire table snapshot, not incremental changes, and scheduling it with AWS Glue adds unnecessary complexity and cost for a full export each day. Option C is wrong because scanning the entire DynamoDB table daily and filtering by last updated timestamp is inefficient, costly (consumes read capacity), and does not capture deletions; it also requires custom scripting and handling of large datasets. Option D is wrong because setting up and managing an Amazon EMR cluster introduces significant operational overhead for a simple incremental export task, and it is overkill compared to the serverless Streams + Lambda approach.

Practice this question →

246

MCQhard

A company is using AWS Glue to run ETL jobs that transform data from multiple sources into a data lake on S3. The jobs are scheduled to run hourly. Recently, the jobs have been failing intermittently with 'MemoryError' exceptions. The data volume has grown over time. The data engineer needs to resolve this issue cost-effectively. Which action should be taken?

A.Increase the number of DPUs allocated to the Glue job and use a larger worker type.

B.Increase the S3 timeout settings in the Glue job configuration.

C.Switch the Glue job type from Spark to Python shell to reduce memory overhead.

D.Repartition the data using Spark's repartition method before processing.

AnswerA

More DPUs and larger worker types provide more memory to handle larger data volumes.

Why this answer

The 'MemoryError' exception indicates that the Glue job is running out of memory as data volume grows. Increasing the number of DPUs (Data Processing Units) and using a larger worker type (e.g., from Standard to G.1X or G.2X) provides more memory and compute capacity per worker, allowing the job to handle larger datasets without failing. This is the most cost-effective approach because it scales resources only as needed, avoiding over-provisioning.

Exam trap

The trap here is that candidates may confuse memory errors with data skew or partitioning issues, leading them to choose repartitioning (Option D) instead of recognizing that the root cause is insufficient total memory for the growing dataset.

How to eliminate wrong answers

Option B is wrong because S3 timeout settings control how long the job waits for S3 operations, not the memory allocation; memory errors are unrelated to network timeouts. Option C is wrong because switching from Spark to Python shell would drastically reduce processing capability and memory, likely causing the job to fail entirely on large datasets, not solve the memory issue. Option D is wrong because repartitioning data with Spark's repartition method can increase parallelism but does not directly increase the total memory available to the job; it may even cause more memory pressure if partitions are increased without adding resources.

Practice this question →

247

MCQeasy

A company is using Amazon Kinesis Data Firehose to load streaming data into Amazon S3. The data is in JSON format, and they want to convert it to Parquet before storage. What should they configure?

A.Enable data format conversion in Firehose and specify a Glue table

B.Use an AWS Lambda function to transform the data

C.Run an AWS Glue ETL job after data is in S3

D.Use Kinesis Data Analytics for Apache Flink to convert the format

AnswerA

Firehose can convert to Parquet using a Glue table schema.

Why this answer

Amazon Kinesis Data Firehose supports built-in data format conversion from JSON to Parquet or ORC. By enabling this feature and specifying an AWS Glue table that defines the schema, Firehose automatically converts incoming JSON records to Parquet before delivering them to the S3 destination. This eliminates the need for additional compute resources or post-processing steps.

Exam trap

The trap here is that candidates often assume they need a separate transformation service like Lambda or Glue, not realizing that Firehose itself has a native, serverless data format conversion feature that directly writes Parquet to S3.

How to eliminate wrong answers

Option B is wrong because using an AWS Lambda function for transformation would require custom code to convert JSON to Parquet, adding complexity and latency, and Lambda has a maximum execution time and payload size limit that may not suit high-throughput streaming data. Option C is wrong because running an AWS Glue ETL job after data is in S3 introduces a batch processing step, which defeats the purpose of real-time or near-real-time conversion and incurs additional storage and compute costs. Option D is wrong because Kinesis Data Analytics for Apache Flink is designed for real-time stream processing and analytics, not for format conversion to Parquet for S3 storage; it would require custom Flink code and does not integrate directly with Firehose's S3 delivery.

Practice this question →

248

Multi-Selecteasy

Which TWO options are best practices for managing access to data stored in Amazon S3 for a data lake?

Select 2 answers

A.Use S3 access control lists (ACLs) for granular permissions

B.Enable default encryption with SSE-S3

C.Use IAM policies to control user and role permissions

D.Use S3 bucket policies to grant cross-account access

E.Generate pre-signed URLs for all data access

AnswersC, D

IAM policies are central to access management.

Why this answer

C is correct because IAM policies are the primary mechanism for controlling access to AWS services, including S3, for users and roles within an AWS account. They allow you to define fine-grained permissions based on identity, which is a best practice for managing access to a data lake. This aligns with the principle of least privilege and centralized access control.

Exam trap

The trap here is that candidates often confuse encryption mechanisms (like SSE-S3) with access control, or mistakenly think that legacy ACLs are still a best practice for granular permissions in modern data lake architectures.

Practice this question →

249

MCQhard

A team is building a data pipeline to process terabytes of log data daily using Amazon EMR. The data arrives in 5-minute windows and must be available for querying within 30 minutes. The data is originally in gzip-compressed CSV files. Which approach will minimize processing time and cost?

A.Use Amazon EMR with Spark to convert data to Parquet and use on-demand instances.

B.Use Amazon EMR with Spark to convert data to Parquet and store in S3, using spot instances for task nodes.

C.Use AWS Glue to convert data to gzip-compressed CSV and query with Athena.

D.Use Amazon EMR with Hive to transform data to compressed CSV and store in S3.

AnswerB

Parquet reduces scan size, spot instances reduce cost.

Why this answer

Converting gzip-compressed CSV to Parquet reduces storage size and improves query performance due to columnar storage and predicate pushdown. Using spot instances for task nodes significantly lowers compute cost, while the 30-minute SLA is achievable with Spark on EMR processing 5-minute windows of data.

Exam trap

The trap here is that candidates may overlook the cost savings of spot instances for transient, fault-tolerant workloads, or assume that any compression (like gzip CSV) is sufficient for performance, ignoring the benefits of columnar formats like Parquet for analytical queries.

How to eliminate wrong answers

Option A is wrong because using on-demand instances for task nodes increases cost unnecessarily; spot instances are suitable for fault-tolerant, transient workloads like data transformation. Option C is wrong because AWS Glue is not optimized for high-volume, low-latency ETL on terabytes of daily log data, and converting to gzip-compressed CSV does not improve query performance over Parquet. Option D is wrong because Hive on EMR is slower than Spark for large-scale data processing, and storing as compressed CSV does not provide the performance benefits of columnar formats like Parquet.

Practice this question →

250

Multi-Selecthard

A company is using Amazon Redshift for data warehousing. The data engineering team observes that query performance degrades over time due to data skew. Which three strategies should the team implement to improve performance?

Select 3 answers

A.Choose appropriate distribution keys based on join and group-by columns.

B.Increase the number of nodes in the Redshift cluster.

C.Run VACUUM and ANALYZE commands regularly.

D.Define appropriate sort keys to minimize the number of blocks scanned.

E.Drop unused indexes on large tables.

AnswersA, C, D

Good distribution keys reduce data movement and improve performance.

Why this answer

Choosing appropriate distribution keys based on join and group-by columns minimizes data movement across nodes during query execution. In Amazon Redshift, data is distributed across compute nodes according to the distribution key; aligning it with frequently joined or aggregated columns ensures that related rows are co-located on the same slice, reducing network shuffling and improving query performance.

Exam trap

The trap here is that candidates often confuse Redshift's distribution and sort keys with traditional database indexes, leading them to select option E, or they mistakenly believe that scaling out nodes (option B) automatically fixes skew-related performance issues.

Practice this question →

251

Multi-Selecthard

A company needs to build a data lake on AWS for analytics. The data includes structured, semi-structured, and unstructured data. The solution must support schema-on-read, provide fine-grained access control, and be cost-effective for storing rarely accessed data. Which THREE services should be used? (Choose THREE)

Select 3 answers

A.AWS Glue Data Catalog for schema-on-read.

B.Amazon Redshift for data warehousing.

C.Amazon S3 as the primary storage layer.

D.Amazon EMR for data processing.

E.S3 Lifecycle policies to transition data to Glacier.

AnswersA, C, E

Glue enables schema-on-read for analytics.

Why this answer

AWS Glue Data Catalog is correct because it provides a centralized metadata repository that enables schema-on-read for data stored in Amazon S3. It allows you to define table schemas and partitions without transforming the underlying data, so analytics tools like Amazon Athena and Amazon EMR can query the data with the schema applied at read time.

Exam trap

The trap here is that candidates often confuse Amazon Redshift as a data lake storage layer due to its analytics capabilities, but it is a data warehouse with schema-on-write and higher costs for infrequently accessed data, making it unsuitable for the described requirements.

Practice this question →

252

MCQeasy

A company wants to store semi-structured data from IoT sensors in a cost-effective manner for occasional querying. The data is not updated once written. Which Amazon S3 storage class is the most cost-effective for this use case?

A.S3 Standard

B.S3 One Zone-Infrequent Access

C.S3 Intelligent-Tiering

D.S3 Glacier Deep Archive

AnswerD

Correct: Deep Archive is the lowest cost for rarely accessed data with long retrieval times.

Why this answer

S3 Glacier Deep Archive is the most cost-effective storage class for semi-structured IoT sensor data that is written once and only occasionally queried. It offers the lowest storage cost among S3 classes (approximately $0.00099/GB/month), making it ideal for long-term archival of immutable data where retrieval times of 12–48 hours are acceptable.

Exam trap

The trap here is that candidates often choose S3 One Zone-Infrequent Access (Option B) because they focus on 'cost-effective' and 'infrequent access' without considering the requirement for durability and the even lower cost of Glacier Deep Archive for immutable archival data.

How to eliminate wrong answers

Option A is wrong because S3 Standard is designed for frequently accessed data with millisecond retrieval, incurring higher storage costs (~$0.023/GB/month) that are unnecessary for rarely queried IoT data. Option B is wrong because S3 One Zone-Infrequent Access, while cheaper than Standard, still costs more than Glacier Deep Archive and stores data in a single Availability Zone, risking data loss if that AZ fails—unacceptable for archival data. Option C is wrong because S3 Intelligent-Tiering automatically moves data between tiers based on access patterns but incurs a monthly monitoring fee ($0.0025 per 1,000 objects) and does not include the Deep Archive tier, so it cannot achieve the lowest cost for data that is almost never accessed.

Practice this question →

253

MCQhard

A company runs a real-time fraud detection system using Amazon Kinesis Data Streams with 100 shards. Data is consumed by a custom Java application running on Amazon EC2 instances in an Auto Scaling group. The application processes records and writes results to a DynamoDB table. Over the past month, the application has experienced intermittent slowdowns and the DynamoDB write capacity has been fully utilized during peak hours. The team wants to improve throughput without losing the ability to reprocess failed records. The application currently uses the Kinesis Client Library (KCL) with DynamoDB as the lease table. The team is considering the following changes: A. Increase the number of EC2 instances to match the number of shards. B. Switch to using AWS Lambda as the consumer to handle scaling automatically. C. Increase the write capacity of the DynamoDB lease table to handle more workers. D. Use enhanced fan-out to have each consumer receive its own 2 MB/second shard throughput. Which change should the team implement first to address the issue?

A.Increase the write capacity of the DynamoDB lease table to handle more workers.

B.Use enhanced fan-out to have each consumer receive its own 2 MB/second shard throughput.

C.Switch to using AWS Lambda as the consumer to handle scaling automatically.

D.Increase the number of EC2 instances to match the number of shards.

AnswerB

Enhanced fan-out gives dedicated throughput per consumer.

Why this answer

The primary bottleneck is DynamoDB write capacity being fully utilized during peak hours. Enhanced fan-out (option B) provides each consumer with a dedicated 2 MB/second read throughput per shard, eliminating the need for consumers to contend for the shared 2 MB/second per shard. This reduces the load on the DynamoDB lease table because workers no longer need to poll for records, which in turn lowers the write operations to the lease table and alleviates the DynamoDB write capacity issue.

Exam trap

The trap here is that candidates assume increasing DynamoDB write capacity (option A) is the direct fix for write capacity exhaustion, but they miss that enhanced fan-out reduces the underlying cause of those writes by eliminating polling-based contention.

How to eliminate wrong answers

Option A is wrong because increasing EC2 instances to match shards does not address the DynamoDB write capacity bottleneck; it may even increase lease table writes due to more workers contending for leases. Option C is wrong because increasing the write capacity of the DynamoDB lease table treats a symptom (high write load from KCL workers) rather than the root cause (contention for shard throughput); enhanced fan-out reduces the need for frequent lease updates. Option D is wrong because switching to AWS Lambda does not inherently solve the DynamoDB write capacity issue; Lambda still uses KCL under the hood with DynamoDB as the lease table, and the same write contention would persist unless enhanced fan-out is also used.

Practice this question →

254

Multi-Selecthard

A company uses AWS Glue to run ETL jobs on a daily basis. The jobs read from Amazon RDS and write to Amazon S3. The data volume has grown, and the jobs are taking longer to complete. The team wants to optimize the jobs for cost and performance. Which combination of techniques should the team implement? (Choose THREE.)

Select 3 answers

A.Use a larger Glue worker type, such as G.2X, for more memory per worker.

B.Enable job bookmarks to process only new data since the last run.

C.Increase the number of partitions in the output S3 data to improve parallelism.

D.Increase the maximum number of DPUs for the job to 100.

E.Use pushdown predicates in the JDBC connection to filter data at the source.

AnswersA, B, E

Larger workers provide more resources per task, improving performance.

Why this answer

(larger worker type) provides more memory and CPU per worker, improving performance for heavy workloads. Option B (job bookmarks) enables incremental processing, reducing the amount of data read on subsequent runs. Option E (pushdown predicates) filters data at the source in the JDBC connection, reducing data transferred across the network.

Option C is incorrect because increasing partitions in the output S3 data does not affect the processing speed of the current job. Option D is incorrect because increasing the maximum number of DPUs increases cost linearly and may not be as effective as using larger workers or other optimizations.

Practice this question →

255

MCQhard

Refer to the exhibit. A data engineer examines the output of 'aws glue get-job-run' for a failed job. The job run state is FAILED, but ErrorMessage is empty. The job ran for 3600 seconds (1 hour) before failing. What is the MOST likely cause of the failure?

A.The JDBC connection to the source database timed out.

B.The IAM role does not have sufficient permissions to access S3.

C.The job ran out of memory due to insufficient DPU allocation.

D.The Python script has a syntax error.

AnswerC

Out-of-memory errors may not always produce a detailed error message in the job run output.

Why this answer

The job ran for 3600 seconds (the default Glue job timeout) before failing with a FAILED state and an empty ErrorMessage. This pattern is characteristic of an out-of-memory (OOM) error in AWS Glue, which occurs when the allocated DPUs (Data Processing Units) are insufficient for the data volume or transformation complexity. Glue kills the job at the timeout boundary without a detailed error message because the JVM or Python process is killed by the OS (OOM killer), not by a Glue service exception.

Exam trap

AWS Glue jobs failing due to resource exhaustion (memory/DPU) will show a FAILED state with an empty ErrorMessage and run until the timeout, unlike permission or syntax errors which produce immediate, descriptive failures.

How to eliminate wrong answers

Option A is wrong because a JDBC connection timeout would produce a specific error message (e.g., 'Connection timed out' or 'Communications link failure') in the ErrorMessage field, not an empty one. Option B is wrong because an IAM permissions issue for S3 would result in an AccessDenied error with a clear message in the job logs or ErrorMessage, and the job would fail almost immediately, not after 3600 seconds. Option D is wrong because a Python syntax error would be caught at script compilation time, causing the job to fail within seconds with a detailed SyntaxError traceback in the ErrorMessage or logs, not after a full hour of execution.

Practice this question →

256

MCQeasy

Refer to the exhibit. A data engineer has deployed this CloudFormation template. The Glue job 'my-etl-job' reads from the S3 bucket 'my-data-lake-bucket' and writes transformed data to another bucket. After 30 days, the data engineer notices that the Glue job fails with 'Input data not found' errors. What is the most likely cause?

A.The temporary directory 'my-temp-dir' is being cleaned up by the lifecycle configuration.

B.The script location 's3://my-scripts/etl.py' is being deleted by the lifecycle rule.

C.The job bookmark option 'job-bookmark-enable' is causing the job to skip newly arriving data.

D.The lifecycle configuration deletes objects from the bucket after 30 days, removing the input data.

AnswerD

The ExpirationInDays: 30 rule deletes objects older than 30 days, which may include input data.

Why this answer

The lifecycle configuration on the S3 bucket 'my-data-lake-bucket' is set to delete objects after 30 days. Since the Glue job 'my-etl-job' reads input data from this bucket, once the 30-day period elapses, the input data is removed, causing the 'Input data not found' error. This matches the symptom of the job failing after exactly 30 days.

Exam trap

A common trap on the AWS Machine Learning Specialty exam is that candidates mistakenly attribute failures to job bookmarks or temporary directories instead of recognizing that the lifecycle rule is deleting the source data after the specified retention period.

How to eliminate wrong answers

Option A is wrong because the temporary directory 'my-temp-dir' is used for intermediate job artifacts (e.g., shuffle data or staging), not for input data; its cleanup would cause job runtime errors, not 'Input data not found' errors. Option B is wrong because the script location 's3://my-scripts/etl.py' is the ETL script itself, which is read at job start and cached; if deleted, the job would fail immediately at launch, not after 30 days of successful runs. Option C is wrong because 'job-bookmark-enable' controls state tracking for incremental processing; if it caused skipping, the error would be about missing new data, not 'Input data not found' for existing data.

Practice this question →

257

MCQeasy

A company uses Amazon Redshift for data warehousing. The data engineering team needs to load data from multiple S3 buckets into Redshift daily. Each bucket contains files in different formats (CSV, JSON, Parquet). Which AWS service is BEST suited to automate this ingestion process?

A.Amazon EMR with Apache Spark

B.AWS Data Pipeline

C.AWS Database Migration Service (DMS)

D.AWS Glue

AnswerD

Glue provides crawlers for schema discovery and ETL jobs for loading into Redshift.

Why this answer

AWS Glue is a fully managed ETL service that can crawl S3 buckets to discover schema, handle various formats (CSV, JSON, Parquet), and load data into Redshift. It automates the ingestion process without the need for manual infrastructure management. Amazon EMR with Spark requires more setup and management, AWS Data Pipeline is less flexible and older, and AWS Database Migration Service is designed for migrating entire databases, not for loading from S3.

Practice this question →

258

Multi-Selecthard

Which THREE factors should be considered when choosing between Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose for a real-time data ingestion pipeline? (Choose 3.)

Select 3 answers

A.Ability to compress data before delivery

B.Ability to encrypt data at rest

C.Need for custom data processing using AWS Lambda

D.Data retention requirements

E.Latency requirements for data delivery to S3

AnswersC, D, E

Kinesis Data Streams supports custom processing with Lambda, Firehose has limited transformation.

Why this answer

Kinesis Data Streams supports custom processing via AWS Lambda consumers (using the Kinesis Client Library or direct integration), enabling real-time transformations, filtering, or enrichment. Kinesis Data Firehose does not natively support custom Lambda processing for transformation; it only allows optional Lambda functions for data format conversion or transformation before delivery, but not for arbitrary real-time processing logic.

Exam trap

A common misconception is that Kinesis Data Firehose supports custom real-time processing like Streams, but Firehose only allows optional Lambda transformations with limited control and no data replay capability.

Practice this question →

259

MCQmedium

A company is using Amazon SageMaker to train a model on a dataset that is updated daily. The data is stored in an S3 bucket. The training pipeline uses AWS Step Functions to orchestrate data preprocessing and model training. The preprocessing step uses a SageMaker Processing job that reads data from S3, cleans it, and writes the output back to S3. The team notices that the training step often fails due to insufficient disk space on the processing instance. Which change should the team make to resolve this issue without increasing cost?

A.Enable automatic scaling for the processing job.

B.Use AWS Batch instead of SageMaker Processing.

C.Use a larger instance type with more memory.

D.Configure the processing job to use local instance store (SSD) for scratch space.

AnswerD

Local instance store provides additional disk space without additional cost.

Why this answer

The issue is insufficient disk space on the processing instance. Option D resolves this by configuring the processing job to use the local instance store (SSD) for scratch space, which provides high-throughput temporary storage without incurring additional cost, as the instance store is included with the instance. This allows the preprocessing step to handle larger intermediate data without requiring a larger or more expensive instance.

Exam trap

The trap here is that candidates may assume increasing instance size (Option C) is the only way to get more disk space, overlooking that local instance store provides additional scratch space at no extra cost, and that automatic scaling (Option A) is not applicable to SageMaker Processing jobs.

How to eliminate wrong answers

Option A is wrong because automatic scaling for a processing job is not supported; SageMaker Processing jobs run on a fixed instance count and cannot scale dynamically. Option B is wrong because using AWS Batch would not inherently resolve disk space issues and could increase complexity and cost due to different pricing models and data transfer overhead. Option C is wrong because using a larger instance type with more memory would increase cost, which contradicts the requirement to not increase cost, and memory is not the bottleneck—disk space is.

Practice this question →

260

MCQhard

A team is building a data lake on Amazon S3 and using AWS Glue to catalog data. They notice that Glue crawlers are taking too long to update the catalog for a large dataset with millions of small files. Which approach will MOST improve crawler performance?

A.Increase the frequency of the crawler runs.

B.Consolidate the small files into larger files (e.g., 100 MB each).

C.Partition the data by date in S3.

D.Use a custom classifier to parse the data.

AnswerB

Fewer, larger files reduce overhead and crawler scan time.

Why this answer

AWS Glue crawlers incur significant overhead when processing millions of small files because each file requires a separate read, schema inference, and metadata write operation. Consolidating small files into larger files (e.g., 100 MB each) reduces the total number of objects that the crawler must scan, dramatically decreasing the time spent on file-level operations and improving overall throughput.

Exam trap

The trap here is that candidates confuse partitioning (which improves query pruning) with file consolidation (which reduces metadata and I/O overhead), leading them to select partitioning as a performance fix for crawlers when it does not address the root cause of high file count.

How to eliminate wrong answers

Option A is wrong because increasing crawler frequency does not reduce the per-run overhead; it only makes the problem occur more often, potentially leading to throttling and higher costs. Option C is wrong because partitioning by date in S3 improves query performance and reduces data scanned by Athena or Spark, but it does not reduce the number of files the crawler must process—each partition still contains many small files. Option D is wrong because custom classifiers are used to interpret non-standard data formats (e.g., custom log formats), not to address performance issues caused by file count or size.

Practice this question →

261

MCQeasy

A company is streaming clickstream data from a website to Amazon Kinesis Data Streams. The data is consumed by a Lambda function that enriches each record with geolocation information before writing to an S3 bucket. Recently, the Lambda function has been failing with throttling errors. What is the MOST likely cause?

A.The Lambda function's payload size exceeds the 6 MB limit

B.The Lambda function's concurrent execution limit has been reached

C.The Lambda function's reserved concurrency is set too high

D.The Kinesis stream has exceeded the default shard limit of 500

AnswerB

Lambda throttles when the number of concurrent executions exceeds the account limit.

Why this answer

The most likely cause is that the Lambda function's concurrent execution limit has been reached. Kinesis Data Streams invokes Lambda functions per shard, and with high throughput or many shards, concurrent invocations can exceed the default Lambda concurrency limit (1000 per region). This results in throttling errors.

Option A is incorrect because the Lambda payload limit for asynchronous invocation (used by Kinesis) is 256 KB, not 6 MB. Option C is incorrect; setting reserved concurrency too high would not cause throttling—it could actually help avoid throttling. Option D is incorrect because the default shard limit is 500, but shard limits are a Kinesis concern, not directly causing Lambda throttling.

Practice this question →

262

MCQhard

A data scientist is training a deep learning model on a GPU instance. The training data is stored in S3 and is 50 GB. To reduce I/O bottlenecks, which storage option should be used to cache the data locally on the instance?

A.Attach an Amazon EFS file system to the instance and copy data from S3

B.Mount an Amazon FSx for Lustre file system linked to the S3 bucket

C.Provision an Amazon EBS io2 volume and copy data from S3 using AWS DataSync

D.Use instance store volumes to cache the data from S3

AnswerB

FSx for Lustre provides high throughput and can cache S3 data locally.

Why this answer

Amazon FSx for Lustre is a high-performance file system designed for HPC and machine learning workloads. By linking it to the S3 bucket, it automatically caches data locally on the Lustre file system attached to the GPU instance, providing low-latency access and reducing I/O bottlenecks. Option A is incorrect because Amazon EFS is a shared file system with lower throughput compared to Lustre, and it is not optimized for the high throughput needed for deep learning training.

Option C is incorrect because while EBS io2 volumes provide high IOPS, copying data using AWS DataSync introduces an extra step and does not provide the seamless caching and high aggregate throughput that FSx for Lustre offers. Option D is incorrect because instance store volumes are ephemeral and not persistent; they would require re-copying the data each time the instance is stopped, and they lack the integration with S3 that FSx for Lustre provides.

Practice this question →

263

MCQeasy

A data scientist needs to transform raw JSON data from an S3 bucket into Parquet format using AWS Glue. The job must be cost-effective and run only when new data arrives. Which solution should be used?

A.Create a Glue crawler that runs continuously.

B.Schedule a Glue ETL job to run every hour.

C.Use Glue DataBrew to transform data and schedule it daily.

D.Create a Glue ETL job triggered by an S3 event notification via Lambda.

AnswerD

Event-driven trigger ensures cost-effectiveness.

Why this answer

It uses an S3 event notification to invoke a Lambda function, which then triggers an AWS Glue ETL job only when new data arrives. This event-driven architecture ensures cost-effectiveness by avoiding continuous or scheduled runs, and it directly transforms raw JSON into Parquet format as required.

Exam trap

The trap here is that candidates may confuse Glue crawlers (which only catalog metadata) with Glue ETL jobs (which transform data), or assume scheduled jobs are always cost-effective without considering event-driven triggers.

How to eliminate wrong answers

Option A is wrong because a Glue crawler runs continuously to update the Data Catalog, not to transform data into Parquet; it would incur unnecessary costs and does not perform ETL transformations. Option B is wrong because scheduling a Glue ETL job every hour runs regardless of whether new data has arrived, leading to wasted compute resources and higher costs. Option C is wrong because Glue DataBrew is a visual data preparation tool, not designed for automated, event-driven ETL transformations; scheduling it daily would also run even without new data and is less cost-effective than an event-triggered approach.

Practice this question →

264

MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data must be transformed before delivery using AWS Lambda. The Lambda function adds a timestamp field. The Firehose stream receives up to 10,000 records per second. The transformation currently takes 500 ms per record. What should the team do to ensure the transformation can keep up with the incoming data without data loss?

A.Increase the number of shards in the Kinesis stream.

B.Place the Lambda function in a VPC to improve network performance.

C.Increase the Lambda concurrency limit for the function to handle parallel invocations.

D.Increase the S3 buffer size and buffer interval in the Firehose delivery stream.

AnswerC

Correct. Increasing the Lambda concurrency limit allows multiple instances of the function to run in parallel, enabling it to keep up with the high record rate and preventing data loss.

Why this answer

Increasing the Lambda concurrency limit allows more parallel invocations, enabling the function to process the high throughput of 10,000 records per second (each taking 500 ms) without falling behind. Option A is incorrect because shards are a concept for Kinesis Data Streams, not Firehose. Option B is incorrect: placing the Lambda function in a VPC typically adds network latency and does not improve performance for this simple transformation.

Option D is incorrect: increasing S3 buffer size/interval may delay data delivery but does not increase transformation capacity.

Practice this question →

265

MCQmedium

A data engineer needs to ingest data from an on-premises Apache Kafka cluster into Amazon S3 with minimal latency (under 5 minutes) for real-time analytics. The data volume is approximately 10 MB per second. Which solution is MOST cost-effective and meets the latency requirement?

A.Use Amazon MSK to mirror the on-premises Kafka cluster, then use Kinesis Firehose to write to S3

B.Use Amazon S3 Transfer Acceleration for direct uploads from on-premises

C.Use Amazon Kinesis Data Streams with a Direct Connect connection from on-premises

D.Set up a VPN connection and use AWS Lambda to consume from Kafka and write to S3

AnswerA

MSK provides managed Kafka with low latency, and Firehose can buffer and write to S3 every 60 seconds.

Why this answer

Amazon MSK (Managed Streaming for Apache Kafka) can mirror the on-premises Kafka topics to the cloud with near-real-time replication (typically under 1 minute), and then Kinesis Firehose can be configured with a 60-second buffer interval to deliver data to Amazon S3, meeting the under-5-minute latency requirement. This solution is cost-effective because MSK eliminates the need to manage Kafka infrastructure, and Firehose provides serverless, pay-per-use data delivery without provisioning servers.

Option B (S3 Transfer Acceleration) is designed for accelerating file uploads over long distances, not for continuous streaming of data from Kafka; it would require additional application changes and is not suitable for real-time streaming.

Option C (Kinesis Data Streams with Direct Connect) would require re-architecting the data pipeline to send data directly to Kinesis, adding complexity and cost (Direct Connect bandwidth). Kinesis Data Streams also requires managing shards and does not natively integrate with Kafka replication.

Option D (VPN + Lambda) introduces additional latency due to VPN overhead and Lambda cold starts, and Lambda is not optimized for high-throughput streaming of 10 MB/s; it would likely cause throttling or increased costs. Therefore, option A is the most cost-effective and meets the latency requirement.

Practice this question →

266

MCQmedium

A data engineering team needs to process streaming data from thousands of IoT devices. They want to aggregate data in 1-minute windows and store results in an S3 data lake for downstream analytics. Which architecture should they use?

A.Use AWS Glue ETL jobs running in streaming mode to read from Kinesis Data Streams, apply window aggregations, and write to S3.

B.Use Kinesis Data Streams with enhanced fan-out and multiple consumers to aggregate windows, then write to S3 via Firehose.

C.Use Kinesis Data Streams, trigger a Lambda function for 1-minute window aggregation using Python, and write results to S3.

D.Use Kinesis Data Analytics for SQL-based windowed aggregations and send results to Kinesis Data Firehose for delivery to S3.

AnswerD

Kinesis Data Analytics supports tumbling windows and continuous queries; Firehose is the natural sink for S3.

Why this answer

Kinesis Data Analytics for SQL Applications is purpose-built for real-time windowed aggregations on streaming data, such as 1-minute tumbling windows. It can directly consume from Kinesis Data Streams, perform the aggregation using standard SQL, and output the results to Kinesis Data Firehose, which reliably delivers the aggregated data to an S3 data lake with built-in buffering and compression.

Exam trap

The trap here is that candidates often assume Lambda is suitable for real-time windowed aggregation, overlooking its stateless nature and execution limits, while Kinesis Data Analytics is the native AWS service for this exact use case.

How to eliminate wrong answers

Option A is wrong because AWS Glue ETL jobs in streaming mode are designed for batch-oriented processing and do not natively support real-time windowed aggregations with sub-minute latency; they are better suited for near-real-time or micro-batch scenarios with higher overhead. Option B is wrong because Kinesis Data Streams with enhanced fan-out and multiple consumers only improves throughput for parallel consumption but does not provide built-in window aggregation logic; writing directly to S3 via Firehose from consumers would require custom aggregation code, defeating the purpose. Option C is wrong because triggering a Lambda function for 1-minute window aggregation is impractical due to Lambda's maximum execution timeout of 15 minutes and lack of state management across invocations; it would require external state stores like DynamoDB or ElastiCache, adding complexity and latency.

Practice this question →

267

Multi-Selecthard

Which THREE AWS services can be used together to build a serverless data pipeline that ingests streaming data, transforms it, and loads it into Amazon Redshift for analysis?

Select 3 answers

A.Amazon EMR

B.Amazon SQS

C.Amazon Kinesis Data Firehose

D.Amazon Kinesis Data Streams

E.AWS Lambda

AnswersC, D, E

Delivers transformed data directly to Redshift.

Why this answer

Amazon Kinesis Data Firehose (C) is the correct service because it is designed to reliably capture, transform, and load streaming data into Amazon Redshift with near-real-time latency. It can invoke AWS Lambda for on-the-fly data transformation (e.g., converting JSON to Parquet) and directly stream the processed records into Redshift via the Redshift COPY command, making it the central orchestration component for a serverless pipeline.

Exam trap

The trap here is that candidates often confuse Amazon EMR as a serverless option, but EMR requires cluster management and is not serverless, whereas Kinesis Data Firehose and Lambda provide a fully managed, serverless ingestion and transformation layer.

Practice this question →

268

MCQmedium

A data engineering team needs to build a data lake on Amazon S3 that will be queried by Amazon Athena and Amazon Redshift Spectrum. The data will be ingested from multiple sources in various formats (CSV, JSON, Parquet). Which partitioning strategy will provide the best query performance for date-range queries?

A.Partition by date with one partition per day in a flat structure.

B.Do not partition; let Athena scan the entire dataset.

C.Partition by year, month, and day in a hierarchical structure.

D.Partition by source system first, then by date.

AnswerC

Hierarchical date partitioning enables partition pruning for date-range queries.

Why this answer

Partitioning by year, month, and day in a hierarchical structure minimizes the amount of data scanned by Amazon Athena and Redshift Spectrum for date-range queries. Athena and Redshift Spectrum both charge per byte scanned, so reducing the scan size directly improves performance and reduces cost. A hierarchical partition structure (e.g., s3://bucket/year=2023/month=11/day=01/) allows the query engine to prune partitions at each level, efficiently skipping irrelevant directories for queries like WHERE date BETWEEN '2023-11-01' AND '2023-11-30'.

Exam trap

The trap here is that candidates may think a flat daily partition is simpler and sufficient, but they overlook that hierarchical partitioning (year/month/day) provides better partition pruning for range queries spanning months or years, which is a key optimization for Athena and Redshift Spectrum's cost and performance model.

How to eliminate wrong answers

Option A is wrong because a flat partition per day structure (e.g., s3://bucket/date=2023-11-01/) does not allow partition pruning at year or month granularity; for a multi-month query, Athena must list and evaluate all day-level partitions, increasing metadata overhead and potentially slowing performance. Option B is wrong because not partitioning forces Athena and Redshift Spectrum to perform a full table scan of all data, which is extremely inefficient for date-range queries, leading to high costs and slow query times due to scanning terabytes of irrelevant data. Option D is wrong because partitioning by source system first then by date is suboptimal for date-range queries; if a query spans multiple source systems, Athena must scan all source system partitions even if the date range is narrow, negating the benefit of date-based pruning and increasing scan size.

Practice this question →

269

MCQmedium

A company uses Amazon Kinesis Data Analytics for Apache Flink to process real-time clickstream data. The application uses event time and watermarks for windowed aggregations. The team notices that the output from tumbling windows is delayed, and many late records are being dropped. What is the MOST likely cause?

A.The checkpointing interval is too long, causing state to be lost

B.The parallelism is too low, causing backpressure

C.The source is marking itself as idle, causing watermarks to stall

D.The allowed lateness is set too low, causing late records to be discarded

AnswerD

Low allowed lateness means records arriving after the watermark are dropped.

Why this answer

The described symptoms—delayed output and dropped late records—are classic indicators that the `allowedLateness` parameter is set too low. In Apache Flink, event-time processing relies on watermarks to determine when a window is complete; if `allowedLateness` is too short, any record arriving after the watermark passes the window's end time is discarded as late. The team's observation that many late records are being dropped directly points to this configuration issue.

Exam trap

The trap here is that candidates confuse watermark stall (which delays output) with late record dropping—both involve watermarks, but stalled watermarks prevent window closure (no output), whereas low `allowedLateness` closes windows on time but discards subsequent late arrivals.

How to eliminate wrong answers

Option A is wrong because checkpointing interval affects fault tolerance and state recovery, not the handling of late-arriving data within a running window; a long checkpoint interval would cause longer recovery time after a failure, not delayed output or dropped records. Option B is wrong because low parallelism can cause backpressure and throughput issues, but it does not cause late records to be dropped—backpressure slows processing but does not discard data based on event time. Option C is wrong because a source marking itself as idle would cause watermarks to stall (stop advancing), which would delay window firing indefinitely, not drop late records; in fact, stalled watermarks would cause windows to never close, so records would accumulate rather than be dropped.

Practice this question →

270

MCQhard

An e-commerce company uses Amazon Redshift for analytics. The data engineering team needs to load daily sales data from an S3 bucket that receives new files every hour. The data must be loaded into Redshift with minimal impact on query performance during the day, and they need to handle late-arriving data (files that appear after the daily load). Which approach should they use?

A.Use AWS Glue ETL to copy the data from S3 to Redshift, overwriting the existing data each day.

B.Use a staging table to load data incrementally with a MERGE operation, and schedule a late-arriving data job to merge files that arrive after the daily load.

C.Stream the data from S3 using Amazon Kinesis Firehose to load into Redshift continuously.

D.Use Amazon Redshift Spectrum to query data directly from S3 and create external tables.

AnswerB

Staging tables allow incremental upserts and handling of late data without blocking queries.

Why this answer

It uses a staging table to incrementally load data with a MERGE operation, which minimizes impact on query performance by avoiding full table overwrites. The separate late-arriving data job handles files that appear after the daily load, ensuring completeness without blocking ongoing queries. This approach aligns with Redshift's best practices for incremental loads and late-arriving data handling.

Exam trap

The trap here is that candidates often confuse continuous streaming (Option C) with batch incremental loading, not realizing that Kinesis Firehose is optimized for real-time streams, not for handling sporadic late-arriving files in a batch context.

How to eliminate wrong answers

Option A is wrong because overwriting existing data each day with AWS Glue ETL would cause significant performance impact during the day, as it requires a full table reload and can block concurrent queries. Option C is wrong because streaming data from S3 using Amazon Kinesis Firehose into Redshift continuously is not designed for batch-oriented late-arriving data scenarios and can lead to high costs and performance degradation due to frequent micro-batches. Option D is wrong because using Redshift Spectrum to query data directly from S3 does not load data into Redshift, so it cannot support the requirement of loading data into Redshift for analytics, and it would not handle late-arriving data efficiently for ongoing queries.

Practice this question →

271

Multi-Selecteasy

Which TWO AWS services are suitable for real-time stream processing?

Select 2 answers

A.Amazon Athena

B.AWS Glue

C.Amazon EMR

D.Amazon Kinesis Data Analytics

E.AWS Lambda

AnswersD, E

Kinesis Data Analytics processes streaming data in real-time.

Why this answer

Amazon Kinesis Data Analytics and AWS Lambda can process streams in real-time. AWS Glue is batch-oriented, Amazon EMR can process streams but is more batch, and Amazon Athena is for ad-hoc SQL queries on S3.

Practice this question →

272

MCQmedium

A data engineer needs to transfer 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The company has a 1 Gbps internet connection and wants to complete the transfer within 5 days. What is the MOST cost-effective and reliable solution?

A.Use AWS Snowball Edge device to physically ship the data

B.Use S3 multipart upload over the internet

C.Set up AWS Direct Connect and transfer over the dedicated line

D.Use S3 Transfer Acceleration to speed up the transfer

AnswerA

Snowball can transfer 50 TB in a few days, cost-effective for large data.

Why this answer

AWS Snowball Edge is the most cost-effective and reliable solution because transferring 50 TB over a 1 Gbps internet connection would take approximately 5.5 days under ideal conditions (50 TB * 1024 GB/TB * 8 bits/byte / (1 Gbps * 86400 seconds/day) ≈ 4.74 days, but real-world overhead, congestion, and retransmissions push it beyond 5 days). Snowball Edge provides a physical appliance that can be shipped, avoiding network bandwidth limitations entirely, and is designed for large-scale data transfers where internet speeds are insufficient.

Exam trap

The trap here is that candidates underestimate the real-world throughput of a 1 Gbps link (which rarely exceeds 800 Mbps due to TCP overhead and congestion) and overestimate the speed of S3 Transfer Acceleration, assuming it can magically bypass bandwidth limits.

How to eliminate wrong answers

Option B is wrong because S3 multipart upload over the internet still relies on the 1 Gbps connection, which cannot reliably transfer 50 TB within 5 days due to bandwidth constraints, latency, and potential packet loss. Option C is wrong because AWS Direct Connect requires weeks to provision and incurs ongoing monthly costs, making it neither cost-effective nor timely for a one-time transfer. Option D is wrong because S3 Transfer Acceleration optimizes network path but does not increase bandwidth beyond the 1 Gbps internet connection, so it cannot meet the 5-day deadline for 50 TB.

Practice this question →

273

MCQhard

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The stream has 8 shards. A Lambda function processes each record and writes to Amazon DynamoDB. The Lambda function sometimes fails due to DynamoDB write throttling, causing duplicate processing of records after retries. The data engineering team needs to ensure exactly-once processing semantics for the DynamoDB writes. What should the team do?

A.Use an Amazon SQS FIFO queue between Kinesis and Lambda to deduplicate records.

B.Configure the Lambda event source mapping with a maximum retry count of 0 and a DLQ.

C.Increase the DynamoDB write capacity units to avoid throttling.

D.Use DynamoDB conditional writes with the Kinesis sequence number as a unique attribute to make writes idempotent.

AnswerD

Conditional writes based on the sequence number ensure each record is written only once.

Why this answer

Using DynamoDB conditional writes with the Kinesis sequence number as a unique attribute ensures idempotency. When processing a record, the Lambda function can attempt a conditional write that only succeeds if an item with that sequence number does not already exist. If the write fails due to a condition check, it means the record was already processed, so the function can skip it.

This achieves exactly-once semantics even if the same record is delivered multiple times due to retries. Option A is incorrect because an SQS FIFO queue between Kinesis and Lambda would add latency and complexity; Kinesis already provides ordering and at-least-once delivery, and using a FIFO queue does not guarantee that the Lambda function will not process the same record multiple times within its retry logic. Option B is incorrect because setting maximum retry count to 0 with a DLQ simply discards failed records after the first failure, which does not provide exactly-once processing; it results in at-most-once semantics and potential data loss.

Option C is incorrect because increasing DynamoDB write capacity may reduce throttling but does not eliminate duplicate processing; the same record can still be retried and written multiple times if the function retries, leading to duplicate writes.

Practice this question →

274

Multi-Selectmedium

A company uses Amazon Kinesis Data Streams to ingest clickstream data. They need to archive raw data to S3 every hour and also enable real-time processing with sub-second latency. Which TWO actions should they take? (Choose two.)

Select 2 answers

A.Use Kinesis Data Analytics to write output to S3.

B.Configure a Lambda function as a consumer of the stream for real-time processing.

C.Use S3 events to trigger a Lambda function that reads from the stream.

D.Create a Kinesis Data Firehose delivery stream with S3 as destination and set a buffer interval of 3600 seconds.

E.Install the Kinesis Agent on an EC2 instance to write data to S3.

AnswersA, B

Correct. Kinesis Data Analytics (Flink) can read raw data from the stream and write it to S3 with a customizable window, such as every hour, serving as an archival mechanism.

Why this answer

AWS Lambda can be configured as a consumer of a Kinesis Data Stream using event source mapping, enabling real-time processing with sub-second latency. Option A is correct because Kinesis Data Analytics (now Amazon Managed Service for Apache Flink) can read from a Kinesis stream and write output to S3, making it suitable for archiving raw data every hour—e.g., using a Flink sink with a tumbling window of 1 hour. Option D is incorrect because Kinesis Data Firehose has a maximum buffer interval of 900 seconds (15 minutes), not 3600 seconds; thus it cannot archive data exactly every hour as specified.

Option C is incorrect because S3 events trigger Lambda on object creation in S3, not on real-time stream data. Option E is incorrect because the Kinesis Agent is used to send data from EC2 to Kinesis, not to S3 directly.

Exam trap

Candidates often overlook that Kinesis Data Firehose has a maximum buffer interval of 900 seconds, making it unsuitable for hourly archives, and they may also underestimate Kinesis Data Analytics (Flink) as an archiving solution, assuming it only processes data rather than writing raw data to S3.

Practice this question →

275

MCQhard

A company uses AWS Glue ETL jobs to transform CSV data from an S3 bucket into Parquet. The jobs often fail with memory errors when processing large datasets. They want to minimize cost and improve reliability. What should they do?

A.Use G.1X or G.2X worker types and increase the number of DPUs per worker.

B.Use Amazon Athena with CTAS queries to convert the data to Parquet.

C.Switch to S3 Batch Operations with AWS Lambda to process the files individually.

D.Increase the number of workers in the Glue job configuration.

AnswerA

G.1X workers provide more memory and vCPU per worker, reducing OOM errors for memory-intensive transformations.

Why this answer

The G.1X and G.2X worker types provide more memory per worker (16 GB and 32 GB, respectively) compared to the standard G.0X worker (4 GB). By using these worker types and increasing the number of DPUs per worker, you allocate more memory to each task, reducing out-of-memory errors when processing large datasets. This approach also optimizes cost by using fewer, more powerful workers instead of many underpowered ones, improving reliability without unnecessary scaling.

Exam trap

The trap here is that candidates often assume increasing the number of workers (Option D) is the universal fix for performance issues, but in Glue, memory errors are typically caused by insufficient per-worker memory, not a lack of parallelism.

How to eliminate wrong answers

Option B is wrong because Amazon Athena with CTAS queries is a serverless query service, not a data transformation engine; it cannot handle the complex ETL logic (e.g., custom transforms, joins) that Glue jobs perform, and it incurs costs per query scanned, which can be higher for large datasets. Option C is wrong because S3 Batch Operations with AWS Lambda processes files individually, which lacks the distributed, parallel processing capabilities of Glue; it would be slower and more error-prone for large datasets, and Lambda has a 15-minute timeout and limited memory (up to 10 GB), making it unsuitable for memory-intensive transformations. Option D is wrong because simply increasing the number of workers does not address the root cause of memory errors; it adds more parallel tasks but each worker still has the same limited memory (4 GB for G.0X), so tasks can still fail if individual partitions exceed that memory.

Practice this question →

276

MCQmedium

A company is building a data pipeline to process streaming data from IoT devices. The data is ingested via Amazon Kinesis Data Streams. Each record is about 1 KB. The company wants to use AWS Lambda for real-time transformations and then store the results in Amazon DynamoDB. The expected throughput is 10,000 records per second. The Lambda function currently runs in about 200 ms. The company is concerned about Lambda concurrency limits and wants to ensure there are no throttling errors. The default concurrency limit for Lambda is 1,000. Which approach should the team take to handle the expected throughput without throttling?

A.Increase the Lambda function memory to 3,000 MB to reduce the execution time below 100 ms.

B.Use Amazon Kinesis Data Firehose instead of Lambda to load data directly into DynamoDB.

C.Reduce the Lambda batch size to 10 so that each invocation processes fewer records, reducing the time per invocation.

D.Increase the number of shards in the Kinesis Data Stream to 10 and set the Lambda batch size to 100.

AnswerD

With 10 shards and batch size 100, at most 10 concurrent Lambda invocations, well within limits.

Why this answer

Increasing the number of shards to 10 ensures that the Kinesis stream can support up to 10 concurrent Lambda invocations (one per shard). With a batch size of 100, each invocation processes 100 records, resulting in 100 invocations per second (10,000 records / 100 per batch). At 200 ms per invocation, the required concurrency is 100 * 0.2 = 20, well within the default 1,000 concurrency limit.

Option A is incorrect because reducing the batch size to 10 would increase invocations to 1,000 per second, requiring 200 concurrent executions, which still fits but is less efficient; the main issue is that reducing batch size does not reduce throttling risk as it increases invocation rate. Option B is incorrect because Kinesis Data Firehose does not natively support Lambda for per-record transformations before writing to DynamoDB; it primarily targets S3, Redshift, or Elasticsearch. Option C is incorrect because increasing Lambda memory typically reduces execution time but does not lower concurrency requirements; moreover, 3,000 MB may not reduce time below 100 ms enough to avoid throttling at high throughput.

Practice this question →

277

Multi-Selecteasy

A data engineer is designing a data pipeline that uses Amazon Kinesis Data Streams to ingest sensor data. The data must be processed in real-time, and the results must be stored in Amazon DynamoDB. Which TWO AWS services can be used together to achieve this? (Choose TWO.)

Select 2 answers

A.Amazon Athena

B.Amazon Kinesis Data Analytics

C.AWS Glue

D.Amazon S3

E.AWS Lambda

AnswersB, E

Kinesis Data Analytics can process streaming data in real-time.

Why this answer

Amazon Kinesis Data Analytics (B) can process streaming data from Kinesis Data Streams in real-time using SQL or Apache Flink, and AWS Lambda (E) can be used as a consumer to write the processed results to DynamoDB. This combination provides a fully managed, serverless pipeline for real-time ingestion, processing, and storage.

Exam trap

The trap here is that candidates often select Amazon Athena or AWS Glue thinking they can handle real-time streaming, but both are batch-oriented services and cannot meet the sub-second latency requirement of Kinesis Data Streams processing.

Practice this question →

278

Multi-Selecthard

A data engineering team is migrating on-premises Hadoop workloads to AWS. The workloads include batch processing using Apache Spark and interactive SQL queries. The data is stored in HDFS. Which TWO AWS services should be used to replace HDFS and provide a scalable, durable storage layer? (Choose TWO.)

Select 2 answers

A.Amazon EMR with EMRFS

B.Amazon S3

C.Amazon EBS

D.Amazon FSx for Lustre

E.Amazon RDS

AnswersA, B

EMRFS allows EMR to use S3 as a replacement for HDFS.

Why this answer

Amazon S3 provides a highly durable (99.999999999% durability), scalable, and cost-effective object storage layer that replaces HDFS for Hadoop workloads. Amazon EMR with EMRFS allows Spark and Hive to read and write data directly from S3, treating it as a native filesystem with features like consistent view and read-after-write consistency, making it the ideal compute layer for batch processing and interactive SQL queries.

Exam trap

The trap here is that candidates often confuse Amazon EBS or FSx for Lustre as viable HDFS replacements, not realizing that HDFS is a distributed filesystem designed for shared access across many nodes, whereas S3 with EMRFS provides the same semantics with superior durability and scalability.

Practice this question →

279

Multi-Selecteasy

Which TWO AWS services can be used to schedule and orchestrate a data pipeline that includes multiple steps such as data extraction, transformation, and loading? (Choose 2.)

Select 2 answers

A.AWS Lambda

B.AWS Glue

C.Amazon Managed Workflows for Apache Airflow (MWAA)

D.AWS Step Functions

E.Amazon CloudWatch Events

AnswersC, D

MWAA is a managed orchestration service for data pipelines.

Why this answer

Amazon Managed Workflows for Apache Airflow (MWAA) and AWS Step Functions are both designed for orchestrating multi-step workflows, including data pipelines with extraction, transformation, and loading (ETL) steps. MWAA provides managed Apache Airflow, allowing you to define complex workflows as Directed Acyclic Graphs (DAGs) with scheduling, dependency management, and monitoring. Step Functions offers state machines with visual workflow design, conditional logic, error handling, and integration with over 200 AWS services.

In contrast, AWS Lambda is ideal for serverless function execution but not for orchestrating multi-step processes with dependencies; AWS Glue is primarily an ETL service with basic job triggers and crawlers, lacking native DAG-based orchestration with conditions; Amazon CloudWatch Events (now Amazon EventBridge) is for event-driven scheduling and does not support complex workflow orchestration with multiple steps and dependencies.

Exam trap

The trap here is that candidates often confuse AWS Glue's built-in job triggers and crawlers as sufficient for orchestration, but Glue lacks native DAG-based workflow management with conditional logic and dependency resolution, which is why MWAA and Step Functions are the correct choices for multi-step pipeline orchestration.

Practice this question →

280

MCQeasy

A data engineer needs to schedule an AWS Glue ETL job to run every hour. Which service should they use for scheduling?

A.AWS Lambda

B.Amazon CloudWatch Events

C.Amazon Simple Queue Service (SQS)

D.AWS Step Functions

AnswerB

CloudWatch Events can trigger Glue jobs on a schedule.

Why this answer

Amazon CloudWatch Events (now part of Amazon EventBridge) can trigger an AWS Glue ETL job on a schedule using a cron or rate expression. This is the native, serverless scheduling service for running jobs at fixed intervals, such as every hour, without needing to manage any infrastructure.

Exam trap

The trap here is that candidates may confuse AWS Lambda as a scheduler because it can be used to run code on a schedule via CloudWatch Events, but the question asks for the service used for scheduling, not for executing the scheduled action.

How to eliminate wrong answers

Option A is wrong because AWS Lambda is a compute service for running code in response to events, not a scheduling service; while Lambda can be used to trigger Glue jobs via custom code, it requires additional setup and is not the direct scheduling mechanism. Option C is wrong because Amazon SQS is a message queue service for decoupling application components, not a scheduler; it cannot natively trigger Glue jobs on a time-based schedule. Option D is wrong because AWS Step Functions is a workflow orchestration service that can coordinate multiple AWS services, but it is not designed for simple time-based scheduling; using it solely for hourly triggers would be over-engineering and incur unnecessary complexity and cost.

Practice this question →

281

MCQmedium

Refer to the exhibit. An IAM policy is attached to a data engineering role. The role is used by an AWS Glue ETL job that reads from 'raw/' and writes to 'processed/'. The job fails with an access denied error when trying to write to 'processed/'. What is the likely cause?

A.The Deny statement on s3:DeleteObject prevents overwriting objects.

B.The role is not correctly attached to the Glue job.

C.The policy does not allow both s3:GetObject and s3:PutObject on the same resource.

D.The policy specifies incorrect ARN for the 'processed' folder.

AnswerA

If the job tries to overwrite an existing object, it needs DeleteObject permission.

Why this answer

The Deny statement on s3:DeleteObject explicitly denies the permission required to overwrite an existing object in S3. When an AWS Glue job writes to 'processed/' and attempts to overwrite an existing object, S3 requires both s3:PutObject and s3:DeleteObject permissions (since overwrite involves delete then put). The Deny on DeleteObject causes the access denied error.

Option B is incorrect because the role is presumed to be attached correctly per the scenario. Option C is incorrect because the policy allows both GetObject and PutObject on the same resource, as is typical. Option D is incorrect because the ARN for the 'processed' folder is correctly specified; the error is due to the Deny, not incorrect ARN.

Practice this question →

282

MCQhard

A company uses Amazon SageMaker to train and deploy machine learning models. The training data is stored in Amazon S3 (Parquet format, 10 TB). The data scientists have been running training jobs using the File mode input, but the jobs are taking too long due to data download time. They want to reduce the training start-up time and overall training time. Which solution is MOST cost-effective and efficient?

A.Configure the SageMaker training job to use Pipe mode, which streams data directly from S3 without downloading to the instance's local storage.

B.Use S3 Transfer Acceleration to speed up the data transfer from S3 to the training instance.

C.Use larger EC2 instances with more vCPUs and memory to speed up the training process.

D.Enable Elastic Fabric Adapter (EFA) on the training instances to improve network throughput.

AnswerA

Pipe mode reduces start-up time by streaming data, and it is cost-effective as it avoids EBS volume costs associated with File mode.

Why this answer

Pipe mode in SageMaker streams training data directly from Amazon S3 to the training algorithm without first downloading it to the instance's local storage. This eliminates the data download step, significantly reducing startup time and overall training time for large datasets like 10 TB. It is the most cost-effective because it avoids the need for larger instances or additional data transfer acceleration services.

Exam trap

The trap here is that candidates often confuse Pipe mode with File mode, assuming both require downloading data, or they over-engineer the solution by choosing expensive network accelerators or larger instances when the simplest streaming approach is both faster and cheaper.

How to eliminate wrong answers

Option B is wrong because S3 Transfer Acceleration is designed to speed up uploads to S3 over long distances, not downloads from S3 to SageMaker training instances, and it incurs additional costs without addressing the core issue of download time. Option C is wrong because using larger EC2 instances with more vCPUs and memory does not reduce the data download time; it only increases compute capacity, which may not help if the bottleneck is I/O from downloading data. Option D is wrong because Elastic Fabric Adapter (EFA) improves inter-node network communication for distributed training, but it does not accelerate data transfer from S3 to the instance, which is the primary bottleneck here.

Practice this question →

283

Multi-Selectmedium

Which TWO AWS services can be used to move data from an on-premises database to Amazon S3 on a recurring schedule without writing custom code? (Choose 2.)

Select 2 answers

A.AWS Glue

B.AWS Snowball Edge

C.AWS Database Migration Service (AWS DMS)

D.Amazon Athena

E.Amazon Kinesis Data Firehose

AnswersA, C

Glue can run scheduled ETL jobs from JDBC sources to S3.

Why this answer

AWS Glue is correct because it provides a fully managed ETL service that can run crawlers and jobs on a recurring schedule to extract data from on-premises databases (via JDBC connections) and write it to Amazon S3 without requiring any custom code. AWS DMS is correct because it supports continuous replication or scheduled tasks to migrate data from on-premises databases to S3 as a target, using built-in transformation capabilities and no custom scripting.

Exam trap

The trap here is that candidates often confuse AWS DMS with a one-time migration tool, overlooking its built-in scheduling and CDC capabilities, or they mistakenly think Kinesis Data Firehose can pull from on-premises databases via JDBC when it only accepts streaming data from AWS sources or custom producers.

Practice this question →

284

Multi-Selectmedium

A company uses AWS Glue to run ETL jobs. The data engineer wants to monitor job performance and troubleshoot failures. Which THREE AWS services or features should they use together? (Choose three.)

Select 3 answers

A.AWS Glue job bookmarks

B.Amazon S3 Event Notifications

C.Amazon Athena

D.Amazon CloudWatch Logs

E.Amazon CloudWatch metrics

AnswersA, D, E

Job bookmarks track processed data and help identify failures.

Why this answer

Correct options: A, D, E. AWS Glue job bookmarks track processed data to avoid reprocessing, Amazon CloudWatch Logs stores Glue job logs for troubleshooting failures, and Amazon CloudWatch metrics provide performance monitoring metrics. Option B (Amazon S3 Event Notifications) can trigger jobs but not monitor performance or troubleshoot.

Option C (Amazon Athena) is used for querying data, not for monitoring Glue jobs.

Practice this question →

285

Matchingmedium

Match each SageMaker optimization technique to its description.

Drag a concept onto its matching description — or click a concept then click the description.

Concepts

Matches

Train across multiple GPUs or instances

Hyperparameter optimization with Bayesian search

Use spot instances for cost savings

Stream data directly from S3 for faster training

Monitor training and detect issues

Why these pairings

Managed Spot Training reduces cost via spot instances; Data Parallelism speeds training by distributing data; Automatic Model Tuning finds optimal hyperparameters; Compilation (Neo) optimizes for inference hardware. Common confusions include swapping definitions between these techniques.

Practice this question →

286

Multi-Selectmedium

Which TWO of the following are valid ways to reduce query costs in Amazon Athena? (Choose 2)

Select 2 answers

A.Use UNLOAD to export query results to S3

B.Partition the data in S3

C.Increase the query timeout limit

D.Use columnar storage formats like Parquet

E.Enable encryption at rest on S3

AnswersB, D

Partitioning limits data scanned per query.

Why this answer

(partitioning data) and Option D (using columnar formats like Parquet) are correct because both reduce the amount of data scanned by Athena queries, directly lowering costs. Option A (UNLOAD) exports results but does not reduce query costs. Option C (increasing query timeout) does not affect data scanned.

Option E (encryption at rest) does not reduce costs.

Practice this question →

287

Multi-Selecteasy

A company wants to centralize logging from multiple AWS accounts and on-premises servers. The logs must be stored cost-effectively and be searchable. Which TWO services should be used? (Choose TWO.)

Select 2 answers

A.Amazon CloudWatch Logs

B.Amazon Redshift

C.Amazon Athena

D.Amazon Kinesis Data Streams

E.Amazon S3

AnswersC, E

Athena can query logs directly on S3.

Why this answer

Amazon S3 (option E) provides a cost-effective, durable storage layer for centralized logs, while Amazon Athena (option C) enables serverless SQL-based searching directly on the log data stored in S3 without needing to load it into a separate database. Together, they allow you to store logs cheaply and query them on demand, meeting the requirements of cost-effectiveness and searchability.

Exam trap

The trap here is that candidates often confuse Amazon CloudWatch Logs with a centralized log storage solution, but it is actually a real-time monitoring service with high storage costs, not a cost-effective long-term archive; similarly, Kinesis Data Streams is mistaken for a storage service when it is purely a streaming ingestion layer.

Practice this question →

288

Multi-Selecthard

A company is using Amazon Kinesis Data Streams with a Lambda consumer. The Lambda function writes results to an S3 bucket. The team wants to ensure that each record is processed exactly once and in order. Which TWO configurations should the team implement? (Choose 2.)

Select 2 answers

A.Set the batch size to 1

B.Increase the Lambda function's reserved concurrency

C.Set the parallelization factor to 1

D.Configure a dead-letter queue for failed records

E.Enable S3 bucket versioning to track duplicates

AnswersC, E

This ensures a single Lambda instance processes each shard, maintaining order.

Why this answer

Setting the parallelization factor to 1 ensures that each shard of the Kinesis Data Stream is processed by only one Lambda instance at a time, preserving the order of records within that shard. Option E is correct because enabling S3 bucket versioning helps track duplicates: if the same record is written multiple times, S3 versioning creates multiple versions, allowing the system to detect and handle duplicates, contributing to exactly-once processing semantics.

Exam trap

The trap here is that candidates often confuse batch size with concurrency control, mistakenly believing that setting batch size to 1 alone is sufficient for ordered processing, while ignoring the parallelization factor that governs how many concurrent invocations can process records from the same shard. Additionally, versioning is key for duplicate detection, not just batch size adjustments.

Practice this question →

289

MCQhard

A company runs a streaming data pipeline using Amazon Kinesis Data Streams with 10 shards. The pipeline ingests sensor data from thousands of devices. Each device sends a JSON payload every 5 seconds. The payload size is approximately 2 KB. The data is consumed by a fleet of EC2 instances running a custom Java application that uses the Kinesis Client Library (KCL). Over the past week, the company has observed that the consumer application is experiencing increased latency, and the Kinesis stream's 'GetRecords.IteratorAgeMilliseconds' CloudWatch metric is consistently above 10 seconds. The company has verified that the EC2 instances have sufficient CPU and memory resources. The KCL application is configured with 10 workers, one per shard. The application processes each record by performing a simple transformation and writing to Amazon DynamoDB. The DynamoDB table has sufficient write capacity and is not throttling. The company wants to reduce the iterator age to under 2 seconds. Which action should the company take?

A.Replace Kinesis Data Streams with Amazon Kinesis Data Firehose

B.Increase the write capacity of the DynamoDB table

C.Increase the number of shards in the Kinesis stream to 20

D.Increase the number of KCL workers to 20

AnswerC

More shards increase the number of concurrent consumers and reduce iterator age.

Why this answer

The observed high iterator age indicates that the consumer is falling behind. Since the EC2 instances have sufficient resources and DynamoDB is not throttling, the bottleneck is the number of shards. Each shard provides a fixed amount of read throughput (up to 2 MB/s and 5 reads/s).

With 10 shards, the consumer's aggregate read throughput is limited. Increasing the number of shards to 20 doubles the read capacity, allowing the KCL workers to fetch records faster and reduce the iterator age to under 2 seconds. Option A is incorrect because Kinesis Data Firehose is a delivery service that buffers and writes to destinations like S3 or Redshift; it does not support custom transformations with a Java application.

Option B is incorrect because the DynamoDB table has sufficient write capacity and is not throttling, so increasing it would not address the read-side bottleneck. Option D is incorrect because KCL enforces one worker per shard; adding more workers than shards does not increase parallelism. The workers would sit idle or conflict, since each shard can only be processed by a single worker at a time.

Practice this question →

290

MCQhard

A data science team uses Amazon SageMaker to train models on a dataset stored in Amazon S3. The dataset is 2 TB and is accessed by multiple training jobs. The team notices that training jobs are slow due to high S3 GET request latency. Which solution would provide the fastest and most cost-effective data access?

A.Place all training instances in a Cluster Placement Group

B.Enable S3 Transfer Acceleration on the bucket

C.Mount an Amazon FSx for Lustre file system integrated with the S3 bucket

D.Use Elastic Fabric Adapter (EFA) for training instances

AnswerC

FSx for Lustre provides a high-performance file system that can read data from S3 with low latency.

Why this answer

Amazon FSx for Lustre provides a high-performance, POSIX-compliant file system that can be directly linked to an S3 bucket, allowing training instances to access data with sub-millisecond latency instead of S3 GET request latency. This integration enables data to be read from the Lustre file system at up to hundreds of gigabytes per second of throughput, which is significantly faster than reading directly from S3, and it is cost-effective because you only pay for the storage and throughput you provision during training.

Exam trap

The trap here is that candidates confuse network-level optimizations (Placement Groups, EFA) or upload acceleration (S3 Transfer Acceleration) with the actual data access bottleneck, which is the latency of S3 GET requests when reading a large dataset repeatedly during training.

How to eliminate wrong answers

Option A is wrong because a Cluster Placement Group only reduces network latency between instances within a single Availability Zone but does not address the bottleneck of S3 GET request latency when reading data from S3. Option B is wrong because S3 Transfer Acceleration speeds up uploads to S3 over long distances by using edge locations, but it does not improve read latency for training jobs that are already in the same region as the bucket. Option D is wrong because Elastic Fabric Adapter (EFA) is a network interface that accelerates inter-instance communication for distributed training (e.g., MPI), not the data access path from S3 to the training instances.

Practice this question →

291

Multi-Selecthard

A company is using Amazon Kinesis Data Streams to ingest real-time clickstream data. The data must be processed and stored in S3 in near real-time. Which THREE services can be used together to achieve this?

Select 3 answers

A.Amazon Kinesis Data Analytics

B.Amazon Kinesis Data Firehose

C.AWS Glue ETL

D.AWS Lambda

E.Amazon EMR

AnswersA, B, D

Can process streaming data in real-time.

Why this answer

Amazon Kinesis Data Analytics is correct because it can process streaming clickstream data in real-time using SQL or Apache Flink, enabling transformations, aggregations, and filtering before the data is delivered downstream. It integrates directly with Kinesis Data Streams as a source and can output processed records to Kinesis Data Firehose for storage in Amazon S3, achieving near real-time processing and storage.

Exam trap

The trap here is that candidates often assume AWS Glue ETL can handle real-time streaming because it supports Spark Streaming, but Glue ETL jobs are fundamentally batch-oriented and not designed for continuous, low-latency ingestion from Kinesis Data Streams into S3.

Practice this question →

292

MCQeasy

A data engineer needs to store streaming data from thousands of IoT devices for real-time analytics. Which AWS service is most suitable for ingesting and storing this data for subsequent processing by Amazon Kinesis Data Analytics?

A.Amazon Kinesis Data Streams

B.Amazon S3

C.Amazon RDS

D.Amazon DynamoDB

AnswerA

Kinesis Data Streams ingests and stores streaming data in real-time for analytics.

Why this answer

Amazon Kinesis Data Streams is the most suitable service for ingesting and storing streaming data from thousands of IoT devices because it is designed for real-time data ingestion, can handle high throughput from many sources, and integrates natively with Amazon Kinesis Data Analytics for real-time analytics without needing additional storage layers. Its shard-based architecture allows for scalable, durable storage of streaming records for up to 365 days, enabling immediate processing by Kinesis Data Analytics.

Exam trap

The trap here is that candidates often confuse durable storage (S3) or database services (RDS, DynamoDB) with the real-time ingestion and buffering capabilities of a stream, overlooking that Kinesis Data Analytics requires a streaming source like Kinesis Data Streams for continuous, low-latency processing.

How to eliminate wrong answers

Option B (Amazon S3) is wrong because S3 is an object storage service optimized for batch storage and retrieval, not for real-time streaming ingestion; it lacks the low-latency, continuous data capture and record-level ordering required by Kinesis Data Analytics. Option C (Amazon RDS) is wrong because RDS is a relational database service designed for transactional workloads and structured querying, not for high-velocity streaming data ingestion, and it cannot natively feed data into Kinesis Data Analytics without custom connectors. Option D (Amazon DynamoDB) is wrong because DynamoDB is a NoSQL key-value and document database optimized for low-latency queries on stored data, not for streaming ingestion; while it can be used as a source via DynamoDB Streams, it is not designed for the primary ingestion and temporary storage of raw streaming data for real-time analytics.

Practice this question →

293

MCQmedium

A company is building a data pipeline using AWS Glue to transform data from Amazon RDS to Amazon S3. The pipeline runs daily and processes about 500 GB of data. The team notices that the job is taking longer than expected. Which change would MOST improve the job performance?

A.Disable job bookmarking

B.Increase the number of DPUs for the Glue job

C.Upgrade the RDS instance to a larger class

D.Use smaller file sizes in S3 output

AnswerB

More DPUs provide more parallelism and can speed up the job.

Why this answer

Increasing the number of DPUs (Data Processing Units) for the AWS Glue job directly allocates more distributed computing resources, allowing the job to process the 500 GB dataset in parallel across multiple workers. This is the most effective way to reduce runtime for a large-scale ETL job, as Glue's Spark-based execution scales horizontally with DPU count.

Exam trap

The trap here is that candidates often confuse increasing DPUs with simply adding more memory, when in fact it scales both CPU and memory, and they may incorrectly assume that optimizing output file sizes or disabling bookmarks is a performance fix, whereas those changes address different concerns like cost or incremental processing.

How to eliminate wrong answers

Option A is wrong because disabling job bookmarking would prevent incremental processing, forcing a full reprocess of all data each run, which would increase runtime, not improve it. Option C is wrong because the bottleneck is in the Glue job's processing capacity, not the RDS instance's throughput; upgrading RDS would only help if the read stage were the constraint, but the pipeline is already extracting data to S3. Option D is wrong because using smaller file sizes in S3 output would increase the number of files and metadata operations, degrading performance due to overhead in S3 listing and Spark task scheduling.

Practice this question →

294

MCQmedium

A company runs a daily batch ETL job using AWS Glue that reads from Amazon RDS (MySQL), transforms the data, and writes to Amazon Redshift. The job takes 6 hours and processes 500 GB of data. Management wants to reduce the runtime. Which action would be MOST effective?

A.Increase the node size of the Redshift cluster

B.Use the Redshift COPY command to load data directly from RDS

C.Use Amazon RDS with Provisioned IOPS SSD storage

D.Increase the number of DPUs allocated to the Glue job

AnswerD

More DPUs allow Glue to process data in parallel, reducing overall runtime.

Why this answer

Increasing the number of DPUs (Data Processing Units) allocated to the AWS Glue job directly increases the parallelism of the Spark-based ETL job, allowing it to process the 500 GB of data faster. Since the bottleneck is the Glue job's compute capacity, adding more DPUs reduces runtime without changing the source or target infrastructure.

Exam trap

The trap here is that candidates often assume the bottleneck is the target database (Redshift) or the source database (RDS), but the question explicitly states the Glue job takes 6 hours, so the compute capacity of Glue itself is the primary constraint.

How to eliminate wrong answers

Option A is wrong because increasing Redshift node size improves query performance and data loading speed, but the bottleneck here is the Glue ETL processing, not Redshift ingestion. Option B is wrong because the COPY command loads data from Amazon S3 or other sources, not directly from RDS; it cannot read from a MySQL database, so this is not a valid alternative. Option C is wrong because Provisioned IOPS SSD storage improves RDS I/O performance for transactional workloads, but the Glue job reads data via JDBC, which is network-bound and not limited by RDS storage IOPS for batch reads of 500 GB.

Practice this question →

295

MCQmedium

A data pipeline uses Amazon Kinesis Data Streams to ingest clickstream data. The data is consumed by an AWS Lambda function that transforms and writes to Amazon DynamoDB. The Lambda function is throttled during traffic spikes, causing data to be reprocessed. Which solution should the team implement to handle the throttling without losing data?

A.Use Amazon SQS as an intermediate buffer between Kinesis and Lambda.

B.Increase the number of shards in the Kinesis stream and configure a dead-letter queue (DLQ) for the Lambda function.

C.Enable DynamoDB auto scaling to handle writes.

D.Reduce the batch size in the Lambda event source mapping.

AnswerB

More shards increase parallelism; DLQ captures failures for reprocessing.

Why this answer

Increasing the number of shards in the Kinesis stream raises the throughput capacity, reducing the likelihood of Lambda throttling. Configuring a dead-letter queue (DLQ) for the Lambda function captures any records that fail processing after exhausting retries, preventing data loss. This combination addresses both the throttling cause and provides a safety net for unprocessed records.

Exam trap

The trap here is that candidates often confuse Lambda throttling with DynamoDB write capacity issues, leading them to choose DynamoDB auto scaling (Option C) instead of addressing the upstream Kinesis shard count and Lambda error handling with a DLQ.

How to eliminate wrong answers

Option A is wrong because inserting an SQS buffer between Kinesis and Lambda would break the native Kinesis-to-Lambda integration, add latency, and still require Lambda to handle the same volume of data; it does not address the root cause of throttling. Option C is wrong because DynamoDB auto scaling handles write capacity issues at the database layer, but the problem is Lambda throttling due to Kinesis throughput, not DynamoDB write limits. Option D is wrong because reducing the batch size in the Lambda event source mapping would increase the number of invocations, potentially worsening throttling, and does not prevent data loss from failed processing.

Practice this question →

296

MCQmedium

A company is building a data lake on Amazon S3. Data arrives from multiple sources in different formats (CSV, JSON, Parquet). The engineering team wants to query this data using Amazon Athena with minimal transformation. Which approach minimizes query cost and improves performance?

A.Use Amazon Redshift Spectrum to query the data directly without transformation

B.Use AWS Glue to convert all data to Parquet format, partition by date, and store in a separate S3 bucket

C.Use Amazon EMR to convert data to CSV format and repartition

D.Store data as-is in S3 and create external tables in Athena for each format

AnswerB

This reduces data scanned, improves performance, and lowers cost.

Why this answer

Converting data to Parquet format (a columnar storage format) significantly reduces the amount of data scanned by Athena, which directly lowers query cost (Athena charges per TB scanned). Partitioning by date further limits scanned data by pruning irrelevant partitions. AWS Glue provides a serverless ETL service to perform this conversion efficiently, and storing the output in a separate S3 bucket avoids polluting the raw data lake.

Exam trap

The trap here is that candidates often choose Option D (store as-is) thinking Athena can handle any format efficiently, but they overlook that Athena’s pricing is based on data scanned, and raw CSV/JSON scans are far more expensive than columnar formats like Parquet.

How to eliminate wrong answers

Option A is wrong because Amazon Redshift Spectrum is a service for querying data in S3 from Redshift, not a direct Athena optimization; it still requires data to be in an efficient format and does not address the cost/performance issue of scanning raw CSV/JSON. Option C is wrong because converting to CSV format is not columnar and does not reduce scan size like Parquet; CSV is row-oriented and requires full scans, leading to higher Athena costs. Option D is wrong because storing data as-is in multiple formats forces Athena to scan entire files for each query, resulting in maximum data scanned and highest cost, with no partitioning or compression benefits.

Practice this question →

297

MCQeasy

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data contains personally identifiable information (PII) that must be redacted before storage. Which AWS service can be integrated with Kinesis Data Firehose to transform the data in real time?

A.Amazon Athena

B.Amazon Kinesis Data Analytics

C.Amazon EMR

D.AWS Lambda

AnswerD

Lambda can be invoked by Firehose to transform records in real time.

Why this answer

AWS Lambda can be integrated as a data transformation function within a Kinesis Data Firehose delivery stream. When Firehose receives incoming records, it can invoke a Lambda function synchronously to process each batch of data, allowing you to redact PII fields in real time before the data is written to Amazon S3.

Exam trap

The trap here is that candidates often confuse Kinesis Data Analytics for transformation, but it is designed for real-time analytics and pattern detection, not for inline record-by-record data masking or redaction within a Firehose delivery stream.

How to eliminate wrong answers

Option A is wrong because Amazon Athena is an interactive query service for analyzing data in S3 using standard SQL; it cannot perform real-time transformations on streaming data within a Firehose delivery stream. Option B is wrong because Amazon Kinesis Data Analytics is used for real-time analytics and stream processing using SQL or Apache Flink, but it does not directly integrate as a Firehose transformation step for record-by-record redaction. Option C is wrong because Amazon EMR is a big data processing service that runs Hadoop/Spark clusters; while it can process streaming data, it cannot be used as an inline transformation function within a Kinesis Data Firehose delivery stream.

Practice this question →

298

MCQhard

A data engineer is configuring an IAM policy to allow users to upload objects to an S3 bucket only if the objects are encrypted using SSE-S3. However, users are getting AccessDenied errors when uploading objects without specifying encryption. What is the most likely cause?

A.The condition should check for aws:SourceIp instead of encryption

B.The condition requires encryption to be specified, but the upload does not specify it

C.The policy is attached to the wrong IAM user

D.The bucket policy denies all PutObject without encryption

AnswerB

The condition requires s3:x-amz-server-side-encryption to be AES256, so without it, access is denied.

Why this answer

The policy allows PutObject only when encryption is AES256, but denies when no encryption is specified because the condition is not met. Option A is wrong because it's not a service control policy; Option C is wrong because the bucket policy is not shown; Option D is wrong because the condition checks for AES256, not KMS.

Practice this question →

299

Multi-Selecthard

A company uses AWS Glue Data Catalog to manage metadata for its data lake on Amazon S3. The data lake contains terabytes of data in CSV format. The data engineering team wants to improve query performance in Amazon Athena and reduce costs. Which actions should the team take? (Select THREE.)

Select 3 answers

A.Create views in Athena to simplify queries.

B.Compress the data using Snappy or GZIP.

C.Partition the data by commonly filtered columns.

D.Convert the data to Parquet format.

E.Convert the data to JSON format.

AnswersB, C, D

Compression reduces storage and data scanned.

Why this answer

Compressing CSV data with Snappy or GZIP reduces the amount of data scanned by Athena, directly lowering query costs (Athena charges per TB scanned). Snappy offers faster decompression for better query performance, while GZIP provides higher compression ratios. Both formats are natively supported by Athena and reduce I/O from S3.

Exam trap

The trap here is that candidates may think simplifying queries (views) or switching to another text format (JSON) improves performance, but only compression, partitioning, and columnar formats reduce the amount of data scanned, which is the key to Athena cost and speed optimization.

Practice this question →

300

Multi-Selecteasy

A company wants to analyze streaming data from IoT devices in near-real-time. They need to store raw data in Amazon S3 and also run SQL queries on the streaming data. Which TWO services should they use?

Select 2 answers

A.AWS Glue

B.Amazon Kinesis Data Streams

C.Amazon Kinesis Data Analytics

D.Amazon Kinesis Data Firehose

E.AWS Lambda

AnswersC, D

Runs SQL on streaming data.

Why this answer

Amazon Kinesis Data Analytics is correct because it enables running SQL queries on streaming data in near-real-time, allowing the company to analyze IoT data as it arrives without needing to store it first. It integrates directly with Kinesis Data Streams or Firehose to process data streams using standard SQL, making it ideal for real-time analytics on streaming data.

Exam trap

The trap here is that candidates often confuse Kinesis Data Streams (which only ingests and stores data) with Kinesis Data Analytics (which provides SQL querying), or they mistakenly think AWS Glue can handle real-time streaming SQL when it is actually designed for batch processing.

Practice this question →