CCNA Ml Data Engineering Questions — Page 3 of 5

151

MCQmedium

An e-commerce company uses Amazon Kinesis Data Firehose to deliver clickstream data to an Amazon S3 bucket. The data is then queried using Amazon Athena. The marketing team wants to run daily reports that aggregate click events by product ID. However, the reports are slow because Athena scans the entire dataset each time. The data is partitioned by date (e.g., s3://bucket/clickstream/2023/01/01/). The product ID is a column within the data. The data engineering team wants to improve query performance without moving the data to another service. Which approach should the team take?

A.Convert the data from JSON to Parquet format

B.Use Amazon Redshift Spectrum to query the data

C.Create a view in Athena that filters by product ID

D.Repartition the data by product ID in addition to date

AnswerD

Partitioning by product ID allows Athena to skip irrelevant partitions.

Why this answer

Repartition the data by product ID in addition to date. This adds a partition level for product ID, so queries that filter on product ID will only scan the relevant partitions. Option A (convert to Parquet) reduces data scanned due to columnar storage and compression, but without partition pruning on product ID, Athena would still scan all partitions for each query.

Option B (Redshift Spectrum) would still require scanning data, and involves additional service complexity. Option C (create a view) does not change physical storage; it only provides a logical filter, but Athena still scans all underlying data. Therefore, repartitioning by product ID provides the most direct improvement for queries filtering by product ID.

Practice this question →

152

MCQhard

A company uses Amazon Redshift for its data warehouse. The data engineering team needs to load 10 TB of data from Amazon S3 into Redshift every night. The team wants to minimize the load time and use the fewest number of COPY commands. The data is in CSV format and is partitioned by date in S3. Which approach should the team take?

A.Use a manifest file with a single COPY command.

B.Use multiple COPY commands, one per partition.

C.Concatenate all data into a single large file before loading.

D.Use AWS Glue to transform the data and then load into Redshift.

AnswerA

A manifest file allows Redshift to load from multiple files in parallel efficiently.

Why this answer

Using a manifest file with a single COPY command is the most efficient approach because it allows Redshift to load data from multiple S3 objects (partitioned by date) in parallel, automatically splitting the workload across cluster nodes. This minimizes load time by leveraging Redshift's parallel processing without requiring multiple COPY commands or manual concatenation, and it avoids the overhead of additional services like AWS Glue for a straightforward bulk load.

Exam trap

The trap here is that candidates assume multiple COPY commands (one per partition) are needed for partitioned data, but Redshift's manifest file allows a single COPY command to load from many S3 objects in parallel, which is faster and simpler.

How to eliminate wrong answers

Option B is wrong because using multiple COPY commands (one per partition) introduces sequential overhead and requires managing multiple statements, which increases load time and complexity compared to a single manifest-based COPY that handles parallelism natively. Option C is wrong because concatenating all data into a single large file eliminates parallelism, forcing Redshift to process the file sequentially on a single slice, which dramatically increases load time for 10 TB of data. Option D is wrong because AWS Glue adds unnecessary transformation overhead and cost for a simple CSV load; Redshift's COPY command can directly load CSV from S3 without an intermediate ETL service, and Glue does not reduce the number of COPY commands or improve load time for this use case.

Practice this question →

153

MCQmedium

A team is using Amazon SageMaker to train a model on a dataset that is 500 GB in size, stored as CSV files in S3. The training job takes 2 hours using a single ml.p3.2xlarge instance. The team wants to reduce training time to under 30 minutes. The model architecture supports distributed training. Which solution will achieve this goal with the LEAST amount of code changes?

A.Use managed spot training to reduce cost and then use cost savings to train with a larger instance.

B.Use a single ml.p3.16xlarge instance with more GPUs and memory.

C.Use multiple ml.p3.2xlarge instances with SageMaker's distributed data parallelism library, enabling automatic sharding of the training data.

D.Change the input mode to Pipe mode to stream data from S3 directly, reducing I/O wait time.

AnswerC

Distributed training across multiple instances reduces time proportionally; minimal code changes with SageMaker's SDK.

Why this answer

SageMaker's distributed data parallelism library automatically shards the training data across multiple ml.p3.2xlarge instances, enabling parallel gradient computation and reducing wall-clock training time from 2 hours to under 30 minutes without requiring manual code changes to the training script. The model architecture already supports distributed training, so the library handles the communication and synchronization (e.g., AllReduce) transparently.

Exam trap

The trap here is that candidates often confuse 'larger instance' (Option B) with 'distributed training' (Option C), failing to realize that a single large instance cannot parallelize data loading and gradient computation across multiple nodes, while distributed data parallelism with multiple smaller instances can achieve the required speedup with minimal code changes.

How to eliminate wrong answers

Option A is wrong because managed spot training reduces cost but does not inherently reduce training time; using a larger instance with spot training still requires code changes for distributed training and may not achieve the sub-30-minute goal. Option B is wrong because a single ml.p3.16xlarge instance, while having more GPUs and memory, still processes data sequentially on one node and cannot scale training time linearly to under 30 minutes for a 500 GB dataset without distributed data parallelism across multiple instances. Option D is wrong because Pipe mode streams data directly from S3 to reduce I/O wait time, but it does not parallelize computation across multiple GPUs or instances, so the training time remains bound by the single-instance compute capacity.

Practice this question →

154

MCQeasy

A company uses Amazon S3 to store log files from various applications. The logs are in JSON format and are appended to existing files every few minutes. A data analyst wants to run SQL queries on the logs using Amazon Athena. However, queries return incomplete results because Athena does not support modifying data. The team needs to enable querying of the latest log data with minimal changes to the existing ingestion process. Which solution should the team implement?

A.Convert the logs to Parquet format using a scheduled AWS Glue job and store them in a separate S3 bucket.

B.Stream the logs to Amazon Kinesis Data Firehose, which writes the data to S3 in Parquet format.

C.Create an Athena table using the Hive JSON SerDe that reads the logs directly from the existing S3 bucket.

D.Use AWS Glue to load the JSON logs into Amazon Redshift and query using Redshift.

AnswerC

Athena can query JSON logs with the correct SerDe without changing the ingestion.

Why this answer

Athena supports reading JSON data with the Hive JSON SerDe. By creating a table with the appropriate SerDe, the analyst can query the JSON logs directly without modifying the ingestion process. Option A is incorrect because converting to Parquet would require changing the ingestion process.

Option B is incorrect because using Kinesis Data Firehose would require altering the ingestion pipeline. Option D is incorrect because loading into Redshift adds complexity and latency.

Practice this question →

155

MCQhard

A company uses AWS Glue ETL jobs to process data from an Amazon RDS for MySQL database into Amazon S3. The job runs daily and takes 6 hours to complete. The team wants to reduce runtime and cost. The source table has 50 million rows and is updated continuously. Which combination of changes would be MOST effective?

A.Use a single worker with a larger instance type.

B.Increase the number of DPUs and enable job bookmarking.

C.Use JDBC connections with pushdown predicates and increase the number of DPUs.

D.Change the job trigger from time-based to event-based.

AnswerC

Pushdown predicates filter data at source, reducing data transfer; more DPUs parallelize the work.

Why this answer

Using JDBC pushdown predicates filters data at the source database, reducing the volume of data transferred over the network and processed by Glue. Increasing the number of DPUs (data processing units) adds parallelism, which directly reduces runtime. Together, these changes minimize both execution time and cost by optimizing data movement and compute resources.

Exam trap

The trap here is that candidates assume simply adding more compute (DPUs) or using job bookmarking will solve performance issues, without realizing that the primary bottleneck is data transfer from the source database, which requires predicate pushdown to reduce the data volume.

How to eliminate wrong answers

Option A is wrong because using a single worker with a larger instance type does not address the bottleneck of reading 50 million rows from RDS; Glue's single-worker architecture cannot parallelize the JDBC read, so runtime remains high and cost may increase due to a more expensive instance. Option B is wrong because increasing DPUs without pushdown predicates still forces Glue to pull all 50 million rows over the network, and job bookmarking only helps with incremental processing on subsequent runs, not the initial full load or the current daily full scan. Option D is wrong because changing the trigger from time-based to event-based does not affect the runtime or cost of the job itself; it only changes when the job starts, not how efficiently it processes data.

Practice this question →

156

Drag & Dropmedium

Drag and drop the steps to set up Amazon SageMaker Ground Truth for a labeling job in the correct order.

Drag steps to the numbered slots on the right, or tap a step then tap a slot.

Steps

Order

1Step 1

2Step 2

3Step 3

4Step 4

Why this order

Ground Truth setup involves dataset preparation, job creation, task configuration, instructions, and execution.

Practice this question →

157

Multi-Selectmedium

A data engineer is designing a streaming pipeline using Amazon Kinesis Data Analytics for Apache Flink. The pipeline reads from a Kinesis data stream and writes to a S3 bucket. The job must recover quickly from failures without reprocessing large amounts of data. Which TWO configurations should be used? (Choose TWO)

Select 2 answers

A.Enable checkpointing with a state backend like RocksDB.

B.Use in-memory state backend for low latency.

C.Configure the S3 sink to use exactly-once delivery semantics.

D.Set the parallelism to the maximum number of shards.

E.Increase the retention period of the Kinesis stream to 365 days.

AnswersA, C

Checkpointing enables state recovery after failure.

Why this answer

Enabling checkpointing with a state backend like RocksDB allows Apache Flink to periodically save the state of the streaming application to durable storage. In the event of a failure, Flink can restart from the last completed checkpoint, avoiding the need to reprocess large amounts of data from the beginning of the stream. RocksDB is specifically designed for large state and provides fast recovery by storing state on disk with memory caching, making it ideal for production streaming pipelines.

Exam trap

The trap here is that candidates often confuse parallelism or stream retention settings with fault-tolerance mechanisms, mistakenly believing that increasing parallelism or retention alone can prevent data reprocessing, when in fact only checkpointing with a durable state backend ensures fast recovery.

Practice this question →

158

MCQmedium

A data engineer needs to transform large CSV files stored in Amazon S3 into Parquet format before loading into Amazon Redshift. The transformation logic is complex and requires custom Python code. Which AWS service should be used to perform this transformation with minimal operational overhead?

A.AWS Glue

B.AWS Lambda

C.Amazon EMR

D.AWS Data Pipeline

AnswerA

Glue is a serverless ETL service that can run complex transformations on data in S3 and write to Parquet.

Why this answer

AWS Glue is the correct answer because it is a fully managed, serverless ETL service that can handle large CSV files, convert them to Parquet, and load into Amazon Redshift with minimal operational overhead. AWS Glue provides a built-in Spark environment and supports custom Python code via Spark jobs. Option B (AWS Lambda) has a 15-minute timeout and is not designed for large-scale data transformations.

Option C (Amazon EMR) requires managing clusters, increasing operational overhead. Option D (AWS Data Pipeline) is a legacy service with less flexibility and is not optimized for complex transformations like CSV to Parquet.

Practice this question →

159

Multi-Selecteasy

A company wants to build a data lake on Amazon S3. The data lake should support both batch and real-time data ingestion. Which AWS services should be used for data ingestion? (Choose TWO.)

Select 2 answers

A.AWS Glue

B.Amazon Kinesis Data Firehose

C.Amazon Redshift

D.Amazon Athena

E.Amazon SQS

AnswersA, B

Glue performs batch ETL and can ingest data into S3.

Why this answer

AWS Glue is correct because it provides a managed ETL service that can handle batch data ingestion into a data lake on Amazon S3. It can be scheduled for periodic batch loads or triggered by events, making it suitable for batch ingestion workflows. Amazon Kinesis Data Firehose is correct because it is a fully managed service for loading streaming data into S3 in near real-time, supporting real-time ingestion with automatic buffering and compression.

Exam trap

The trap here is that candidates often confuse data ingestion services with data query or storage services, mistakenly selecting Amazon Redshift or Athena because they interact with data in S3, but they do not perform the ingestion itself.

Practice this question →

160

MCQhard

A data engineer is designing a data pipeline that transforms raw JSON files (each 50-200 KB) in Amazon S3 into Parquet format using AWS Glue. The pipeline must minimize data processing costs and handle a high volume of small files (millions per day). The engineer configures a Glue ETL job with Spark, but the job is slow and expensive due to overhead of reading many small files. Which optimization should the engineer implement to reduce cost and improve performance?

A.Increase the worker type to G.2X for more memory per worker.

B.Increase the number of DPUs allocated to the Glue job.

C.Change the output format from Parquet to CSV to reduce compression overhead.

D.Use S3 object grouping or batch operations to combine small files before Glue processing.

AnswerD

Combining small files reduces task overhead, leading to faster and cheaper jobs.

Why this answer

The primary performance bottleneck with many small files in S3 is the overhead of listing, opening, and reading each file individually in Spark. By grouping or batching small files into larger objects (e.g., using S3 Batch Operations or a pre-processing step), you reduce the number of input splits and task launches, which dramatically lowers the cost and runtime of the Glue ETL job. This directly addresses the root cause of the inefficiency rather than merely scaling resources.

Exam trap

The trap here is that candidates often assume scaling up resources (more memory or DPUs) will fix performance issues, but the real problem is the small-file overhead, which is a data layout issue that cannot be solved by adding compute power.

How to eliminate wrong answers

Option A is wrong because increasing the worker type to G.2X provides more memory per worker but does not reduce the overhead of reading millions of small files; the bottleneck is the number of files, not memory capacity. Option B is wrong because increasing the number of DPUs adds more parallel workers, which can actually worsen performance by increasing the overhead of scheduling and managing tasks for many small files, and it raises costs without solving the file-size issue. Option C is wrong because changing the output format from Parquet to CSV would increase storage size and I/O, and CSV lacks compression and predicate pushdown benefits, making the job slower and more expensive, not less.

Practice this question →

161

MCQeasy

A data scientist needs to run a one-time SQL query on a large dataset in Amazon S3. The dataset is stored in Parquet format and is about 500 GB. The query requires complex aggregations and joins. Which AWS service should be used to minimize cost and setup time?

A.Amazon Redshift

B.Amazon Athena

C.Amazon RDS for MySQL

D.Amazon EMR with Spark SQL

AnswerB

Serverless, pay-per-query, no setup required.

Why this answer

Amazon Athena is the correct choice because it is a serverless query service that allows you to run SQL directly on data stored in S3 without provisioning any infrastructure. For a one-time query on 500 GB of Parquet data, Athena minimizes cost (pay-per-query, no idle cluster costs) and setup time (no cluster creation or data loading). Its ability to handle complex aggregations and joins on columnar formats like Parquet makes it ideal for this ad-hoc use case.

Exam trap

The trap here is that candidates often choose Amazon EMR with Spark SQL (Option D) because they associate Spark with complex joins and large datasets, but they overlook the fact that for a one-time query, the setup time and cost of provisioning a cluster make Athena a more efficient and cost-effective choice.

How to eliminate wrong answers

Option A is wrong because Amazon Redshift requires provisioning a cluster, loading data into it, and paying for compute even when idle, which is overkill and costly for a one-time query. Option C is wrong because Amazon RDS for MySQL is a transactional database not designed for analytical queries on large datasets in S3; it would require importing 500 GB of data and lacks native Parquet support. Option D is wrong because Amazon EMR with Spark SQL involves provisioning a cluster, managing Spark configurations, and incurring costs for cluster uptime, adding unnecessary setup time and expense for a single query.

Practice this question →

162

MCQhard

A data engineer is designing a data pipeline that ingests 500 GB of data daily from an on-premises Oracle database to Amazon S3. The pipeline must minimize data loss and support change data capture (CDC). Which combination of services should they use?

A.AWS Database Migration Service (DMS) with ongoing replication

B.AWS Data Pipeline with SQL query

C.Amazon Kinesis Data Streams with a custom Oracle CDC connector

D.AWS Glue ETL jobs running on a schedule

AnswerA

DMS supports CDC and can write to S3.

Why this answer

AWS DMS with ongoing replication is the correct choice because it provides continuous change data capture (CDC) from an Oracle source database using Oracle LogMiner or binary reader technology, replicating transactions in near real-time to Amazon S3. This minimizes data loss by capturing incremental changes without requiring batch snapshots, and it supports the 500 GB daily volume efficiently with parallel tasks and task tuning.

Exam trap

The trap here is that candidates may confuse Amazon Kinesis Data Streams with a custom CDC connector as a viable alternative, but AWS does not provide a managed Oracle CDC connector for Kinesis, making DMS the only fully managed, production-ready service for this use case.

How to eliminate wrong answers

Option B is wrong because AWS Data Pipeline with SQL query only supports scheduled batch extracts, not real-time CDC, and cannot capture ongoing changes without manual intervention, leading to potential data loss between runs. Option C is wrong because Amazon Kinesis Data Streams does not natively support a custom Oracle CDC connector; building and maintaining such a connector is complex, unreliable, and not a managed service, unlike DMS which provides built-in Oracle CDC. Option D is wrong because AWS Glue ETL jobs running on a schedule are batch-oriented and lack native CDC capabilities; they would require full table scans or custom logic to detect changes, which is inefficient for 500 GB daily and risks data loss between job runs.

Practice this question →

163

MCQeasy

A company needs to move 10 TB of data from an on-premises NAS to Amazon S3 over a 100 Mbps internet connection. The transfer must complete within 3 days. Which solution is the most appropriate?

A.Use AWS DataSync to transfer over the internet

B.Enable S3 Transfer Acceleration on the bucket

C.Use AWS CLI to copy data directly over the internet

D.Use AWS Snowball Edge to transfer the data

AnswerD

Snowball Edge provides physical transport, faster than internet for large data.

Why this answer

AWS Snowball Edge is a physical device that can transfer large data volumes much faster than over the internet. Option A is wrong: AWS DataSync over the internet at 100 Mbps would take approximately 10 days to transfer 10 TB, exceeding the 3-day requirement. Option B is wrong: S3 Transfer Acceleration optimizes the network path but still relies on internet bandwidth; even with a 200% speed improvement, it would take over 4.6 days.

Option C is wrong: using AWS CLI to copy directly over the internet has the same bandwidth limitation and would also take about 10 days.

Practice this question →

164

Multi-Selecthard

A company is designing a data pipeline that ingests streaming data from social media feeds. The data must be processed in real-time to detect trending topics, and results must be stored in Amazon DynamoDB for low-latency access. Which services should the company use? (Choose TWO.)

Select 2 answers

A.Amazon Kinesis Data Firehose

B.AWS Lambda

C.Amazon Simple Queue Service (SQS)

D.Amazon Kinesis Data Analytics

E.Amazon Kinesis Data Streams

AnswersD, E

Provides real-time analytics to detect trending topics.

Why this answer

Amazon Kinesis Data Analytics (D) is correct because it provides real-time SQL-based processing of streaming data, enabling the detection of trending topics from social media feeds without requiring custom code. It directly analyzes data from Kinesis Data Streams and can output results to DynamoDB via a Lambda function or Firehose, meeting the low-latency storage requirement.

Exam trap

The trap here is that candidates often confuse Kinesis Data Firehose (a delivery service) with Kinesis Data Analytics (a real-time processing service), or assume Lambda alone can handle streaming analytics, when in fact Kinesis Data Analytics is the only option that provides built-in SQL-based stream processing for real-time trend detection.

Practice this question →

165

MCQmedium

A company is streaming real-time sensor data from IoT devices to Amazon Kinesis Data Streams. The data is then consumed by an AWS Lambda function that enriches the records with metadata from an Amazon DynamoDB table and writes the results to an Amazon S3 bucket. Recently, the Lambda function has been failing with 'ProvisionedThroughputExceededException' errors from DynamoDB. The data volume is variable, with occasional bursts. Which solution should a data engineer implement to resolve this issue without losing data?

A.Increase the DynamoDB table's provisioned read capacity units to a high static value.

B.Use an Amazon SQS queue to buffer the Lambda requests before querying DynamoDB.

C.Enable DynamoDB auto scaling for the table to automatically adjust read capacity based on demand.

D.Configure an Amazon SNS topic to throttle the data stream before it reaches Lambda.

AnswerC

Auto scaling adjusts capacity dynamically to handle bursts without manual intervention.

Why this answer

DynamoDB auto scaling dynamically adjusts the table's provisioned read capacity based on actual traffic patterns, handling bursty sensor data without manual intervention. This prevents ProvisionedThroughputExceededExceptions while ensuring no data loss, as the Lambda function can retry failed operations. Auto scaling is the most cost-effective and operationally efficient solution for variable workloads.

Exam trap

The trap here is that candidates confuse buffering the Lambda invocation (Option B) with addressing the DynamoDB throttling error, but the error occurs inside the Lambda function after invocation, so an SQS queue does not solve the read capacity issue.

How to eliminate wrong answers

Option A is wrong because setting a high static read capacity is wasteful and costly, and it does not adapt to the variable bursty nature of the data, leading to either over-provisioning or continued throttling during unexpected spikes. Option B is wrong because an SQS queue buffers Lambda invocation requests, but the error occurs during DynamoDB queries within the Lambda function, not at the invocation layer; SQS does not address the read capacity limit on the DynamoDB table. Option D is wrong because an SNS topic is a pub/sub messaging service that does not throttle data streams; it would add latency and complexity without solving the DynamoDB throughput issue, and it could cause data loss if the topic is not configured for retries.

Practice this question →

166

Multi-Selectmedium

A data engineer is designing a data pipeline that uses Amazon S3 events to trigger an AWS Lambda function for processing. The pipeline must handle high throughput with low latency. Which TWO configurations should be applied?

Select 2 answers

A.Configure Lambda with reserved concurrency

B.Use an SQS queue between S3 and Lambda

C.Place Lambda in a VPC to reduce network latency

D.Use Amazon Kinesis Data Streams as an intermediary

E.Enable S3 Event Notifications to invoke Lambda directly

AnswersA, E

Ensures Lambda has enough capacity to handle bursts.

Why this answer

Reserved concurrency ensures that the Lambda function always has a guaranteed number of concurrent executions available, preventing it from being throttled by other functions in the same AWS account. This is critical for high-throughput, low-latency pipelines because S3 event notifications can burst many invocations simultaneously, and without reserved concurrency, the function might hit the account-level concurrency limit and drop events.

Exam trap

The trap here is that candidates often confuse 'reducing latency' with 'using a VPC' or 'adding a queue,' but for S3-triggered Lambda, direct invocation with reserved concurrency is the simplest and lowest-latency path, while VPCs and queues add overhead.

Practice this question →

167

MCQhard

Refer to the exhibit. A CloudFormation template creates an S3 bucket. The data engineering team stores daily log files in this bucket and queries them using Amazon Athena. After 30 days, queries on logs older than 30 days start failing with 'Access Denied' errors. What is the MOST likely reason?

A.The lifecycle rule transitions objects to GLACIER after 30 days, making them inaccessible to Athena.

B.The bucket uses default encryption with SSE-S3, which Athena does not support.

C.The lifecycle rule deletes objects after 30 days.

D.The bucket policy denies access to objects older than 30 days.

AnswerA

Athena cannot query GLACIER objects; they must be restored first.

Why this answer

Amazon Athena reads data directly from S3 and does not support querying objects stored in the GLACIER storage class because GLACIER objects are not retrievable in real time. The lifecycle rule transitions objects to GLACIER after 30 days, so when Athena attempts to read those older objects, it receives 'Access Denied' errors because the objects are no longer in a queryable storage class.

Exam trap

The trap here is that candidates often confuse 'Access Denied' errors with permission issues (bucket policies or IAM) rather than recognizing that the error is caused by the storage class transition to GLACIER, which makes objects unreadable by Athena without restoration.

How to eliminate wrong answers

Option B is wrong because Athena fully supports SSE-S3 (default encryption with Amazon S3-managed keys) and can query objects encrypted with SSE-S3 without any issues. Option C is wrong because if objects were deleted after 30 days, Athena queries would return 'No data' or 'Zero records' rather than 'Access Denied' errors. Option D is wrong because a bucket policy denying access to objects older than 30 days would produce consistent 'Access Denied' errors for all operations on those objects, but the scenario describes queries failing only after 30 days, which aligns with a lifecycle transition to GLACIER, not a policy change.

Practice this question →

168

MCQhard

A data engineer needs to build a pipeline that ingests CSV files from an S3 bucket, validates the schema, and loads the data into an Amazon Redshift cluster. The pipeline must handle schema evolution gracefully by adding new columns as they appear in the source files. Which combination of AWS services and configurations would meet these requirements with minimal operational overhead?

A.Use AWS Glue to create a crawler that updates the schema, then use Redshift Spectrum to query the data directly from S3

B.Use Amazon Kinesis Data Firehose to ingest the files and load into Redshift, with a Lambda function to detect schema changes

C.Use Amazon Athena to create external tables with schema-on-read, and insert results into Redshift using INSERT INTO

D.Use AWS Glue to create a crawler and an ETL job that writes to Redshift, with 'resolveChoice' to handle new columns

AnswerD

Glue handles schema evolution via DynamicFrame and resolveChoice, and loads into Redshift.

Why this answer

AWS Glue provides a fully managed ETL service that can automatically detect schema changes via crawlers and handle new columns in CSV files using the 'resolveChoice' transformation. The Glue ETL job can write directly to Amazon Redshift with minimal operational overhead, as it manages schema evolution without requiring manual intervention or additional infrastructure.

Exam trap

The trap here is that candidates often assume Redshift Spectrum or Athena can load data into Redshift, but they are query engines, not data loading services, and do not handle schema evolution for batch ingestion into a Redshift cluster.

How to eliminate wrong answers

Option A is wrong because Redshift Spectrum queries data directly from S3 without loading it into Redshift, which does not meet the requirement to load data into the Redshift cluster. Option B is wrong because Kinesis Data Firehose is designed for streaming data, not batch CSV file ingestion from S3, and using a Lambda function to detect schema changes adds operational overhead and complexity. Option C is wrong because Athena uses schema-on-read for external tables, but inserting results into Redshift with INSERT INTO is inefficient for large datasets and does not handle schema evolution automatically or gracefully.

Practice this question →

169

MCQeasy

A data engineer needs to process streaming data from an IoT fleet and store the results in Amazon S3 for analysis. The solution must be serverless and handle data that arrives at irregular intervals. Which AWS service should be used to ingest the data?

A.Amazon S3

B.AWS IoT Core

C.Amazon Simple Queue Service (SQS)

D.Amazon Kinesis Data Streams

AnswerB

AWS IoT Core provides secure device connectivity, message routing, and integrates with serverless processing.

Why this answer

AWS IoT Core is the correct choice because it is a fully managed, serverless service designed specifically to ingest data from IoT devices at scale, handling irregular and high-frequency message arrivals via MQTT, HTTP, or LoRaWAN protocols. It can directly route data to Amazon S3 using IoT Rules, making it ideal for this streaming IoT fleet scenario without requiring any server management.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Streams as the default for streaming data, but for IoT-specific ingestion with irregular intervals and native MQTT support, AWS IoT Core is the correct serverless choice.

How to eliminate wrong answers

Option A is wrong because Amazon S3 is an object storage service, not a data ingestion service; it cannot natively receive streaming data from IoT devices without an intermediary like IoT Core or Kinesis. Option C is wrong because Amazon SQS is a message queue service that decouples application components but lacks native IoT protocol support (e.g., MQTT) and does not provide built-in rules for direct S3 storage of streaming IoT data. Option D is wrong because Amazon Kinesis Data Streams is a real-time data streaming service but is not serverless in the same sense (requires provisioning shards) and lacks native IoT protocol endpoints, making it less suitable for direct ingestion from an IoT fleet compared to IoT Core.

Practice this question →

170

MCQhard

Refer to the exhibit. An ML engineer applies this bucket policy to an S3 bucket. The SageMaker execution role MySageMakerRole is used to train a model. The training data is located in s3://my-bucket/data/. The SageMaker training job fails with an access error. What is the most likely cause?

A.The policy allows GetObject only from the data/ prefix, but the training job uses a different prefix.

B.The role is not in the same AWS account as the bucket.

C.The Deny statement on s3:ListBucket prevents the role from listing objects in the bucket.

D.The bucket has default encryption enabled, causing a conflict.

AnswerC

SageMaker may need to list objects to iterate over files; the explicit deny blocks this.

Why this answer

The Deny statement on s3:ListBucket explicitly denies the s3:ListBucket action for the MySageMakerRole. SageMaker training jobs require the ability to list objects in the bucket to discover and read training data, even if the GetObject permission is granted. The explicit Deny overrides any Allow, causing the access error.

Exam trap

The trap here is that candidates assume GetObject alone is sufficient for reading data, but SageMaker training jobs also require ListBucket to enumerate objects in the prefix, and an explicit Deny on ListBucket overrides any Allow.

How to eliminate wrong answers

Option A is wrong because the policy allows GetObject from the data/ prefix, and the training data is located at s3://my-bucket/data/, so there is no prefix mismatch. Option B is wrong because the bucket policy does not include any condition restricting access based on AWS account, and SageMaker roles can be used cross-account if properly configured; the error is not due to account mismatch. Option D is wrong because default encryption on an S3 bucket does not cause access errors for SageMaker training jobs; SageMaker can read encrypted objects as long as the role has the necessary KMS permissions, which are not mentioned as missing.

Practice this question →

171

MCQmedium

A company is using Amazon SageMaker to train machine learning models. The training data is stored in Amazon S3, but the data includes personally identifiable information (PII) that must be anonymized before training. What is the most efficient way to anonymize the data?

A.Use an AWS Glue ETL job to read from S3, apply anonymization, and write to another S3 bucket.

B.Use Amazon Athena to query the data and apply anonymization functions.

C.Use Amazon Redshift Spectrum to query and anonymize data in S3.

D.Use a SageMaker Processing job to read from S3 and apply anonymization.

AnswerA

Glue is a serverless ETL service that can efficiently transform large datasets.

Why this answer

AWS Glue ETL jobs are purpose-built for serverless data transformation at scale, making them the most efficient choice for anonymizing PII in S3 before training. Glue can read directly from S3, apply built-in or custom anonymization transforms (e.g., masking, hashing) using PySpark or Scala, and write the cleaned data to a separate S3 bucket without provisioning any infrastructure. This approach decouples the data preparation from SageMaker, avoids unnecessary compute costs during training, and scales automatically with data volume.

Exam trap

The trap here is that candidates often choose SageMaker Processing (Option D) because it is a SageMaker-native service, but the question asks for the 'most efficient' approach for standalone data anonymization, and AWS Glue is the correct serverless ETL service for this task, not a processing job tied to the training pipeline.

How to eliminate wrong answers

Option B is wrong because Amazon Athena is an interactive query service for ad-hoc SQL analysis, not a data transformation engine; it lacks built-in support for complex anonymization logic (e.g., regex-based masking, tokenization) and would require inefficient row-by-row processing with UDFs, making it unsuitable for large-scale ETL. Option C is wrong because Amazon Redshift Spectrum is designed for querying external data in S3 from Redshift, not for performing ETL transformations; it would require moving data through Redshift clusters, adding latency and cost, and does not natively support anonymization functions. Option D is wrong because a SageMaker Processing job is intended for data processing within the ML workflow (e.g., feature engineering, validation) but is less efficient for standalone anonymization as it requires spinning up SageMaker instances and managing lifecycle, whereas Glue is serverless and optimized for pure ETL tasks.

Practice this question →

172

Multi-Selectmedium

A company is designing a data pipeline to analyze customer behavior. The pipeline must handle real-time streaming data and batch data. The data must be stored in a data lake on Amazon S3 and also made available for interactive queries. Which THREE services should be combined to build this pipeline? (Choose THREE.)

Select 3 answers

A.Amazon Kinesis Data Streams

B.AWS Glue

C.Amazon Redshift

D.Amazon DynamoDB Streams

E.Amazon Athena

AnswersA, B, E

Real-time data ingestion.

Why this answer

Amazon Kinesis Data Streams is correct because it is the primary AWS service for ingesting and processing real-time streaming data at scale. It can capture and store streaming data from sources like clickstreams or IoT devices, making it available for downstream consumers such as AWS Glue or Amazon Athena for analysis.

Exam trap

The trap here is that candidates often confuse Amazon Redshift as a query engine for S3 data, but Redshift requires data to be loaded into its cluster, whereas Athena queries data in place, making Athena the correct choice for interactive queries on the data lake.

Practice this question →

173

Multi-Selecteasy

Which TWO AWS services can be used to transform data in a streaming fashion without using a persistent cluster? (Choose 2.)

Select 2 answers

A.AWS Glue

B.Amazon EMR

C.AWS Lambda

D.Amazon Kinesis Data Analytics

E.AWS Data Pipeline

AnswersC, D

Lambda can process streaming data from Kinesis or DynamoDB Streams serverlessly.

Why this answer

(Lambda) and Option D (Kinesis Data Analytics) are serverless streaming transformation services. Option A (Glue) is serverless but not low-latency streaming. Option B (EMR) requires a persistent cluster.

Option E (Data Pipeline) is for batch.

Practice this question →

174

MCQhard

A financial services company needs to build a data lake on Amazon S3 that meets regulatory requirements for data retention and encryption. Data must be encrypted at rest and in transit, and access must be audited. The data lake will be queried by Amazon Athena and Amazon Redshift Spectrum. Which combination of actions should be taken?

A.Enable S3 default encryption with SSE-KMS and enable AWS CloudTrail for S3 data events.

B.Use IAM policies to control access and enable S3 server access logging.

C.Use SSL/TLS for all connections and enable S3 versioning.

D.Enable S3 default encryption with SSE-S3 and use S3 access logs.

AnswerA

SSE-KMS provides encryption with managed keys; CloudTrail logs data events for auditing.

Why this answer

S3 default encryption with SSE-KMS provides encryption at rest with customer-managed keys, and enabling AWS CloudTrail for S3 data events provides comprehensive auditing of access to the data lake. This combination meets the regulatory requirements for data retention, encryption, and audit. Option B is incorrect because SSL/TLS only ensures encryption in transit, and versioning does not provide encryption at rest or auditing.

Option C is incorrect because IAM policies control access but do not provide encryption. Option D is incorrect because SSE-S3 does not allow key management control, which may be required, and S3 access logs are less detailed than CloudTrail for auditing.

Practice this question →

175

MCQeasy

A company is using Amazon Kinesis Data Firehose to load streaming data into an S3 bucket. The data schema evolves over time, with new columns added. The data must be queryable using Amazon Athena. What is the BEST way to handle schema changes?

A.Manually update the Athena table definition each time a new column is added

B.Configure Firehose to convert the data to Apache JSON format

C.Use AWS Glue Crawlers to automatically detect schema changes and update the table metadata

D.Recreate the Athena table daily to pick up new columns

AnswerC

Glue Crawlers can run on a schedule to discover new columns and update the Data Catalog.

Why this answer

AWS Glue Crawlers can automatically detect schema changes in the data stored in S3 and update the AWS Glue Data Catalog metadata used by Athena. This allows Athena to query the evolving schema without manual intervention. Option A (manual update) is not the best because it requires manual effort and is error-prone.

Option B (converting to JSON) is not necessary; Athena can handle various formats including Parquet, ORC, etc., and schema evolution is better handled by Glue Crawlers. Option D (recreating the table daily) is disruptive and not the best practice.

Practice this question →

176

MCQeasy

A data engineer needs to run a one-time ETL job to transform 500 GB of data from Amazon RDS to Amazon S3. The job should be cost-effective and require minimal infrastructure management. Which AWS service should be used?

A.AWS Glue

B.Amazon EMR

C.Amazon Athena

D.AWS Data Pipeline

AnswerA

Glue is serverless, cost-effective, and ideal for one-time ETL.

Why this answer

AWS Glue is the correct choice because it is a fully managed, serverless ETL service designed for one-time or scheduled data transformation jobs. It automatically provisions and scales the underlying Spark environment, requires no infrastructure management, and charges only for the resources consumed during job execution, making it highly cost-effective for a 500 GB ETL workload from RDS to S3.

Exam trap

The trap here is that candidates often choose Amazon EMR because they associate it with big data ETL, but they overlook that EMR requires cluster management and is not cost-effective for a one-time job, while AWS Glue's serverless, pay-per-use model is explicitly designed for such use cases.

How to eliminate wrong answers

Option B (Amazon EMR) is wrong because it requires manual cluster provisioning, configuration, and ongoing management, which increases operational overhead and cost for a one-time job, and is not the most cost-effective or minimal-management solution. Option C (Amazon Athena) is wrong because it is an interactive query service for analyzing data in S3 using SQL, not an ETL service; it cannot directly transform data from RDS and does not support complex ETL transformations or writing transformed data back to S3 in a single job. Option D (AWS Data Pipeline) is wrong because it is a workflow orchestration service that requires managing compute resources (e.g., EC2 instances) and is less suited for a one-time ETL job compared to Glue's serverless, pay-per-use model.

Practice this question →

177

MCQmedium

A company is using Amazon Kinesis Data Streams to ingest real-time clickstream data. The data must be transformed before being stored in Amazon S3. The transformations include enrichment with reference data from Amazon DynamoDB. Which AWS service should be used to perform the transformation with minimal operational overhead?

A.Amazon Kinesis Data Firehose with data transformation

B.AWS Lambda functions invoked by Kinesis Data Streams

C.Amazon Kinesis Data Analytics for Apache Flink

D.Amazon EMR with Apache Spark Streaming

AnswerC

Managed Flink application can perform complex transformations and enrichments with low operational overhead.

Why this answer

Amazon Kinesis Data Analytics for Apache Flink (Option C) is the correct choice because it provides a fully managed, stateful stream processing engine that can read directly from Kinesis Data Streams, enrich records with reference data from DynamoDB via Flink's Async I/O or JDBC connectors, and write the transformed data to S3—all without provisioning or managing any infrastructure. This minimizes operational overhead compared to self-managed solutions like EMR or Lambda-based architectures that require custom checkpointing and scaling logic.

Exam trap

The trap here is that candidates often choose Kinesis Data Firehose (Option A) because it directly integrates with S3 and DynamoDB via Lambda, but they overlook that Firehose cannot perform stateful joins or handle reference data enrichment at scale without complex custom code, whereas Kinesis Data Analytics for Apache Flink is purpose-built for exactly this pattern with minimal operational overhead.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Firehose with data transformation uses Lambda functions for per-record transformations, but it cannot natively perform stateful operations like joining with DynamoDB reference data; it is designed for simple, stateless transformations and direct S3 delivery, not complex enrichment. Option B is wrong because AWS Lambda functions invoked by Kinesis Data Streams can handle per-record enrichment, but they require manual management of batch sizes, retries, and scaling, and they lack built-in support for stateful operations like windowed joins or exactly-once semantics, leading to higher operational overhead for complex transformations. Option D is wrong because Amazon EMR with Apache Spark Streaming introduces significant operational overhead for cluster provisioning, tuning, and maintenance, and is overkill for a stream enrichment use case that can be handled by a fully managed service like Kinesis Data Analytics.

Practice this question →

178

MCQhard

A company uses Amazon EMR to run Spark jobs on a large dataset stored in Amazon S3. The jobs are failing with 'OutOfMemoryError' in the executors. The data is not skewed. Which configuration change will most likely resolve the issue?

A.Enable Kryo serialization

B.Decrease the number of shuffle partitions

C.Increase the spark.executor.memoryOverhead setting

D.Increase the number of executor cores

AnswerC

Memory overhead handles JVM overhead and off-heap memory, preventing OOM errors.

Why this answer

When Spark executors run out of memory during shuffle operations, the `spark.executor.memoryOverhead` setting is often the culprit. This parameter allocates off-heap memory for JVM overhead, internal metadata, and shuffle buffers. Increasing it provides more room for these operations without reducing the executor heap, directly addressing OutOfMemoryError in non-skewed data scenarios.

Exam trap

The trap here is that candidates often confuse executor memory (heap) with memoryOverhead (off-heap), assuming that increasing heap or reducing partitions will fix all OutOfMemoryErrors, when in fact shuffle-heavy workloads require explicit off-heap tuning.

How to eliminate wrong answers

Option A is wrong because Kryo serialization reduces memory used for object serialization but does not increase the total memory available to executors; it cannot resolve an OutOfMemoryError caused by insufficient off-heap or shuffle memory. Option B is wrong because decreasing the number of shuffle partitions reduces parallelism and can actually increase the memory pressure per partition, potentially worsening the OutOfMemoryError. Option D is wrong because increasing executor cores increases the number of concurrent tasks per executor, which consumes more memory per core and can exacerbate memory exhaustion rather than resolve it.

Practice this question →

179

MCQmedium

A data engineering team is building a pipeline to process terabytes of log data daily using Amazon EMR with Spark. The data arrives in hourly batches and must be processed within 4 hours. The team needs to minimize cost. Which cluster configuration is MOST cost-effective?

A.Use a single large instance with multiple cores to avoid data shuffling.

B.Use a transient cluster with a mix of on-demand and spot instances, terminated after the job completes.

C.Use a long-running cluster of on-demand instances to avoid startup time.

D.Use Amazon EMR Serverless to automatically scale.

AnswerB

Transient clusters reduce idle cost, spot instances lower compute cost.

Why this answer

A transient cluster with a mix of on-demand and spot instances minimizes cost for batch workloads that have a defined lifecycle. Spot instances offer significant discounts (up to 90%) for fault-tolerant Spark jobs, and terminating the cluster after processing eliminates idle compute charges. This approach aligns with the 4-hour processing window and hourly batch arrival, as EMR can provision and tear down clusters quickly.

Exam trap

The trap here is that candidates overestimate the cost savings of EMR Serverless or long-running clusters, failing to recognize that transient spot-based clusters are the most cost-effective for fixed-window batch processing due to zero idle time and spot pricing discounts.

How to eliminate wrong answers

Option A is wrong because a single large instance creates a single point of failure and cannot horizontally scale to process terabytes of data within 4 hours; Spark relies on distributed parallelism across multiple nodes, and avoiding shuffles is not a cost optimization strategy. Option C is wrong because a long-running cluster of on-demand instances incurs continuous costs for idle time between hourly batches, wasting resources when no processing is needed. Option D is wrong because Amazon EMR Serverless, while autoscaling, typically incurs higher per-unit costs for sustained batch workloads compared to transient clusters with spot instances, and it lacks the fine-grained cost control of spot pricing.

Practice this question →

180

MCQeasy

A company is using AWS Glue to run ETL jobs that transform data from Amazon S3 to Amazon Redshift. The jobs are failing intermittently with timeouts. What is the most likely cause?

A.The S3 bucket policy is too restrictive.

B.The AWS Glue job does not have enough DPUs (Data Processing Units) allocated.

C.The Amazon Redshift cluster is in maintenance mode.

D.The source data is not compressed.

AnswerB

Insufficient resources can cause timeouts.

Why this answer

Intermittent timeouts in AWS Glue ETL jobs typically indicate insufficient resource allocation. DPUs (Data Processing Units) define the compute capacity for the job; if too few are allocated, the job may run slowly and exceed the default timeout (e.g., 2880 minutes) or internal service limits, especially when processing large datasets from S3 to Redshift. Increasing the DPU count or using the G.1X/G.2X worker types can resolve this.

Exam trap

The trap here is that candidates often confuse intermittent failures with configuration issues (like policies or maintenance) rather than recognizing that resource starvation (insufficient DPUs) is the classic cause of sporadic timeouts in distributed ETL jobs.

How to eliminate wrong answers

Option A is wrong because a restrictive S3 bucket policy would cause consistent access denied errors (HTTP 403), not intermittent timeouts. Option C is wrong because Redshift maintenance mode is a planned event that blocks all queries and connections, leading to immediate, persistent failures, not intermittent timeouts. Option D is wrong because uncompressed source data increases data volume and network transfer time, which can contribute to slower performance but does not directly cause intermittent timeouts; AWS Glue can handle uncompressed data, and compression is an optimization, not a requirement.

Practice this question →

181

MCQeasy

A data engineer needs to schedule an AWS Glue ETL job to run every hour. The job reads from an Amazon DynamoDB table and writes to Amazon S3. Which AWS service should the engineer use to trigger the Glue job on schedule?

A.Amazon Kinesis Data Streams

B.AWS Step Functions

C.Amazon EventBridge (CloudWatch Events)

D.AWS Lambda

AnswerC

EventBridge can schedule events to trigger Glue jobs.

Why this answer

Amazon EventBridge (formerly CloudWatch Events) can trigger AWS Glue jobs on a schedule using cron or rate expressions. Option A is incorrect because Amazon Kinesis Data Streams is for real-time streaming data, not scheduling. Option B is incorrect because AWS Step Functions is for orchestrating workflows, but scheduling is typically done via EventBridge.

Option D is incorrect because AWS Lambda is a compute service, not a scheduler, though it can be used in conjunction with EventBridge to trigger Glue, but the direct scheduler is EventBridge.

Practice this question →

182

MCQmedium

A company is streaming e-commerce events to Amazon Kinesis Data Streams. The data science team needs to join events from multiple shards in near real-time and then store the joined results in Amazon S3. Which solution would meet these requirements with the LEAST operational overhead?

A.Use AWS Lambda functions with Kinesis triggers to process each record, join across shards using a DynamoDB table for state, and write to S3.

B.Use Amazon Kinesis Data Firehose to buffer the data and write to S3, then use Amazon Athena to join the data after it is stored.

C.Use AWS Glue ETL jobs that read from the Kinesis stream via the Kinesis connector and write the joined results to S3.

D.Use Amazon Kinesis Data Analytics for Apache Flink to read from the Kinesis stream, perform a join operation using Flink SQL, and write the results to S3 using a sink connector.

AnswerD

Kinesis Data Analytics for Apache Flink supports stateful stream processing and can join across shards natively.

Why this answer

Amazon Kinesis Data Analytics for Apache Flink can read from a Kinesis stream, perform stateful joins across shards using Flink SQL or the DataStream API, and write the results to Amazon S3 via a sink connector, all with minimal operational overhead. Option A is wrong because AWS Lambda functions process each shard independently; joining across shards would require managing external state (e.g., DynamoDB), increasing complexity and latency. Option B is wrong because Amazon Kinesis Data Firehose buffers data and writes to S3, but it cannot perform joins; using Athena to join after storage introduces batch-like delays.

Option C is wrong because AWS Glue ETL jobs are batch-oriented and not designed for near real-time streaming; Glue Streaming ETL would still require significant configuration and is less optimized for stateful joins across shards.

Practice this question →

183

MCQhard

A company uses AWS Glue to run ETL jobs that transform data from Amazon RDS for MySQL to Amazon S3. The current job runs daily and takes 3 hours to process 100 GB of data. The company expects data volume to grow 10x in the next year. They need to reduce job runtime and cost. Which approach should they take?

A.Use S3 Select with Glue to filter data before transformation.

B.Use parallel reads with pushdown predicates in the Glue job's source connection, and write the output in columnar format (Parquet) partitioned by date.

C.Increase the number of Glue DPUs to 100 and enable job bookmarking.

D.Use Amazon Redshift Spectrum to perform transformations in place on S3.

AnswerB

Parallel reads with partition pushdown reduce load on RDS and speed up extraction; Parquet with partitioning reduces storage and query costs.

Why this answer

Using parallel reads with pushdown predicates reduces the amount of data transferred from RDS to Glue by filtering at the database level, which lowers extraction time and load on the source. Writing output in columnar format (Parquet) reduces storage size and improves query performance for downstream analytics. Partitioning by date enables efficient pruning.

Option A is incorrect because S3 Select is used for server-side filtering of data already in S3, not for tuning extraction from RDS. Option C is incorrect because increasing DPUs alone does not solve the bottleneck from the source database; pushdown predicates are more effective. Option D is incorrect because Redshift Spectrum is used for querying data in S3, not for performing transformations in ETL jobs.

Practice this question →

184

MCQeasy

A data engineer needs to transform large CSV files stored in S3 into Parquet format and load them into a data warehouse for analysis. The transformation must be cost-effective and serverless. Which AWS service should be used?

A.Amazon Athena

B.Amazon EMR with Spark

C.AWS Glue

D.AWS Data Pipeline

AnswerC

AWS Glue is a serverless ETL service that can perform the transformation efficiently.

Why this answer

AWS Glue is the correct choice because it provides a fully managed, serverless ETL service that can automatically convert CSV files from S3 into Parquet format using its built-in Spark engine. It is cost-effective as you only pay for the resources consumed during the job execution, and it integrates directly with data warehouses like Amazon Redshift for loading transformed data.

Exam trap

The trap here is that candidates confuse Amazon Athena's ability to query Parquet files with the ability to transform CSV into Parquet, overlooking that Athena is a query engine, not an ETL service, while Glue is purpose-built for serverless data transformation.

How to eliminate wrong answers

Option A is wrong because Amazon Athena is an interactive query service that can query CSV and Parquet files directly in S3, but it does not perform ETL transformations or convert file formats; it is for ad-hoc analysis, not data transformation. Option B is wrong because Amazon EMR with Spark requires provisioning and managing clusters, which is not serverless; it incurs costs for running EC2 instances even when idle, making it less cost-effective for occasional transformations. Option D is wrong because AWS Data Pipeline is a workflow orchestration service that can move and transform data, but it is not serverless (it relies on EC2 instances for task runners) and is primarily designed for scheduled data movement, not optimized for converting CSV to Parquet with built-in Spark capabilities.

Practice this question →

185

MCQmedium

A company uses Amazon Kinesis Data Firehose to ingest streaming data and deliver it to an S3 bucket. The data is in JSON format with a timestamp field. The data science team wants to query the data using Athena with partitioning by year/month/day. How should the S3 data be organized?

A.Configure Firehose to use dynamic partitioning with custom prefix

B.Store data in a single prefix and use Athena's 'partition projection' feature

C.Use AWS Glue crawler to partition the data after delivery

D.Use Amazon EMR to partition the data after delivery

AnswerA

Firehose dynamic partitioning creates directories based on record fields or timestamps.

Why this answer

Kinesis Firehose can partition data using custom prefixes like 'year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/'. This creates Hive-style partitions that Athena can automatically discover.

Practice this question →

186

MCQhard

An S3 event notification triggers an AWS Lambda function when a new object is created. The Lambda function parses the event and processes the object. The function is failing with a timeout error for large objects. Which approach should be used to handle large objects efficiently?

A.Increase the Lambda function timeout to 15 minutes

B.Use an SQS queue to buffer event notifications and configure Lambda with a batch window

C.Stream events to Amazon Kinesis Data Streams and process with Lambda

D.Use AWS Step Functions to orchestrate the processing

AnswerA

Increasing the Lambda timeout is the direct solution to timeout errors. Lambda allows a maximum timeout of 15 minutes, which can accommodate processing of larger objects.

Why this answer

Increasing the Lambda function timeout to the maximum of 15 minutes directly addresses the timeout error for large objects. This allows the function to complete processing within the allotted time. Option B (SQS queue) does not solve the timeout issue because the processing time itself is too long; buffering events does not reduce processing time.

Option C (Kinesis) adds unnecessary streaming complexity. Option D (Step Functions) adds orchestration overhead without addressing the timeout.

Practice this question →

187

MCQhard

A data pipeline uses AWS Glue to transform data from Amazon RDS to Amazon S3. The team wants to ensure that only new or updated records are processed in each run, minimizing cost and time. Which AWS Glue feature should be used?

A.Use Glue triggers to run the job on a schedule.

B.Use Glue partition pruning to filter data.

C.Use Glue crawlers to detect new data.

D.Enable Glue Job Bookmarks.

AnswerD

Job Bookmarks maintain state and process only new or changed data.

Why this answer

Glue Job Bookmarks track processed data and persist state between job runs, enabling incremental processing of new or updated records from a source like Amazon RDS. This minimizes cost and time by avoiding full table scans and reprocessing unchanged data.

Exam trap

The trap here is confusing scheduling (triggers) or schema discovery (crawlers) with stateful incremental processing, leading candidates to overlook the bookmark mechanism that specifically tracks record-level changes.

How to eliminate wrong answers

Option A is wrong because Glue triggers schedule job execution but do not track which records have been processed, so they cannot ensure only new/updated records are handled. Option B is wrong because partition pruning filters data based on partition columns in S3, not on record-level changes from a relational database like RDS. Option C is wrong because Glue crawlers update the Data Catalog schema and detect new partitions, but they do not track incremental record changes or support stateful processing of updated rows.

Practice this question →

188

Multi-Selecthard

A data engineer is designing a data pipeline that ingests data from a relational database into a data lake on Amazon S3. The data must be incrementally loaded daily. Which TWO AWS services can be used together to achieve this?

Select 2 answers

A.AWS Glue

B.Amazon Kinesis Data Streams

C.Amazon Athena

D.Amazon Redshift

E.AWS Database Migration Service (DMS)

AnswersA, E

Glue can use job bookmarks for incremental loads.

Why this answer

AWS Glue is correct because it provides a managed ETL service that can extract data from a relational database using JDBC connections, transform it, and write it incrementally to Amazon S3. Glue's built-in job bookmarking feature tracks processed data, enabling incremental loads by automatically skipping already-processed records during subsequent runs.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Streams (real-time streaming) with batch ingestion, or assume Amazon Athena can perform data ingestion when it is only a query engine, leading them to overlook the correct combination of Glue and DMS for scheduled incremental loads.

Practice this question →

189

MCQmedium

A company is using Amazon Athena to query a data lake in S3. Queries are slow and expensive. The data is stored as JSON. Which action will improve query performance and reduce cost?

A.Compress the JSON files using gzip

B.Partition the data by date

C.Convert the data to Parquet format

D.Increase the number of Athena workers

AnswerC

Parquet is columnar, reducing scanned data and improving performance.

Why this answer

Converting JSON data to Parquet format significantly improves Athena query performance and reduces cost. Parquet is a columnar storage format that allows Athena to scan only the columns needed for a query, drastically reducing the amount of data read from S3. This minimizes I/O and compute costs, as Athena charges based on the amount of data scanned.

In contrast, JSON is row-based and requires scanning entire files even for queries that only touch a few columns.

Exam trap

The trap here is that candidates often assume compression alone (gzip) is sufficient to improve performance, but they overlook that columnar formats like Parquet provide both compression and column pruning, which is the key to reducing scanned data and cost in Athena.

How to eliminate wrong answers

Option A is wrong because compressing JSON with gzip reduces storage size but does not change the row-based nature of JSON; Athena still must decompress and scan entire rows, limiting performance gains and cost reduction compared to columnar formats. Option B is wrong because partitioning by date improves query performance only if queries filter on that partition key, but it does not address the fundamental inefficiency of scanning entire JSON rows; partitioning alone is less effective than combining it with a columnar format. Option D is wrong because increasing the number of Athena workers (i.e., concurrency or DML query slots) does not reduce the amount of data scanned per query; it only allows more queries to run in parallel, which does not fix the root cause of slow and expensive queries.

Practice this question →

190

MCQhard

A company runs a real-time fraud detection pipeline using Amazon Kinesis Data Analytics. The pipeline reads from a Kinesis data stream, performs sliding window aggregations, and writes results to a DynamoDB table. The application is experiencing high latency during peak hours. Which action would MOST effectively reduce latency?

A.Enable DynamoDB auto scaling to handle write spikes.

B.Decrease the parallelism level in the Kinesis Data Analytics application.

C.Increase the number of shards in the Kinesis data stream.

D.Increase the sliding window size to reduce computational frequency.

AnswerC

More shards increase parallelism and reduce processing backlog.

Why this answer

Increasing the number of shards in the Kinesis data stream directly increases the ingestion capacity and parallelism of the stream, allowing the Kinesis Data Analytics application to consume and process records faster. This addresses the root cause of high latency during peak hours by scaling the data source throughput, which is the bottleneck in a streaming pipeline.

Exam trap

The trap here is that candidates confuse the symptom (high latency) with a downstream issue (DynamoDB write capacity) or computational efficiency (window size), rather than recognizing that the bottleneck is upstream at the data ingestion layer, which is the most common cause of latency in Kinesis-based streaming pipelines.

How to eliminate wrong answers

Option A is wrong because DynamoDB auto scaling adjusts write capacity based on load, but the latency issue originates upstream in the stream processing, not in the write destination; the pipeline is already writing to DynamoDB, and auto scaling would not reduce the time data spends waiting in the stream or being processed. Option B is wrong because decreasing parallelism in Kinesis Data Analytics reduces the number of concurrent processing tasks, which would increase latency by slowing down the sliding window aggregations, not reduce it. Option D is wrong because increasing the sliding window size reduces the frequency of computations but does not address the underlying throughput limitation; it may even increase latency by requiring more data to be buffered before results are emitted.

Practice this question →

191

MCQmedium

A company uses AWS Glue ETL jobs to transform data from Amazon RDS for MySQL to Amazon S3. The transformation includes aggregations and joins. The job runs daily and processes approximately 100 GB of data. Recently, the job started failing with memory errors on the worker nodes. Which approach would MOST effectively resolve the issue without changing the logic?

A.Switch from a Spark ETL job to a Python shell job

B.Decrease the number of workers to reduce overhead

C.Change the worker type from G.2X to G.1X to increase memory per worker

D.Increase the number of workers in the job configuration

AnswerD

More workers distribute the data processing, reducing memory per node.

Why this answer

Increasing the number of workers distributes the memory load across more nodes, which directly addresses memory errors in a Spark ETL job without altering the transformation logic. AWS Glue Spark jobs process data in memory across workers, and insufficient total memory causes out-of-memory errors when handling 100 GB of data with aggregations and joins.

Exam trap

The trap here is that candidates might confuse worker type (memory per worker) with number of workers (total cluster memory), incorrectly assuming a larger worker type always helps, when in fact increasing the number of workers is the direct fix for memory errors in distributed Spark jobs.

How to eliminate wrong answers

Option A is wrong because a Python shell job runs on a single node with limited memory and cannot handle 100 GB of data or Spark-based aggregations and joins. Option B is wrong because decreasing the number of workers reduces total cluster memory, worsening the memory errors. Option C is wrong because changing from G.2X (16 GB memory per worker) to G.1X (8 GB memory per worker) decreases memory per worker, which would exacerbate memory issues rather than resolve them.

Practice this question →

192

MCQeasy

A data engineer needs to transform raw clickstream data (JSON files) stored in S3 into a partitioned Parquet dataset for querying with Athena. The transformation includes cleaning, deduplication, and enrichment. The pipeline should run daily. Which solution is MOST cost-effective and requires the least operational overhead?

A.Launch an Amazon EMR cluster with Spark, transform the data, and terminate the cluster after completion.

B.Use an AWS Glue ETL job with a schedule trigger to perform the transformation and write to S3.

C.Use AWS Lambda functions triggered by S3 events to transform each file incrementally.

D.Use Amazon Athena to run CTAS queries to convert and partition the data daily.

AnswerB

Glue ETL is serverless, can handle complex transformations, and scheduling is built-in.

Why this answer

AWS Glue ETL jobs are serverless, require no cluster management, and can be easily scheduled for daily runs. Glue also integrates with the Data Catalog for partitioning. Option A (Amazon EMR) is not the most cost-effective or least operational overhead because it requires managing a cluster, even if it can be terminated after completion.

Option C (AWS Lambda) is not suitable for large-scale clickstream data due to execution time limits and lack of built-in support for complex transformations like deduplication. Option D (Amazon Athena CTAS) is not appropriate because Athena is primarily a query engine, not a transformation tool; CTAS queries are good for converting data formats but lack the flexibility for cleaning and enrichment.

Practice this question →

193

MCQeasy

A machine learning team is preparing a dataset for model training. The data is stored in an Amazon S3 bucket with objects that are each approximately 100 MB in size. The team wants to use Amazon SageMaker for training. To optimize training performance, which data format and storage configuration should be used?

A.Store data as RecordIO-Protobuf files and use SageMaker File input mode

B.Store data as RecordIO-Protobuf files and use SageMaker Pipe input mode

C.Store data as CSV files and use SageMaker Pipe input mode

D.Store data as CSV files and use SageMaker File input mode

AnswerB

Pipe mode streams data directly from S3, and RecordIO-Protobuf provides efficient binary format.

Why this answer

RecordIO-Protobuf is the optimal format for SageMaker because it stores data in a binary, sharded structure that allows for efficient random access and parallel I/O. Pipe input mode streams data directly from S3 to the training algorithm, eliminating disk writes and reducing startup latency, which is critical for large datasets with 100 MB objects.

Exam trap

The trap here is that candidates often assume 'File input mode' is always faster because it loads data locally, but they overlook that Pipe mode's streaming avoids disk I/O bottlenecks and is specifically optimized for binary formats like RecordIO-Protobuf.

How to eliminate wrong answers

Option A is wrong because File input mode downloads the entire dataset to the training instance's local disk before training begins, which adds significant startup time and I/O overhead for large objects. Option C is wrong because CSV files are text-based and require parsing line by line, which is slower than binary formats and does not support the sharded, parallel access that Pipe mode leverages. Option D is wrong because CSV files with File input mode combine the worst of both: text parsing overhead and full-disk download latency, making it the least performant choice.

Practice this question →

194

MCQmedium

A company uses AWS Glue to run ETL jobs that process data from Amazon RDS for PostgreSQL and load it into Amazon Redshift. The Glue job runs nightly and takes 6 hours to complete. The Redshift cluster is a single dc2.large node. The team needs to reduce the load time to under 3 hours. The data volume is 200 GB per night. The team is considering using Amazon Redshift Spectrum to query data directly from S3 instead of loading it. However, the data transformation logic is complex and requires multiple joins and aggregations that are currently performed in Glue. Which approach should the team recommend to meet the time requirement?

A.Use Redshift Spectrum to create external tables and run the transformations directly in Redshift, bypassing the Glue job.

B.Increase the Redshift cluster to a multi-node cluster with dc2.8xlarge nodes to improve COPY and query performance.

C.Split the Glue job into multiple parallel jobs that each load a portion of the data into separate Redshift tables, then use UNION ALL views.

D.Stage the data in S3 in Parquet format and use a COPY command with the PARQUET option to load data faster.

AnswerB

More nodes increase parallelism for loading and any post-load transformations.

Why this answer

Increasing the Redshift cluster to a multi-node configuration with dc2.8xlarge nodes significantly improves COPY performance by parallelizing data loading across multiple slices. The current single dc2.large node may be the bottleneck for the data loading portion of the Glue job. Faster loading can reduce the total 6-hour runtime to under 3 hours if loading is the dominant factor.

Note: the complex transformations remain in AWS Glue, not on Redshift. Option A is incorrect because Redshift Spectrum allows querying external tables without loading but does not accelerate the complex ETL transformations. Option C splits the load but does not address the single-node ingestion bottleneck and adds complexity.

Option D improves load speed with Parquet but the cluster's ingestion capacity is still the limiting factor.

Practice this question →

195

Multi-Selecthard

A company is using AWS Glue to catalog data stored in Amazon S3. The data is partitioned by year, month, day, and hour. The company runs hourly ETL jobs that add new partitions. The Glue crawler is scheduled to run every hour to update the Data Catalog. However, the crawler is taking longer than expected and is not completing before the next crawler run starts. Which action could the company take to resolve this issue?

Select 1 answer

A.Increase the throughput of the crawler by configuring the 'Schema updates' option

B.Enable partition indexing on the table to speed up the crawler

C.Decrease the crawler schedule frequency to every 2 hours to avoid overlapping runs

D.Use multiple crawlers, each configured to crawl a different path (e.g., one for year=2023, one for year=2024)

E.Increase the number of crawler instances by configuring the 'Crawler queue' to process multiple partitions in parallel

AnswersD

Correct. Using multiple crawlers configured to crawl different paths (e.g., by year) parallelizes the crawling work, reducing overall time and preventing overlaps.

Why this answer

Using multiple crawlers to crawl different paths allows parallel processing of partitions, reducing crawler time. Option A is incorrect because the 'Schema updates' option does not increase throughput. Option B is incorrect because partition indexing speeds up queries, not the crawler itself.

Option C is incorrect because reducing frequency does not speed up the crawler. Option E is incorrect because AWS Glue does not have a 'Crawler queue' feature; to increase parallelism, you would increase DPUs.

Practice this question →

196

Multi-Selectmedium

A company is building a data pipeline that uses Amazon Kinesis Data Streams to ingest real-time events. The pipeline then uses AWS Lambda to process the events and store results in Amazon DynamoDB. The company wants to ensure that the Lambda function can process all events without data loss and without duplicating processing. Which TWO configuration steps should the company take?

Select 2 answers

A.Increase the data retention period of the Kinesis stream to 7 days to allow reprocessing

B.Set the Lambda function's batch window to a small value (e.g., 1 second) to reduce processing latency

C.Enable the 'iterator age' metric in Amazon CloudWatch to monitor consumer lag

D.Use a single shard for the Kinesis stream to ensure order and avoid parallel processing issues

E.Configure the Lambda function to disable retries on failure to avoid duplicate processing

AnswersA, E

Correct. Increasing retention allows reprocessing of events, ensuring no data loss.

Why this answer

To ensure all events are processed without data loss and without duplicating processing, the company should increase the Kinesis stream retention to 7 days (option A). This allows reprocessing of events that fail initial processing, thus preventing data loss. The company should also configure the Lambda function to disable retries on failure (option E).

Automatic retries can cause the same batch of events to be processed multiple times, leading to duplication unless the Lambda function is idempotent. By disabling retries, the company can handle failures manually, ensuring exactly-once processing when combined with idempotent reprocessing logic. Option B (reducing batch window) does not prevent duplicates and may increase invocations.

Option C (enabling iterator age metric) is for monitoring, not preventing loss/duplication. Option D (single shard) does not prevent duplication and can cause throughput limitations.

Practice this question →

197

MCQhard

A data engineering team is designing a data lake on Amazon S3. Raw data is ingested in JSON format and must be partitioned by year, month, and day. The team expects high query performance for recent data but infrequent queries for older data. The data is immutable. Which storage tier configuration minimizes costs while meeting performance requirements?

A.Store all data in S3 Standard, then move to S3 Glacier after 30 days using a lifecycle policy

B.Store recent partitions in S3 Standard, older partitions in S3 One Zone-IA

C.Keep all data in S3 Standard because query performance is critical

D.Use S3 Intelligent-Tiering for the entire data lake

AnswerD

Intelligent-Tiering automatically moves data between access tiers based on usage, optimizing cost without retrieval delays.

Why this answer

S3 Intelligent-Tiering automatically moves objects between access tiers (frequent, infrequent, and archive instant access) based on changing access patterns, without performance impact or lifecycle management overhead. This matches the workload: recent data is queried frequently (automatic frequent tier), older data is queried rarely (automatic infrequent/archive instant tiers), and data is immutable, so no write/delete penalties apply. It minimizes cost by charging only for the tiers actually used, while maintaining millisecond latency for all tiers.

Exam trap

The trap here is that candidates assume S3 Standard is required for all queryable data, overlooking that S3 Intelligent-Tiering provides the same low-latency performance as S3 Standard for all tiers (including Infrequent Access and Archive Instant Access) while automatically reducing storage costs for infrequently accessed data.

How to eliminate wrong answers

Option A is wrong because moving all data to S3 Glacier after 30 days would cause high retrieval latency (minutes to hours) for any queries on data older than 30 days, violating the requirement for high query performance on recent data (which is fine) but also failing to provide acceptable performance for the infrequent queries on older data. Option B is wrong because S3 One Zone-IA does not provide the same durability (99.999999999% vs 99.99%) and availability as S3 Standard, and it is not designed for data that may be accessed infrequently but still requires low-latency retrieval; also, manually managing partitions across tiers is error-prone and does not adapt to changing access patterns. Option C is wrong because storing all data in S3 Standard incurs unnecessary costs for older data that is queried infrequently, as S3 Standard charges a higher per-GB storage price than infrequent access tiers, and the requirement explicitly asks to minimize costs.

Practice this question →

198

Multi-Selectmedium

A data engineer needs to transform and move 2 TB of data from an Amazon RDS for PostgreSQL instance to Amazon S3 daily. The transformation includes filtering, joining with data in S3, and aggregating. Which AWS services can be used together to accomplish this with minimal operational overhead? (Choose THREE.)

Select 3 answers

A.Amazon EMR

B.Amazon Redshift

C.Amazon S3

D.AWS Glue Data Catalog

E.AWS Glue

AnswersC, D, E

Target storage for transformed data.

Why this answer

Amazon S3 is correct because it serves as the target storage location for the transformed data. The daily 2 TB output from the ETL pipeline must be stored durably and cost-effectively, and S3 provides the ideal object storage layer for this purpose, especially when combined with AWS Glue for the transformation logic.

Exam trap

The trap here is that candidates often assume Amazon EMR or Redshift are necessary for large-scale data processing, but AWS Glue's serverless Spark engine can handle 2 TB daily without any cluster management, making it the lower-overhead choice.

Practice this question →

199

MCQmedium

A data engineer needs to transfer 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The on-premises network has a 100 Mbps connection to AWS. The transfer must be completed within one week. Which approach should the engineer use?

A.Use AWS Snowball Edge device to physically transfer the data.

B.Use Amazon S3 Transfer Acceleration.

C.Use AWS DataSync to transfer the data over the network.

D.Use multiple concurrent AWS CLI copy commands over VPN.

AnswerA

Snowball Edge can handle large data volumes without network limitations.

Why this answer

The on-premises network has a 100 Mbps connection, which yields a theoretical maximum transfer of about 1.08 TB per day (100 Mbps * 86400 seconds / 8 bits per byte / 1024^4 bytes per TB). To transfer 50 TB within 7 days, the required throughput is approximately 7.14 TB per day, far exceeding the available bandwidth. AWS Snowball Edge provides a physical shipping method that bypasses network constraints entirely, making it the only viable option for this volume and timeline.

Exam trap

The trap here is that candidates may overestimate the effectiveness of acceleration or parallelization techniques, failing to calculate that a 100 Mbps link can only transfer approximately 1.08 TB per day, making any network-based option mathematically impossible for 50 TB in one week.

How to eliminate wrong answers

Option B is wrong because Amazon S3 Transfer Acceleration optimizes network paths using AWS edge locations but does not increase the available bandwidth of the 100 Mbps link; the maximum transfer rate is still capped by the on-premises connection, making it impossible to transfer 50 TB in one week. Option C is wrong because AWS DataSync is a network-based transfer service that also depends on the 100 Mbps bandwidth; even with compression and parallelization, the total transfer time would exceed the one-week deadline. Option D is wrong because multiple concurrent AWS CLI copy commands over VPN still share the same 100 Mbps network bottleneck; while parallelism can improve utilization, it cannot overcome the fundamental bandwidth limitation, and the VPN overhead further reduces effective throughput.

Practice this question →

200

Multi-Selectmedium

A company is using Amazon Kinesis Data Streams with 10 shards to ingest clickstream data. Each record is approximately 50 KB. The data is consumed by a Lambda function that writes to DynamoDB. The Lambda function is experiencing throttling errors. Which TWO actions should the data engineer take to resolve the issue? (Choose TWO.)

Select 2 answers

A.Increase the record size to 1 MB to reduce the number of records

B.Switch to Kinesis Data Firehose instead of Data Streams

C.Request a limit increase for the Lambda function's concurrent execution limit

D.Increase the number of shards in the Kinesis stream

E.Increase the batch size in the Lambda event source mapping

AnswersC, E

This directly alleviates throttling by allowing more concurrent executions.

Why this answer

The Lambda function is experiencing throttling errors because it is being invoked too frequently. To resolve this, the data engineer should increase the Lambda function's concurrent execution limit (option C) to allow more simultaneous executions, and increase the batch size in the Lambda event source mapping (option E) to process more records per invocation, reducing the number of invocations. Option A (increase record size) is irrelevant as it would increase data volume.

Option B (switch to Kinesis Data Firehose) changes the architecture and does not directly address Lambda throttling. Option D (increase the number of shards) would increase throughput but also potentially increase concurrency without solving the throttling issue. Therefore, the correct answers are C and E.

Practice this question →

201

MCQmedium

A data engineer needs to automate the transformation of CSV files to Parquet format as soon as they are uploaded to an S3 bucket. The transformed files should be stored in another S3 bucket. Which solution is the most cost-effective and requires the least maintenance?

A.Configure an S3 event notification to invoke a Lambda function.

B.Configure an S3 event notification to invoke an AWS Glue job.

C.Run an Amazon EMR cluster continuously to watch for new files.

D.Set up an EC2 instance with a cron job to poll the S3 bucket.

AnswerA

Lambda is serverless, pay-per-execution, ideal for this use case.

Why this answer

AWS Lambda, triggered by S3 event notifications, provides a serverless, event-driven architecture that automatically converts CSV to Parquet upon file upload. This approach is cost-effective because you pay only for compute time during execution, and it requires minimal maintenance as AWS manages the infrastructure, scaling, and fault tolerance.

Exam trap

The trap here is that candidates often over-engineer the solution by choosing AWS Glue or EMR for a simple file format conversion, failing to recognize that Lambda is the most cost-effective and low-maintenance option for lightweight, event-driven transformations.

How to eliminate wrong answers

Option B is wrong because invoking an AWS Glue job via S3 event notification incurs higher costs and longer startup times (Glue job startup overhead) compared to Lambda, and Glue is designed for complex ETL pipelines, not simple file-by-file conversions. Option C is wrong because running an Amazon EMR cluster continuously to watch for new files is expensive (cluster running 24/7) and requires ongoing maintenance of cluster configuration, scaling, and monitoring. Option D is wrong because setting up an EC2 instance with a cron job to poll the S3 bucket introduces ongoing costs for the running instance, manual maintenance of the OS and cron scripts, and potential latency from polling intervals.

Practice this question →

202

MCQeasy

A company wants to perform real-time analytics on streaming data from clickstreams. The data needs to be ingested, processed, and made available for querying within seconds. Which AWS service should be used for the processing step?

A.AWS Glue

B.Amazon Redshift

C.Amazon Kinesis Data Analytics

D.Amazon Athena

AnswerC

Kinesis Data Analytics processes streaming data in real-time.

Why this answer

Amazon Kinesis Data Analytics is the correct choice because it enables real-time processing and analysis of streaming data using SQL or Apache Flink. It can ingest data from Kinesis Data Streams or Kinesis Data Firehose, process it with sub-second latency, and output results to destinations like Kinesis Data Streams or Firehose for further querying, meeting the requirement for analytics within seconds.

Exam trap

The trap here is that candidates often confuse AWS Glue's streaming ETL capability (which still relies on Spark Structured Streaming with higher latency) with Kinesis Data Analytics' native real-time processing, or they assume Athena can query streaming data directly when it only queries data at rest in S3.

How to eliminate wrong answers

Option A is wrong because AWS Glue is a serverless ETL service designed for batch processing and data cataloging, not for real-time stream processing with sub-second latency. Option B is wrong because Amazon Redshift is a data warehouse optimized for analytical queries on large datasets, but it is not designed for real-time stream processing; it ingests data in batches or via streaming ingestion with higher latency. Option D is wrong because Amazon Athena is an interactive query service for analyzing data in Amazon S3 using SQL, but it operates on data at rest and cannot process streaming data in real time.

Practice this question →

203

Multi-Selecthard

Which THREE factors should be considered when choosing between Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose for a real-time data ingestion pipeline? (Choose three.)

Select 3 answers

A.The need for built-in data transformation and analytics.

B.The need for custom real-time processing logic using consumer applications.

C.The required end-to-end latency (seconds vs. minutes).

D.The need to manually manage shard capacity and scaling.

E.The requirement for exactly-once delivery semantics.

AnswersB, C, D

Correct: Data Streams supports custom consumers; Firehose does not.

Why this answer

Kinesis Data Streams supports custom real-time processing via consumer applications using the Kinesis Client Library (KCL) or AWS Lambda, enabling fine-grained control over record processing, checkpointing, and custom logic. This is a key differentiator from Kinesis Data Firehose, which only supports built-in transformations via Lambda and does not allow direct consumer access to the stream.

Exam trap

The trap here is that candidates often confuse Kinesis Data Firehose's built-in Lambda transformations with the custom real-time processing capabilities of Kinesis Data Streams, overlooking that Firehose does not allow direct consumer applications or sub-second latency.

Practice this question →

204

MCQmedium

A company is building a data lake on Amazon S3. Raw data is ingested from multiple sources in different formats (CSV, JSON, Parquet). The data must be cataloged and made queryable using Amazon Athena. The data schema may evolve over time. Which approach minimizes manual effort and supports schema evolution?

A.Use Athena only, without a catalog, by directly querying files

B.Use Amazon EMR to process data and write to a Hive metastore

C.Use AWS Glue Crawlers to automatically create and update the Glue Data Catalog

D.Manually create tables in Athena using DDL statements

AnswerC

Crawlers automatically detect schema changes and update the catalog.

Why this answer

AWS Glue Crawlers automatically infer schema from data in S3, create and update the Glue Data Catalog tables, and handle schema evolution by detecting changes such as new columns or partitions. This minimizes manual effort because the crawler runs on a schedule or trigger, and the catalog is natively integrated with Athena for querying without any additional setup.

Exam trap

The trap here is that candidates may think Athena can query files directly without a catalog (Option A), but Athena relies on the Glue Data Catalog (or an external Hive metastore) to map file locations and schemas, making a catalog mandatory for querying.

How to eliminate wrong answers

Option A is wrong because Athena requires a catalog (the Glue Data Catalog or an external Hive metastore) to query data; directly querying files without a catalog is not supported. Option B is wrong because while EMR can write to a Hive metastore, this adds operational overhead for managing EMR clusters and does not automatically catalog data from multiple sources or handle schema evolution as seamlessly as Glue Crawlers. Option D is wrong because manually creating tables with DDL statements requires ongoing manual effort to update schemas as data evolves, and it does not scale well for multiple sources and frequent schema changes.

Practice this question →

205

MCQhard

A data engineer is investigating a slow Athena query on a partitioned table. The table is partitioned by year, month, and day, and the data is stored in S3 with the prefix pattern 'raw/YYYY/MM/DD/'. The engineer runs the above CLI command and sees that there are many small files. Which action would most improve query performance?

A.Convert the data to columnar format like Parquet or ORC.

B.Use S3DistCp to coalesce files into fewer, larger files.

C.Increase the number of partitions in the Athena DDL.

D.Add more partitions to reduce the amount of data scanned per query.

AnswerB

Coalescing reduces the number of files, improving query performance.

Why this answer

The core issue is that the Athena query is slow due to many small files, which increases the overhead of S3 LIST operations and task scheduling in the Presto/Trino engine underlying Athena. Coalescing these small files into fewer, larger files with S3DistCp reduces the number of S3 GET requests and minimizes the scheduling overhead, directly improving query throughput. This is a classic small-files problem, not a data format or partitioning issue.

Exam trap

The trap here is that candidates often confuse the small-files problem with data format optimization, choosing Parquet/ORC (Option A) because they know columnar formats are faster, but they miss that the primary bottleneck is file count, not encoding.

How to eliminate wrong answers

Option A is wrong because while columnar formats like Parquet or ORC improve compression and reduce I/O, they do not address the root cause of many small files; converting to columnar without coalescing still leaves the small-file overhead. Option C is wrong because increasing the number of partitions would create even more small files, worsening the problem and increasing metadata overhead. Option D is wrong because adding more partitions does not reduce the amount of data scanned per query if the query already filters on existing partitions; it would actually increase the number of files and metadata operations, making performance worse.

Practice this question →

206

Multi-Selecteasy

A company has a large number of small CSV files (hundreds of thousands) in an S3 bucket. A data engineer needs to run a SQL query on this data using Amazon Athena. The queries are currently slow and expensive. Which two actions will improve query performance and reduce cost?

Select 2 answers

A.Increase the S3 request rate per prefix to improve read throughput.

B.Compress the CSV files using gzip.

C.Partition the data by a commonly filtered column (e.g., date).

D.Increase the number of partitions by splitting files into smaller ones.

E.Convert the data to Parquet or ORC columnar format.

AnswersC, E

Partitioning limits the data scanned per query, improving performance and reducing cost.

Why this answer

The correct answers are C and E. Partitioning the data by a commonly filtered column (e.g., date) reduces the amount of data scanned by Athena, improving performance and cost. Converting the data to Parquet or ORC columnar format further reduces the data scanned and improves compression and query speed.

Option B (gzip compression) helps reduce storage and scan volume but is less impactful than partitioning and columnar format. Option A (increasing S3 request rate) does not directly improve Athena query performance. Option D (splitting into smaller files) can increase the overhead of reading many small files, potentially hurting performance.

Practice this question →

207

MCQeasy

A data scientist wants to query a dataset stored in Amazon S3 using standard SQL without provisioning any servers. The dataset is in CSV format and is updated daily. Which AWS service should be used?

A.Amazon Athena

B.Amazon Redshift

C.Amazon RDS

D.Amazon DynamoDB

AnswerA

Athena is serverless and supports SQL queries on S3 data.

Why this answer

Amazon Athena is a serverless interactive query service that allows you to query data directly from Amazon S3 using standard SQL without provisioning any servers. It is ideal for querying CSV datasets that are updated daily because it supports schema-on-read, meaning you can define the table schema at query time without loading or transforming the data beforehand.

Exam trap

The trap here is that candidates may confuse Amazon Athena with Amazon Redshift Spectrum, but Redshift Spectrum still requires a provisioned Redshift cluster, whereas Athena is truly serverless and directly queries S3 without any infrastructure.

How to eliminate wrong answers

Option B (Amazon Redshift) is wrong because it requires provisioning and managing a cluster of nodes, which incurs ongoing costs and administrative overhead, contradicting the 'without provisioning any servers' requirement. Option C (Amazon RDS) is wrong because it is a managed relational database service that requires provisioning a database instance and does not natively query data stored in S3 without additional tools like AWS Glue or federated queries. Option D (Amazon DynamoDB) is wrong because it is a NoSQL key-value and document database that does not support standard SQL queries and is not designed for querying CSV files stored in S3.

Practice this question →

208

MCQeasy

A data scientist needs to query a dataset stored as Parquet files in Amazon S3 using standard SQL without managing any infrastructure. Which service should they use?

A.Amazon Athena

B.Amazon QuickSight

C.AWS Glue

D.Amazon Redshift

AnswerA

Athena is serverless and supports SQL on S3.

Why this answer

Amazon Athena is a serverless interactive query service that allows you to query data stored in Amazon S3 using standard SQL without any infrastructure to manage. It directly reads Parquet files from S3 and executes queries using Presto under the hood, making it the ideal choice for this use case.

Exam trap

The trap here is that candidates often confuse AWS Glue (which has a 'crawler' and 'catalog' feature) as a query service, but it is an ETL and cataloging tool, not an interactive SQL query engine; similarly, Redshift Spectrum might seem serverless but still requires a provisioned Redshift cluster.

How to eliminate wrong answers

Option B (Amazon QuickSight) is wrong because it is a business intelligence (BI) visualization tool, not a query engine; it can connect to Athena but cannot directly query Parquet files with standard SQL on its own. Option C (AWS Glue) is wrong because it is a serverless data integration and ETL service, not an interactive query engine; while it can catalog Parquet metadata, it does not support ad-hoc SQL queries. Option D (Amazon Redshift) is wrong because it requires provisioning and managing a cluster (infrastructure), even with Redshift Spectrum; the question explicitly states 'without managing any infrastructure,' so a serverless option like Athena is required.

Practice this question →

209

MCQhard

A data engineer created a CloudFormation template for a Glue ETL job as shown. The job processes 500 GB of data and takes 90 minutes to complete. However, the job fails after 60 minutes. What is the MOST likely cause?

A.The IAM role does not have sufficient permissions.

B.The ScriptLocation S3 bucket is in a different region.

C.The Timeout property is set to 60 minutes, but the job requires more time.

D.The MaxRetries property is set to 0, so the job does not retry on failure.

AnswerC

The job is killed when it exceeds the timeout, causing failure.

Why this answer

The Glue ETL job has a `Timeout` property set to 60 minutes, but the job requires 90 minutes to complete. When the timeout is reached, AWS Glue forcibly terminates the job, causing it to fail. This is the most direct cause of the failure after exactly 60 minutes of execution.

Exam trap

The trap here is that candidates often confuse the `Timeout` property with the `MaxRetries` property, assuming that a job failing after a specific duration must be due to a retry limit rather than a timeout limit, or they overlook the exact 60-minute failure point as a clear indicator of a timeout being reached.

How to eliminate wrong answers

Option A is wrong because insufficient IAM permissions would typically cause an immediate failure at job start (e.g., when reading from S3 or writing to the target), not a failure after 60 minutes of processing. Option B is wrong because if the ScriptLocation S3 bucket were in a different region, the job would fail immediately at launch due to cross-region access restrictions, not after 60 minutes. Option D is wrong because MaxRetries set to 0 means the job will not be retried automatically after a failure, but it does not cause the initial failure itself; the job still runs until it encounters an error or timeout.

Practice this question →

210

MCQmedium

A company is using AWS Glue to run ETL jobs that process data from Amazon RDS to Amazon S3. The ETL jobs are failing intermittently with write timeout errors when writing to S3. The company wants to implement a retry mechanism for transient errors. What should the company do?

A.Configure the AWS Glue job to retry on failure by setting the 'Max retries' parameter

B.Increase the size of the Amazon EBS volumes attached to the Glue job

C.Use Amazon CloudWatch to monitor the job and manually restart on failure

D.Place the failed job messages in an Amazon SQS queue and reprocess them

AnswerA

Glue jobs can automatically retry up to a specified number of times.

Why this answer

AWS Glue provides a built-in retry mechanism via the 'Max retries' parameter, which automatically retries the job when transient errors like write timeouts to S3 occur. Option A directly addresses the need for a retry mechanism. Option B is incorrect because increasing EBS volume size does not fix S3 write timeout errors, which are network-related.

Option C is incorrect because manual restart via CloudWatch is not an automated retry mechanism. Option D is incorrect because AWS Glue does not natively integrate with SQS for job retries; the built-in retry parameter is the proper solution.

Practice this question →

211

MCQmedium

An AWS Glue ETL job failed with the error 'Insufficient memory allocated for the job'. The job run details show AllocatedCapacity: 5, WorkerType: Standard, NumberOfWorkers: 5. Which change should be made to resolve the issue?

A.Delete and recreate the job with a different name

B.Increase the job timeout to 3600 minutes

C.Increase the number of workers to 10

D.Change the worker type to G.2X

AnswerC, D

Increasing the number of workers increases the total allocated memory and DPUs, resolving the insufficiency.

Why this answer

The error indicates insufficient memory for the job. Both increasing the number of workers (Option C) and changing the worker type to G.2X (Option D) increase the total allocated memory. Option C scales out by adding more Standard workers, each with 16 GB memory and 1 DPU.

Option D scales up by switching to G.2X workers, each with 32 GB memory and 2 DPUs, effectively doubling the DPU and total memory for the same number of workers. Either change can resolve the insufficient memory error. Option A is incorrect because deleting and recreating the job does not change resource allocation.

Option B is incorrect because job timeout does not affect memory.

Practice this question →

212

Multi-Selecteasy

A company needs to transfer 10 TB of data from an on-premises data center to Amazon S3. The network bandwidth is limited to 100 Mbps, and the transfer must complete within 5 days. Which TWO options are viable? (Choose TWO.)

Select 2 answers

A.Use S3 Transfer Acceleration to speed up the transfer

B.Use S3 Multipart Upload to upload files in parallel

C.Use AWS Snowball Edge device to ship the data

D.Use AWS DataSync over the existing internet connection

E.Use AWS Direct Connect to establish a dedicated network connection

AnswersC, E

Snowball Edge is ideal for large data volumes over slow networks; physical shipping is faster.

Why this answer

AWS Snowball Edge is a physical data transport solution designed for large-scale data transfers over slow or unreliable networks. With 10 TB of data and a 100 Mbps link, the theoretical transfer time is over 9 days, exceeding the 5-day window. Snowball Edge bypasses the network bottleneck entirely by shipping the data via courier.

Option E is also correct because AWS Direct Connect provides a dedicated network connection that can offer higher and more consistent bandwidth than the existing internet connection. If provisioned with sufficient capacity (e.g., 1 Gbps), the transfer can complete within the 5-day window. Options A and B are not viable because they cannot overcome the physical bandwidth limitation of 100 Mbps.

Option D is not viable because DataSync still relies on the existing internet connection, which is insufficient.

Exam trap

The trap here is that candidates assume S3 Transfer Acceleration or Multipart Upload can magically overcome bandwidth limitations, but they only optimize existing throughput—they cannot exceed the physical capacity of the network link. Another trap is to overlook Direct Connect as a viable option, thinking it requires long lead times, but it can be provisioned quickly in some cases and effectively increases bandwidth.

Practice this question →

213

MCQmedium

A company is using AWS Glue to run ETL jobs that process data in an S3 data lake. The jobs are failing with out-of-memory errors when processing large files. Which configuration change should be made to resolve this issue?

A.Change the worker type to G.1X

B.Increase the number of DPUs allocated to the job

C.Partition the input data into smaller files

D.Enable job bookmark to process only new data

AnswerB

More DPUs provide more memory and compute resources.

Why this answer

Out-of-memory errors in AWS Glue ETL jobs indicate that the allocated memory (DPUs) is insufficient for the data being processed. Increasing the number of DPUs allocates more memory and compute capacity to the job, directly resolving the memory constraint. This is the standard approach for scaling Glue jobs handling large datasets.

Exam trap

The trap here is that candidates confuse scaling vertically (changing worker type) with scaling horizontally (adding DPUs), but for large files, increasing DPUs is the more effective and direct solution for out-of-memory errors in AWS Glue.

How to eliminate wrong answers

Option A is wrong because changing the worker type to G.1X (16 GB memory) from the default G.0X (8 GB) might help, but it does not increase total memory as effectively as adding DPUs; the question specifies 'large files' where scaling out with more DPUs is the correct fix. Option C is wrong because partitioning input data into smaller files is a data preparation step that can improve parallelism but does not resolve out-of-memory errors caused by insufficient DPU allocation; it addresses file size issues, not memory limits. Option D is wrong because enabling job bookmarks only tracks processed data to avoid reprocessing, which reduces runtime but does not increase memory or fix out-of-memory errors.

Practice this question →

214

MCQhard

A company runs a critical ETL job using AWS Glue that writes to an Amazon Redshift cluster. The job occasionally fails due to insufficient disk space on the Redshift cluster. How can the company automate the process to prevent this failure?

A.Use a CloudWatch alarm to trigger a Lambda function that resizes the cluster.

B.Use RA3 node types with managed storage.

C.Increase the number of slices in the Redshift cluster.

D.Reserve additional nodes for the Redshift cluster.

AnswerA

This automates scaling based on disk usage.

Why this answer

Using Amazon CloudWatch to monitor disk space and automatically resize the cluster is the best automated solution. Reserving nodes does not address space. Using RA3 nodes with managed storage is a good proactive step but does not automate resizing.

The correct answer is to monitor and auto-resize.

Practice this question →

215

MCQmedium

An IAM policy is attached to a data engineering role that writes to an S3 bucket. The policy is shown in the exhibit. What is the effect of this policy?

A.The role can write objects with any encryption, but reading is restricted to SSE-KMS only

B.The role can read and write any object without encryption restrictions

C.The role can only read objects; writing is always denied

D.The role must use SSE-KMS when writing objects; reading is allowed only if the object is encrypted with SSE-KMS

AnswerD

The Allow statement grants GetObject only when SSE-KMS is specified, and the Deny statement enforces SSE-KMS for PutObject.

Why this answer

The IAM policy uses a `Condition` block with `s3:x-amz-server-side-encryption` set to `aws:kms`, which enforces that any `PutObject` request must include SSE-KMS encryption. The `Deny` effect on `s3:GetObject` when encryption is not `aws:kms` ensures that reading objects without SSE-KMS is blocked. This combination allows writing only with SSE-KMS and reading only of objects encrypted with SSE-KMS, making option D correct.

Exam trap

The trap here is that candidates often overlook the `Deny` effect on `s3:GetObject` and assume the policy only restricts writes, missing that reading is also conditionally denied unless the object uses SSE-KMS.

How to eliminate wrong answers

Option A is wrong because the policy does not allow writing objects with any encryption; it explicitly denies PutObject if SSE-KMS is not used. Option B is wrong because the policy restricts both reading and writing based on encryption type, so unrestricted read/write is not permitted. Option C is wrong because the policy allows writing objects as long as SSE-KMS is used, and reading is allowed for SSE-KMS encrypted objects, so it is not a blanket denial of writes.

Practice this question →

216

MCQhard

Refer to the exhibit. A company is using the Kinesis stream 'my-stream' with one shard. The producer is sending 1000 records per second, each 1 KB. The consumer is reading from the stream using the Kinesis Client Library (KCL). The consumer is able to process 500 records per second per shard. What is the most likely cause of the consumer falling behind?

A.The retention period is set to 24 hours, which is too short.

B.The stream uses KMS encryption, which adds latency.

C.The stream has only one shard, which limits the read throughput to 1 MB/s.

D.The consumer application is not using enhanced fan-out.

AnswerB

KMS encryption adds latency for both producer and consumer. Since the consumer is processing at half the producer rate, decryption overhead is a likely contributor to the consumer falling behind.

Why this answer

The consumer is processing 500 records per second per shard, which is half the producer's rate of 1000 records per second. While the shard's read throughput limit is 2 MB/s, not 1 MB/s, the consumer's processing speed is the bottleneck. Among the options, KMS encryption is the most plausible cause because decryption latency can significantly slow down the consumer, especially if the consumer application is not optimized for encryption overhead.

Exam trap

Candidates often mistake the shard's read throughput limit as 1 MB/s (the write limit) rather than the actual 2 MB/s. In this scenario, the shard is not the bottleneck; the consumer's processing rate is lower due to factors like KMS encryption latency.

How to eliminate wrong answers

Option A is wrong because the retention period (default 24 hours, max 365 days) controls how long records are stored in the stream, not the rate at which data can be consumed; a short retention period does not cause the consumer to fall behind—it only causes data to expire sooner. Option B is wrong because KMS encryption adds latency only during key retrieval and decryption, but the consumer's processing rate of 500 records per second is a software limitation, not a network or encryption overhead issue; KMS encryption does not reduce the shard's throughput. Option D is wrong because enhanced fan-out is a feature that provides dedicated read throughput of 2 MB/s per consumer per shard, but the consumer is already processing only 500 records per second (0.5 MB/s), which is well below the standard shard read limit of 2 MB/s; enhanced fan-out would not help if the consumer's processing logic is the bottleneck.

Practice this question →

217

MCQmedium

A company uses Amazon DynamoDB as the primary data store for a real-time recommendation engine. The data engineering team needs to export a daily snapshot of the DynamoDB table to S3 for offline analytics. The table is large (10 TB) and has a high read/write throughput. Which method will export the data with the least impact on the production workload?

A.Use AWS Data Pipeline to export the DynamoDB table to S3.

B.Use DynamoDB Scan API with parallel scans to export data to S3.

C.Use the DynamoDB export to S3 feature available in the AWS Console or CLI.

D.Use AWS Glue ETL job with a DynamoDB connection to export data.

AnswerC

This feature exports data without consuming read capacity units, minimizing impact.

Why this answer

The native DynamoDB export to S3 feature uses the table's internal backup mechanism (point-in-time recovery) to export data without consuming any read capacity units (RCUs) from the production table. This ensures zero impact on the live workload, even for a 10 TB table with high throughput.

Exam trap

The trap here is that candidates assume any data extraction from DynamoDB must use the Scan API (options A, B, D) and overlook the native export feature that bypasses the live table entirely, which is the only zero-impact method for large, high-throughput tables.

How to eliminate wrong answers

Option A is wrong because AWS Data Pipeline uses the DynamoDB Scan API under the hood, which consumes RCUs and can throttle production reads on a high-throughput table. Option B is wrong because the DynamoDB Scan API, even with parallel scans, consumes RCUs and can degrade performance for a large table with high read/write throughput. Option D is wrong because AWS Glue ETL jobs with a DynamoDB connection also use the Scan API, consuming RCUs and potentially causing throttling or increased latency for the production workload.

Practice this question →

218

MCQeasy

A data pipeline uses AWS Glue to crawl an S3 bucket and create a table in the AWS Glue Data Catalog. The data is in Parquet format with partitions by date. After a new partition is added to S3, the crawler runs but the new partition is not reflected in the table. What is the most likely cause?

A.The crawler requires an AWS Lambda trigger to be configured for new partitions.

B.The Parquet schema in the new partition does not match the existing table schema.

C.The new partition folder does not follow the Hive-style partition naming convention expected by the crawler.

D.The S3 bucket has too many partitions, exceeding the Glue crawler limit.

AnswerC

Glue crawlers require partition folders to follow the key=value pattern to automatically detect partitions.

Why this answer

The most likely cause is that the new partition folder does not follow the Hive-style partition naming convention expected by the crawler. AWS Glue crawlers expect partition directories to be named in the format key=value (e.g., date=2023-01-01). If the partitions are named differently, the crawler will not recognize them as partitions.

Option A is incorrect because Glue crawlers do not require Lambda triggers. Option B is incorrect because schema mismatch would cause a different error, not just missing partitions. Option D is incorrect because while there is a limit on partitions, it is high enough that 'too many partitions' is less likely than a naming issue.

Practice this question →

219

MCQeasy

A company wants to use Amazon SageMaker to train a model on a dataset stored in Amazon S3. The dataset is 100 GB and consists of millions of small JSON files. What should the data engineering team do to optimize training performance?

A.Combine the small JSON files into larger Parquet files using a Spark job on Amazon EMR.

B.Copy the data to an Amazon EBS volume attached to the training instance.

C.Use Amazon Athena to convert the data into a single CSV file.

D.Use S3 Select to filter data before training.

AnswerA

Parquet with larger files improves read efficiency and reduces overhead.

Why this answer

Combining millions of small JSON files into larger Parquet files using a Spark job on Amazon EMR is correct because it reduces the overhead of S3 LIST and GET requests during training. Parquet's columnar format also improves compression and allows SageMaker to read only the necessary columns, significantly accelerating I/O-bound training workloads.

Exam trap

The trap here is that candidates assume S3 Select or Athena can magically optimize small-file performance, but they fail to realize that the core issue is the sheer number of S3 API requests, which only consolidation into larger files can solve.

How to eliminate wrong answers

Option B is wrong because copying 100 GB of small files to an EBS volume attached to the training instance does not address the fundamental problem of millions of small files; it merely moves the I/O bottleneck from S3 to EBS, and EBS volumes have limited throughput and size constraints that can throttle training. Option C is wrong because using Amazon Athena to convert the data into a single CSV file would create a massive single file that SageMaker must read sequentially, eliminating parallelism and causing severe I/O bottlenecks; CSV also lacks the compression and columnar efficiency of Parquet. Option D is wrong because S3 Select only filters data server-side but does not consolidate the millions of small files; the training job still must issue a separate request for each file, overwhelming the S3 API rate limits and causing significant latency.

Practice this question →

220

MCQmedium

A company is building a data lake on Amazon S3. They need to enforce encryption at rest for all objects. Which combination of actions will achieve this? (Assume the bucket is versioned.)

A.Use AWS KMS with automatic key rotation

B.Enable S3 default encryption and set a bucket policy to deny PutObject without encryption headers

C.Enable S3 default encryption only

D.Enable S3 Block Public Access

AnswerB

This ensures all objects are encrypted.

Why this answer

Combining S3 default encryption with a bucket policy that denies PutObject requests lacking encryption headers ensures that every object stored in the bucket is encrypted at rest, even if the PutObject call does not include encryption parameters. Default encryption alone can be overridden by a client that explicitly sets encryption headers, but the bucket policy enforces encryption for all uploads, closing that loophole. This dual approach guarantees compliance with encryption-at-rest requirements for a versioned bucket.

Exam trap

The trap here is that candidates assume S3 default encryption alone is sufficient, but the exam tests the nuance that default encryption can be overridden by client-supplied headers, requiring a bucket policy to enforce encryption for all PutObject requests.

How to eliminate wrong answers

Option A is wrong because using AWS KMS with automatic key rotation only manages the encryption key lifecycle but does not enforce that every object is encrypted at rest; it is a key management feature, not an enforcement mechanism. Option C is wrong because enabling S3 default encryption only applies encryption to objects that are uploaded without encryption headers, but clients can still upload unencrypted objects by explicitly providing a `x-amz-server-side-encryption` header set to `AES256` or `aws:kms`, bypassing the default. Option D is wrong because S3 Block Public Access is a security control that prevents public access to buckets and objects, but it has no effect on encryption at rest; it addresses network access control, not data protection at rest.

Practice this question →

221

MCQmedium

A company uses Amazon DynamoDB as the primary data store for a real-time application. The data science team wants to analyze the data using Amazon Athena. What is the most efficient way to make the DynamoDB data available for Athena queries?

A.Use AWS Glue to extract data from DynamoDB and load into S3 on a schedule.

B.Use Amazon Redshift Spectrum to query DynamoDB directly.

C.Use DynamoDB Streams to invoke an AWS Lambda function that writes data to Amazon S3 in Parquet format. Then query the data in S3 using Athena.

D.Use Amazon EMR to read directly from DynamoDB and run Hive queries.

AnswerC

This provides a decoupled, cost-effective solution for analytics.

Why this answer

DynamoDB Streams captures real-time changes, and an AWS Lambda function can efficiently write these changes to Amazon S3 in Parquet format, which is optimized for columnar storage and Athena queries. This approach minimizes the overhead of scheduled batch jobs and provides near-real-time data availability for analytics.

Exam trap

The trap here is that candidates may assume scheduled batch extraction (Option A) is sufficient for real-time analysis, overlooking the efficiency of streaming-based incremental updates that avoid full table scans and reduce costs.

How to eliminate wrong answers

Option A is wrong because using AWS Glue to extract data from DynamoDB and load into S3 on a schedule introduces latency and is less efficient for real-time analysis compared to streaming-based approaches. Option B is wrong because Amazon Redshift Spectrum cannot query DynamoDB directly; it only supports querying data in Amazon S3 or other data sources via external tables, not DynamoDB. Option D is wrong because Amazon EMR reading directly from DynamoDB and running Hive queries is inefficient for Athena-based analysis, as it requires managing a separate cluster and does not directly make data available in S3 for Athena.

Practice this question →

222

Multi-Selecteasy

A team wants to move data from an on-premises Oracle database to Amazon S3 for analytics. The pipeline must run daily and handle incremental updates. Which THREE services should they use together? (Choose three.)

Select 3 answers

A.Amazon SageMaker

B.Amazon S3

C.Amazon Athena

D.AWS Database Migration Service (DMS)

E.AWS Glue

AnswersB, D, E

S3 is the target data lake storage.

Why this answer

Amazon S3 is the correct destination for storing the data because it provides a scalable, durable, and cost-effective object storage solution ideal for analytics workloads. The pipeline requires daily incremental updates, and S3 integrates seamlessly with AWS DMS for continuous replication and AWS Glue for ETL processing, making it the central storage layer for the analytics pipeline.

Exam trap

The trap here is that candidates often confuse Amazon Athena as a data ingestion service because it can query S3 data, but it is purely a query engine and cannot move or replicate data from an on-premises database.

Practice this question →

223

Multi-Selecteasy

Which TWO services can be used to transform data in transit within a Kinesis Data Firehose delivery stream? (Choose 2)

Select 2 answers

A.AWS Lambda

B.Amazon Kinesis Data Analytics

C.Amazon Athena

D.Amazon S3

E.AWS Glue

AnswersA, E

Firehose can invoke a Lambda function to transform records.

Why this answer

AWS Lambda is correct because it can be invoked as a transformation function within a Kinesis Data Firehose delivery stream. When you enable data transformation, Firehose buffers incoming records and then calls a Lambda function you specify, passing batches of records for processing. The Lambda function can modify, enrich, filter, or reformat the data before Firehose continues delivering it to the destination.

Exam trap

The trap here is that candidates often confuse 'transformation' with 'analytics' and select Kinesis Data Analytics, not realizing that Firehose's built-in transformation feature is specifically powered by Lambda, not by a separate analytics engine.

Practice this question →

224

Multi-Selecthard

A machine learning team is using Amazon SageMaker to train a model on a dataset stored in S3. The training job reads data from S3 using Pipe input mode, but the training is slow. The team wants to improve data throughput. Which THREE actions should they take?

Select 3 answers

A.Enable S3 Transfer Acceleration on the bucket.

B.Mount the S3 bucket using an S3 file system and use File mode with a larger instance type.

C.Use Amazon S3 VPC Gateway Endpoint to reduce data transfer costs and improve latency.

D.Use Amazon EFS as the data source for training.

E.Use Amazon ElastiCache to cache the training data.

AnswersB, C, D

File mode with high-bandwidth instances can improve throughput.

Why this answer

Mounting an S3 bucket using an S3 file system (e.g., via mount-s3 or s3fs) and switching to File mode allows the training instance to access data as local files, eliminating the overhead of streaming decompression and per-record parsing inherent in Pipe mode. Using a larger instance type provides more network bandwidth and CPU resources to handle the file I/O, directly improving data throughput for large datasets.

Exam trap

The trap here is that candidates often assume Pipe mode is always faster because it avoids disk writes, but they overlook that File mode with a larger instance can achieve higher throughput by leveraging parallel downloads and local caching, especially when the dataset is large or the algorithm benefits from random access.

Practice this question →

225

MCQhard

A data engineer runs the above CLI command and sees that the bucket contains many small Parquet files (1 MB each) under the prefix. When querying this data with Athena, the query performance is poor and costs are high. Which approach would MOST improve performance and reduce cost?

A.Convert the files to JSON format

B.Convert the files to CSV format

C.Consolidate the small files into fewer, larger Parquet files

D.Add more partitions by including hour in the prefix

AnswerC

Fewer, larger files reduce overhead and improve compression.

Why this answer

C is correct because consolidating many small Parquet files into fewer, larger files (e.g., 128–256 MB each) reduces the overhead of Amazon Athena's file listing and metadata operations, and improves compression and predicate pushdown efficiency. Parquet is a columnar format optimized for analytics, so keeping it while reducing file count directly addresses the root cause of poor performance and high cost.

Exam trap

The trap here is that candidates may think adding more partitions always improves query performance, but in this scenario with many tiny files, more partitions would exacerbate the small-file problem and increase Athena's overhead.

How to eliminate wrong answers

Option A is wrong because converting to JSON, a text-based row-oriented format, would increase storage size, eliminate columnar compression and predicate pushdown, and worsen Athena performance and cost. Option B is wrong because CSV is also a row-oriented text format that lacks compression and columnar optimizations, leading to higher scan volumes and slower queries. Option D is wrong because adding more partitions (e.g., by hour) would create even more small files and partitions, increasing metadata overhead and potentially degrading performance further, not improving it.

Practice this question →