Courseiva

CCNA Data Engineering Questions

75 of 350 questions · Page 1/5 · Data Engineering · Answers revealed

1
MCQhard

A company uses Amazon EMR with Spark to process data daily. The job reads from S3 and writes to S3. Recently, the job started failing with 'S3AccessDenied' errors. The IAM role used by EMR has not changed. What is the MOST likely cause?

A.The EMR cluster's security group blocks outbound traffic
B.The S3 bucket policy was updated to deny access to the EMR role
C.The S3 bucket was deleted and recreated
D.The EMR service role was rotated
AnswerB

Bucket policies can deny access even if IAM allows.

Why this answer

The IAM role used by EMR has not changed, but the S3 bucket policy can be updated independently to deny access to that specific role. An explicit deny in a bucket policy overrides any allow in the IAM policy, causing the 'S3AccessDenied' error even though the role itself remains unchanged. This is the most likely cause because the job was previously working and only the bucket policy could have been modified without touching the EMR configuration.

Exam trap

The trap here is that candidates assume the IAM role is the only factor in access control, overlooking that S3 bucket policies can be modified independently and can explicitly deny access to a specific role, causing 'AccessDenied' errors even when the role itself is unchanged.

How to eliminate wrong answers

Option A is wrong because security groups control network traffic, not IAM authorization; an outbound block would cause a timeout or connection failure, not an 'S3AccessDenied' error. Option C is wrong because deleting and recreating a bucket with the same name would result in a 'NoSuchBucket' error, not an access denied error, and the bucket name would need to be globally unique. Option D is wrong because rotating the EMR service role would change the role's credentials or ARN, which would break the job entirely (e.g., 'AssumeRole' failures), not produce an 'S3AccessDenied' error; the question states the IAM role used by EMR has not changed.

2
MCQeasy

A data engineer is building a data pipeline that ingests streaming data from IoT devices. The data must be processed in near real-time and stored in Amazon S3 for further analysis. Which AWS service should be used to capture and process the streaming data before storing it in S3?

A.Use Amazon S3 with S3 Event Notifications to trigger AWS Lambda for processing.
B.Use AWS Glue to perform ETL on the streaming data.
C.Use Amazon Kinesis Data Streams to capture the data and Amazon Kinesis Data Firehose to deliver it to S3.
D.Use Amazon Simple Queue Service (SQS) to buffer the data and then process it with AWS Lambda.
AnswerC

Kinesis Data Streams ingests real-time data and Kinesis Data Firehose delivers it to S3.

Why this answer

Amazon Kinesis Data Streams is designed for real-time data ingestion and can capture streaming data from IoT devices with low latency. Amazon Kinesis Data Firehose then reliably loads that streaming data into Amazon S3, handling buffering, compression, and partitioning automatically. This combination provides the near real-time processing and durable storage required for the pipeline.

Exam trap

The trap here is that candidates often confuse batch ETL services (like AWS Glue) or message queues (like SQS) with purpose-built streaming ingestion services, failing to recognize that Kinesis Data Streams and Firehose are the only AWS-native combination that captures, buffers, and delivers streaming data to S3 in near real-time without custom code.

How to eliminate wrong answers

Option A is wrong because S3 Event Notifications are triggered after data is already stored in S3, not for capturing or processing streaming data before storage; they lack the low-latency ingestion and buffering needed for near real-time streaming. Option B is wrong because AWS Glue is a batch ETL service designed for scheduled or on-demand transformation of data at rest, not for continuous, near real-time capture and processing of streaming data. Option D is wrong because Amazon SQS is a message queue service that does not natively support streaming data ingestion or direct delivery to S3; it would require custom Lambda logic to buffer and write to S3, adding complexity and latency compared to the purpose-built Kinesis Data Firehose integration.

3
MCQhard

A company is running a data pipeline that uses Amazon EMR with Spark to process 100 TB of data daily. The pipeline must complete within 6 hours. Currently, it takes 8 hours. Which optimization will most likely reduce the runtime?

A.Consolidate small input files into fewer larger files
B.Enable EMR Managed Scaling
C.Increase the memory of each node by using r5 instances
D.Use Spot Instances for all core nodes
AnswerB

Managed Scaling dynamically adds resources to meet deadlines.

Why this answer

EMR Managed Scaling automatically adjusts the number of core and task nodes based on workload, increasing parallelism and reducing runtime for large data pipelines. This is the most effective optimization when the current cluster is under-provisioned for the 100 TB workload. Option A is incorrect because consolidating input files reduces small file overhead but does not address a compute capacity bottleneck.

Option C is incorrect because increasing memory per node (e.g., r5 instances) helps memory-bound tasks, but the primary bottleneck here is likely throughput and parallelism, not memory. Option D is incorrect because Spot Instances can be interrupted, which may cause delays and increase runtime, making them unsuitable for a time-sensitive pipeline.

4
Multi-Selecthard

A company is building a real-time anomaly detection system for network traffic logs. The logs are ingested via Amazon Kinesis Data Streams and processed with an Amazon SageMaker endpoint for inference. The team needs to ensure that the inference results are stored durably and can be replayed for model retraining. The system must handle at least 10,000 records per second with low latency. Which three AWS services should the team use to build this architecture? (Select THREE.)

Select 3 answers
A.AWS Glue ETL
B.Amazon Kinesis Data Streams
C.Amazon Kinesis Data Analytics for Apache Flink
D.Amazon Kinesis Data Firehose
E.Amazon SageMaker
AnswersB, C, E

Kinesis Data Streams provides the ingestion layer with low latency and high throughput.

Why this answer

Amazon Kinesis Data Streams is the correct ingestion layer because it provides durable, real-time data streaming with the ability to handle over 10,000 records per second. It acts as the source of truth for network traffic logs, enabling low-latency processing and replay for model retraining.

Exam trap

The trap here is that candidates often confuse Kinesis Data Firehose with Kinesis Data Streams, assuming Firehose's simplicity and S3 integration make it suitable for real-time inference, but Firehose lacks the record-level replay and low-latency processing required for this use case.

5
MCQhard

A company uses Amazon Redshift as a data warehouse. They need to load 50 TB of clickstream data from S3 into Redshift daily. The data arrives in 5-minute intervals as gzipped CSV files. The target table has a sort key and a distribution key. The load must complete within 2 hours. Which approach is MOST efficient?

A.Use AWS Glue to transform the data and write to Redshift using JDBC.
B.Use a staging table and then merge using a stored procedure.
C.Use a series of INSERT statements from a Lambda function.
D.Use the COPY command with a manifest file and gzip compression.
AnswerD

COPY is optimized for bulk loading from S3.

Why this answer

The COPY command is the most efficient way to load large volumes of data into Amazon Redshift because it uses the cluster's massively parallel processing (MPP) architecture to read data directly from S3 in parallel across all nodes. With a manifest file, you can specify multiple gzipped CSV files, and the gzip compression reduces network I/O and storage overhead. This approach can easily load 50 TB within 2 hours, especially when the target table has a sort key and distribution key, as COPY automatically leverages these for optimal data distribution and sorting during the load.

Exam trap

The trap here is that candidates may overcomplicate the solution by choosing AWS Glue or staging tables, not realizing that Redshift's COPY command is purpose-built for high-speed parallel ingestion from S3 and is the most efficient method for bulk data loads.

How to eliminate wrong answers

Option A is wrong because AWS Glue writing to Redshift via JDBC is a row-by-row or small-batch operation that cannot match the parallel throughput of the COPY command, and it would introduce unnecessary transformation overhead for already-structured CSV data. Option B is wrong because using a staging table and a stored procedure merge adds extra steps and complexity without improving load speed; the COPY command can directly load into the target table with proper sort and distribution keys, making a staging table redundant for this bulk load scenario. Option C is wrong because a series of INSERT statements from a Lambda function would be extremely slow and inefficient for 50 TB of data, as each INSERT is a single-row operation that cannot leverage Redshift's parallel processing, and Lambda has a 15-minute execution timeout that would require complex orchestration to handle the full load.

6
MCQhard

A company uses Amazon Kinesis Data Streams to ingest real-time clickstream data from a website. The data is consumed by a Lambda function that writes records to an S3 bucket. Recently, the number of shards was increased from 2 to 4 to handle higher throughput. After the change, the Lambda function started processing records with increased latency and some records were being written out of order. What is the MOST likely cause?

A.The S3 bucket is not configured with versioning, causing overwrites.
B.The Lambda function is reading from the oldest sequence number, causing high IteratorAgeSeconds.
C.The Lambda function’s reserved concurrency is too low for the increased shard count.
D.The partition key used by the producer does not ensure that related records go to the same shard after resharding.
AnswerD

After resharding, the mapping of partition keys to shards changes. If ordering matters, the partition key must be chosen to keep related records together.

Why this answer

After resharding from 2 to 4 shards, the mapping of partition keys to shards changes. If the producer does not use a partition key that ensures related records (e.g., same user session) are routed to the same shard, records that were previously ordered within a shard may now be split across multiple shards. Since the Lambda consumer processes shards independently, records from the same logical sequence can arrive out of order, and the increased shard count can also cause higher latency if the consumer is not properly parallelized.

Exam trap

The trap here is that candidates often confuse increased shard count with a need for more concurrency (Option C), but the real issue is that resharding changes the partition-to-shard mapping, which can break ordering guarantees unless the producer explicitly handles the new hash range.

How to eliminate wrong answers

Option A is wrong because S3 versioning controls object overwrites and deletions, not the ordering or latency of records written by Lambda; out-of-order writes are caused by upstream data distribution, not S3 configuration. Option B is wrong because the Lambda function reads from the latest sequence number by default when using the Kinesis trigger, not the oldest; high IteratorAgeSeconds would indicate a slow consumer, not a configuration to read from the oldest record. Option C is wrong because reserved concurrency limits the maximum number of concurrent Lambda executions, but the default unreserved concurrency is usually sufficient for 4 shards; low concurrency would cause throttling (e.g., 429 errors), not out-of-order processing.

7
MCQeasy

A data scientist needs to process a large volume of streaming data from IoT devices and store the results in Amazon S3 for further analysis. Which AWS service is most suitable for ingesting and processing this data in near real-time?

A.Amazon Redshift
B.AWS Glue
C.Amazon Kinesis Data Analytics
D.Amazon EMR
AnswerC

Kinesis Data Analytics processes streaming data in real-time.

Why this answer

Amazon Kinesis Data Analytics is the most suitable service because it can process streaming data from IoT devices in near real-time using SQL or Apache Flink, and directly output the results to Amazon S3. It is designed for continuous, low-latency ingestion and analysis of data streams, making it ideal for this use case.

Exam trap

The trap here is that candidates often confuse AWS Glue (batch ETL) with real-time processing, or assume Amazon Redshift can handle streaming ingestion via its COPY command, but neither supports true near real-time stream processing with sub-second latency.

How to eliminate wrong answers

Option A is wrong because Amazon Redshift is a data warehouse for batch analytics on structured data, not designed for real-time streaming ingestion or processing. Option B is wrong because AWS Glue is a serverless ETL service primarily for batch data preparation and cataloging, not for real-time stream processing. Option D is wrong because Amazon EMR is a big data platform for batch and stream processing using frameworks like Spark or Flink, but it requires more operational overhead and is not as directly optimized for simple near real-time ingestion and S3 output as Kinesis Data Analytics.

8
Multi-Selecthard

A data engineering team uses AWS Glue to run ETL jobs. They notice that jobs are taking longer to complete as data volume grows. They want to optimize performance without increasing cost significantly. Which THREE strategies should they consider?

Select 3 answers
A.Remove partitioning from the output
B.Partition the input data in S3
C.Use Amazon EMR instead of Glue
D.Convert input data to columnar format (e.g., Parquet)
E.Increase the number of DPUs (workers)
AnswersB, D, E

Enables parallel processing.

Why this answer

Partitioning input data in S3 (B) allows AWS Glue to use partition pruning, reading only the relevant subsets of data instead of scanning the entire dataset. This reduces I/O and processing time, directly addressing the performance degradation caused by growing data volumes.

Exam trap

The trap here is that candidates often assume adding more DPUs (E) is the only way to speed up Glue jobs, overlooking that data optimization strategies (partitioning and columnar formats) can yield similar or better performance gains without increasing cost.

9
MCQmedium

A company is using AWS Glue ETL jobs to process data stored in Amazon S3. The jobs currently run sequentially and take too long. The data engineer wants to reduce job duration without rewriting the code. Which action is most effective?

A.Change the underlying EC2 instance type to a compute-optimized instance
B.Increase the number of DPUs (Data Processing Units) for the job
C.Convert the data from CSV to Parquet format
D.Enable job bookmarks to skip already processed data
AnswerB

More DPUs allow parallel execution, reducing job duration.

Why this answer

Increasing the number of DPUs (Data Processing Units) for the AWS Glue ETL job directly allocates more distributed computing resources, enabling parallel execution of the job's stages. This reduces the overall runtime without requiring any code changes, as Glue automatically distributes the workload across the additional DPUs.

Exam trap

The trap here is that candidates often confuse improving data format efficiency (Parquet) or incremental processing (job bookmarks) with solving a sequential execution bottleneck, when the direct solution is to increase parallelism via DPUs.

How to eliminate wrong answers

Option A is wrong because changing the EC2 instance type to compute-optimized only improves per-node performance, but the job still runs sequentially on a single node; without parallelization, the overall duration is limited by the sequential processing bottleneck. Option C is wrong because converting data from CSV to Parquet improves I/O efficiency and reduces data scan volume, but it requires rewriting the ETL code to read Parquet format and does not directly address the sequential execution issue. Option D is wrong because enabling job bookmarks only skips already processed data in incremental runs, which does not reduce the duration of the initial full run or the current sequential processing bottleneck.

10
MCQmedium

A data engineering team needs to ingest streaming data from thousands of IoT devices into Amazon S3 for near-real-time analytics. The solution must handle data that arrives in bursts and must be able to reprocess failed records automatically. Which combination of AWS services should the team use?

A.AWS Glue with Amazon S3
B.Amazon SQS with AWS Lambda
C.Amazon Kinesis Data Streams with AWS Lambda
D.Amazon DynamoDB Streams with AWS Lambda
AnswerC

Kinesis Data Streams can ingest bursty streaming data and retain it for replay; Lambda can process and load to S3.

Why this answer

Amazon Kinesis Data Streams is designed for real-time ingestion of large volumes of streaming data, such as from thousands of IoT devices, and can handle bursty traffic by scaling shards. AWS Lambda can be used as a consumer to process records in near-real-time, and Kinesis Data Streams supports automatic retries and checkpointing, enabling reprocessing of failed records without data loss.

Exam trap

The trap here is that candidates often confuse SQS with Kinesis, but SQS lacks the ordered, replayable stream semantics and high-throughput shard scaling needed for bursty IoT data ingestion and reprocessing.

How to eliminate wrong answers

Option A is wrong because AWS Glue is a batch ETL service, not designed for real-time streaming ingestion or automatic reprocessing of failed records in a streaming context. Option B is wrong because Amazon SQS with AWS Lambda is a message queue pattern that does not natively support ordered, replayable data streams or the high-throughput, bursty ingestion required from thousands of IoT devices; SQS lacks the shard-based parallelism and retention for reprocessing. Option D is wrong because Amazon DynamoDB Streams is designed to capture changes to DynamoDB tables, not for direct ingestion of raw IoT streaming data, and it does not provide the same level of throughput scaling or data retention for bursty IoT workloads.

11
MCQmedium

A company needs to perform complex transformations on large datasets stored in Amazon S3 using Apache Spark. They want to minimize operational overhead. Which AWS service should they use?

A.Amazon EMR
B.Amazon EC2 with manually configured Spark
C.Amazon Athena
D.AWS Glue
AnswerA

EMR provides managed Spark clusters for complex transformations.

Why this answer

Amazon EMR is the correct choice because it is a managed big data platform that natively runs Apache Spark, allowing you to perform complex transformations on large datasets stored in Amazon S3 without provisioning or managing underlying infrastructure. EMR automatically handles cluster lifecycle, scaling, and tuning, minimizing operational overhead while providing full Spark compatibility.

Exam trap

The trap here is that candidates confuse AWS Glue's Spark-based ETL engine with a full-fledged Spark cluster, overlooking that Glue is optimized for simpler, serverless ETL jobs and lacks the fine-grained control and performance tuning capabilities of Amazon EMR for complex transformations.

How to eliminate wrong answers

Option B is wrong because manually configuring Spark on Amazon EC2 requires you to manage cluster setup, software installation, scaling, and fault tolerance, which increases operational overhead rather than minimizing it. Option C is wrong because Amazon Athena is a serverless interactive query service based on Presto, not Apache Spark, and it is designed for SQL-based ad-hoc queries, not for running complex Spark transformations. Option D is wrong because AWS Glue is a serverless ETL service that uses Apache Spark under the hood but abstracts away cluster management; however, it is primarily designed for batch ETL jobs with limited customization and does not provide the same level of control and performance tuning for complex Spark transformations as Amazon EMR.

12
MCQhard

A company uses AWS Glue crawlers to populate the AWS Glue Data Catalog from Amazon S3. The data is partitioned by year/month/day/hour. The crawler runs every hour and adds new partitions. However, the data engineer notices that the crawler is taking longer to run as the number of partitions grows, and sometimes it misses new partitions. What is the most cost-effective and reliable way to address this?

A.Enable the crawler's partition index feature.
B.Manually add new partitions using ALTER TABLE ADD PARTITION in Athena.
C.Use the Athena MSCK REPAIR TABLE command after the crawler runs.
D.Increase the crawler's schedule to run every 30 minutes.
AnswerA

Partition indexes allow the crawler to efficiently discover new partitions without scanning the entire dataset.

Why this answer

Enabling the crawler's partition index feature allows AWS Glue to quickly find new partitions without re-scanning the entire table, reducing runtime and improving reliability. Option B is incorrect because manually adding partitions is error-prone and does not scale. Option C is incorrect because the MSCK REPAIR TABLE command is a manual step that does not automate the process.

Option D is incorrect because increasing the crawler's frequency does not address the underlying issue of scanning overhead and may lead to resource contention.

13
MCQmedium

A company runs a daily ETL job that reads data from Amazon RDS, transforms it using AWS Glue, and writes the results to Amazon S3. The job started failing yesterday with the error: 'Rate exceeded'. What is the most likely cause and solution?

A.The Glue job is using too many DPUs; reduce the number of DPUs
B.The RDS database is overwhelmed by the number of connections; reduce the Glue job's parallelism or increase RDS instance size
C.The S3 bucket has reached its request rate limit; request a limit increase
D.Enable job bookmarks in the Glue job to process only new data
AnswerB

Rate exceeded errors often come from RDS when connection or IO limits are reached.

Why this answer

The 'Rate exceeded' error in an AWS Glue job reading from Amazon RDS typically indicates that the database is being overwhelmed by too many concurrent connections or queries. AWS Glue jobs can spawn multiple executors, each opening connections to RDS, and if the database's max_connections or IOPS limit is exceeded, RDS throttles requests. Reducing the Glue job's parallelism (e.g., setting the number of executors or DPUs lower) or scaling up the RDS instance (e.g., increasing instance size or provisioned IOPS) resolves the issue.

Exam trap

The trap here is that candidates often assume 'Rate exceeded' always refers to AWS API throttling (e.g., S3 or Glue API limits) rather than recognizing it as a database connection limit error, especially when the data source is RDS.

How to eliminate wrong answers

Option A is wrong because reducing DPUs might reduce parallelism but does not directly address the RDS connection limit; the error is from RDS, not from Glue resource limits. Option C is wrong because S3 bucket request rate limits are extremely high (thousands of requests per second) and typically cause 'SlowDown' errors, not 'Rate exceeded'; the error originates from the database, not S3. Option D is wrong because enabling job bookmarks only helps with incremental processing of new data, not with rate limiting; the job is failing due to connection overload, not because it is reprocessing old data.

14
MCQmedium

A company uses Amazon Kinesis Data Streams to collect IoT sensor data. The stream has 4 shards. A consumer application reads from the stream using the Kinesis Client Library (KCL). The application processes records and stores them in Amazon DynamoDB. Recently, the data volume has increased, and the consumer is falling behind. Which action should the team take to increase the processing throughput?

A.Deploy additional consumer instances using the same application name.
B.Increase the write capacity of the DynamoDB table.
C.Increase the data retention period of the stream to 7 days.
D.Increase the number of shards in the Kinesis stream.
AnswerD

More shards provide more read capacity units and allow more parallel consumers.

Why this answer

Increase the number of shards in the Kinesis stream. Kinesis Data Streams throughput is determined by the number of shards; each shard provides 1 MB/s or 1000 records/s for reading. Since the consumer is falling behind, increasing shards directly increases the read throughput.

Option A is incorrect because adding more consumer instances without increasing shards will not improve throughput; KCL ensures each shard is processed by one worker, so extra workers are idle. Option B is incorrect because increasing DynamoDB write capacity may reduce throttling but does not address the root cause of low read throughput from Kinesis. Option C is incorrect because increasing the data retention period does not affect the rate at which data can be read.

15
MCQeasy

A company uses Amazon Redshift for its data warehouse. The data engineering team notices that queries are slow and wants to improve performance without changing the schema. Which action is most likely to improve query performance?

A.Decrease the number of nodes to reduce network overhead.
B.Disable compression on all tables to reduce CPU overhead.
C.Increase the number of nodes in the cluster.
D.Change the distribution style from AUTO to EVEN.
AnswerC

Adding nodes increases parallelism and improves query performance.

Why this answer

Increasing the number of nodes in an Amazon Redshift cluster distributes data and query processing across more compute resources, which directly improves parallel execution and reduces query execution time. This is the most effective way to boost performance without altering the schema, as it scales the cluster's CPU, memory, and I/O capacity.

Exam trap

The trap here is that candidates may confuse 'distribution style' with 'node count' and assume that changing to EVEN will always balance data evenly and improve performance, but in practice EVEN can cause costly data redistribution during joins, whereas scaling out nodes is a safer and more direct performance lever.

How to eliminate wrong answers

Option A is wrong because decreasing the number of nodes reduces the cluster's compute capacity and parallelism, which typically degrades query performance, and network overhead is not the primary bottleneck in Redshift. Option B is wrong because disabling compression on all tables increases the amount of data that must be read from disk and transferred over the network, raising I/O and CPU overhead, which slows queries. Option D is wrong because changing the distribution style from AUTO to EVEN may not improve performance; EVEN distributes rows evenly but can cause excessive data shuffling during joins, whereas AUTO lets Redshift choose the optimal style based on table usage, and forcing EVEN often worsens performance.

16
MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver streaming data to an S3 bucket. The data is JSON and must be partitioned by year, month, and day. The delivery stream is configured with a buffer interval of 60 seconds and buffer size of 5 MB. The data producer sends about 1 MB per second. The data is arriving in S3 but the partitions are not being created as expected. What is the MOST likely reason?

A.The data is encrypted with AWS KMS and Firehose cannot write to encrypted buckets.
B.The delivery stream does not have dynamic partitioning enabled with the appropriate custom prefix.
C.The buffer interval is too short for the data volume, causing incomplete records.
D.The S3 bucket has versioning enabled, which prevents partitioning.
AnswerB

Without dynamic partitioning and the correct prefix, Firehose will not partition the data by year/month/day.

Why this answer

Kinesis Data Firehose requires dynamic partitioning to be explicitly enabled and configured with a custom prefix (e.g., 'year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/') to automatically partition data by year, month, and day. Without this setting, Firehose writes all data to a single S3 prefix, ignoring the desired partition structure.

Exam trap

The trap here is that candidates assume simply setting a prefix with date-like placeholders (e.g., 'data/year=2025/') is enough, but Firehose requires explicit dynamic partitioning to be enabled and the prefix must use the correct !{timestamp:...} syntax for automatic date-based partitioning.

How to eliminate wrong answers

Option A is wrong because Firehose can write to KMS-encrypted S3 buckets when the correct IAM permissions and KMS key policies are in place; encryption does not prevent partitioning. Option C is wrong because a 60-second buffer interval is sufficient for 1 MB/s data (60 MB per interval), and Firehose buffers complete records, not partial ones. Option D is wrong because S3 versioning does not affect Firehose's ability to write partitioned data; versioning simply maintains multiple versions of objects.

17
MCQmedium

A data engineer is troubleshooting an AWS Glue job that reads from an S3 bucket and writes to another S3 bucket. The job fails with an 'Access Denied' error when trying to write to the output bucket. The IAM policy attached to the Glue service role is shown. What is the MOST likely cause of the failure?

A.The user who runs the job does not have S3 permissions
B.The Glue job role does not have permissions to start a job run
C.The output bucket is not listed in the Resource of the IAM policy
D.The S3 bucket policy denies access to the Glue service
AnswerC

The policy only allows PutObject on example-bucket, not the output bucket.

Why this answer

The IAM policy attached to the Glue service role explicitly lists the output bucket in the Resource field. If the output bucket is not listed, the Glue job will receive an 'Access Denied' error when attempting to write to it, because the policy does not grant the necessary s3:PutObject permission for that bucket. This is the most direct cause of the failure.

Exam trap

The trap here is that candidates may overlook the Resource field and assume the error is due to missing actions or user permissions, rather than recognizing that the IAM policy must explicitly list the destination bucket ARN for the write operation to succeed.

How to eliminate wrong answers

Option A is wrong because the user who runs the job is not the entity making the S3 API calls; the Glue service role is. The IAM policy attached to that role is what matters, not the user's permissions. Option B is wrong because the error occurs during the write operation, not during job initiation; the job is already running, so permissions to start a job run are irrelevant.

Option D is wrong because the question states the IAM policy is the issue, and there is no mention of an S3 bucket policy; if a bucket policy denied access, it would be a separate explicit deny, but the most likely cause given the information is the missing resource in the IAM policy.

18
MCQhard

A data scientist needs to run a one-time training job on a 5 TB dataset stored in Amazon S3. The training algorithm requires random access to individual records. Which SageMaker input mode and data format combination would be MOST appropriate?

A.Use Pipe mode with Parquet format
B.Use Pipe mode with RecordIO-Protobuf format
C.Use File mode with RecordIO-Protobuf format
D.Use Pipe mode with CSV format
AnswerC

File mode downloads data to disk, allowing random access; Protobuf is efficient.

Why this answer

File mode loads the entire 5 TB dataset onto the SageMaker instance's local SSD, providing low-latency random access to individual records, which is required by the training algorithm. RecordIO-Protobuf format is optimized for SageMaker's internal data pipeline, enabling efficient deserialization and batching during training. This combination ensures the algorithm can randomly access any record without the sequential streaming constraints of Pipe mode.

Exam trap

Common misconception: Pipe mode is always faster or more efficient. However, because the algorithm requires random access to individual records, Pipe mode's sequential streaming makes it unsuitable. File mode with local SSD storage is necessary for non-sequential access to the 5 TB dataset.

How to eliminate wrong answers

Option A is wrong because Pipe mode streams data sequentially from S3, which does not support random access to individual records; Parquet format, while columnar, is not natively optimized for SageMaker's Pipe mode and would require additional parsing overhead. Option B is wrong because Pipe mode, even with RecordIO-Protobuf format, streams data in order and cannot provide random access; the algorithm would be forced to process records sequentially, violating the requirement. Option D is wrong because Pipe mode with CSV format streams data row by row, preventing random access, and CSV parsing is slower and less efficient than binary formats like RecordIO-Protobuf for SageMaker training jobs.

19
MCQmedium

Refer to the exhibit. A data engineer is creating an IAM policy for an AWS Glue ETL job that reads encrypted objects from an S3 bucket, transforms them, and writes the results back to the same bucket. The bucket uses SSE-KMS encryption with the KMS key specified. The ETL job is failing with an "Access Denied" error when trying to write data. What is the likely cause?

A.The policy is missing the kms:Decrypt permission
B.The policy is missing the s3:PutObjectAcl permission
C.The policy is missing the s3:PutObject permission
D.The policy is missing the kms:Encrypt permission
AnswerD

Writing with SSE-KMS requires kms:Encrypt.

Why this answer

The IAM policy must include the kms:Encrypt permission for the AWS Glue ETL job to write encrypted objects to the S3 bucket using SSE-KMS. The policy likely includes s3:PutObject, kms:Decrypt, and kms:GenerateDataKey, but kms:Encrypt is required for the write operation. Options A, B, and C are incorrect because the necessary permissions (kms:Decrypt, s3:PutObjectAcl, and s3:PutObject) are either already present or not required for writing.

20
MCQmedium

A machine learning team is preparing a large dataset for training. The dataset consists of 10,000 CSV files, each about 100 MB, stored in Amazon S3. The team wants to transform the data using AWS Glue ETL jobs. The transformation involves filtering rows, adding new columns, and joining with a small reference table (100 KB). The team is concerned about job performance and cost. They currently have a Glue job with 10 DPU (Data Processing Units) and it takes about 2 hours to complete. The team wants to reduce the runtime and cost. Which approach should they take?

A.Use Amazon Athena to transform the data.
B.Increase the number of DPUs to 100.
C.Use Amazon EMR with Spot Instances instead of AWS Glue.
D.Convert the CSV files to Parquet format and partition the data by a column.
AnswerD

Parquet reduces I/O and partitioning reduces data scanned.

Why this answer

Converting the CSV files to Parquet format and partitioning the data by a column significantly reduces the amount of data scanned and processed by AWS Glue. Parquet is a columnar storage format that allows Glue to read only the necessary columns, and partitioning enables predicate pushdown to skip irrelevant partitions. This directly reduces I/O and compute requirements, leading to faster job runtime and lower cost without increasing DPU count.

Exam trap

AWS often tests the misconception that simply adding more compute resources (DPUs) will linearly improve performance, ignoring the critical impact of data format and partitioning on I/O and shuffle efficiency.

How to eliminate wrong answers

Option A is wrong because Amazon Athena is an interactive query service, not a data transformation engine; it cannot perform complex ETL transformations like adding columns or joining with a reference table in a single job, and it would still incur costs based on data scanned, which would be high with CSV files. Option B is wrong because increasing DPUs to 100 would linearly increase cost without addressing the root cause of slow performance—inefficient data format and lack of partitioning—and Glue jobs have diminishing returns beyond a certain DPU count due to overhead. Option C is wrong because while Amazon EMR with Spot Instances can be cost-effective, it introduces additional operational complexity (cluster management, provisioning) and does not inherently solve the performance bottleneck caused by CSV format; the team would still need to optimize data format and partitioning.

21
Multi-Selecthard

A company uses Amazon Redshift to run analytics on sales data. The data is loaded daily from S3 using COPY commands. The team notices that the COPY command performance degrades over time due to table bloat. The team needs to maintain query performance and reduce storage costs. Which combination of maintenance operations should the team perform regularly? (Choose THREE.)

Select 3 answers
A.Run the UNLOAD command to export data to S3 and then reload.
B.Change the distribution style of the table to KEY.
C.Run the VACUUM command to reclaim space and re-sort rows.
D.Run a DEEP COPY to recreate the table with optimal physical storage.
E.Run the ANALYZE command to update table statistics.
AnswersC, D, E

VACUUM removes deleted rows and re-sorts data.

Why this answer

The correct maintenance operations are VACUUM, DEEP COPY, and ANALYZE. VACUUM reclaims space from deleted or updated rows and re-sorts data if sort keys are defined, reducing bloat. DEEP COPY recreates the table to eliminate bloat completely by copying data to a new table and renaming.

ANALYZE updates table statistics, which helps the query planner optimize query performance. Option A (UNLOAD) is wrong because it exports data to S3, not a maintenance operation. Option B (changing distribution style) is a schema change that affects data distribution, not a regular maintenance task for bloat removal.

22
MCQmedium

A company is building a data pipeline that ingests data from on-premises databases into Amazon S3 using AWS Database Migration Service (AWS DMS). The company wants to capture continuous changes from the source database and replicate them to S3 in near-real time. Which AWS DMS configuration should the company use?

A.Create a full-load task to copy the existing data
B.Create a full-load plus CDC task with S3 target
C.Create a validation task to compare source and target
D.Create a CDC-only task with S3 as the target endpoint
AnswerD

CDC-only captures and replicates changes in near-real time.

Why this answer

Using a CDC-only task with S3 as the target endpoint replicates continuous changes to S3. Option A is wrong because a full-load task only migrates existing data. Option B is wrong because a full-load plus CDC task includes both, but the requirement is only changes.

Option C is wrong because a validation task is for data validation, not replication.

23
Multi-Selecthard

A company is using AWS Glue ETL jobs to transform data. The jobs are failing due to insufficient memory. The data processing involves complex joins and aggregations. Which THREE actions can improve job performance and reduce memory usage?

Select 3 answers
A.Filter and project data early in the transformation to reduce data volume
B.Decrease the number of DPUs allocated to the job
C.Repartition the data and use bucketing to reduce shuffle size
D.Increase the number of DPUs (workers) allocated to the job
E.Use a single node cluster to avoid shuffle overhead
AnswersA, C, D

Reduces memory footprint.

Why this answer

Filtering and projecting data early in the transformation reduces the volume of data that must be processed in subsequent operations like joins and aggregations. By using pushdown predicates and selecting only necessary columns, you minimize the data shuffled across the cluster, which directly reduces memory pressure and improves job performance in AWS Glue ETL.

Exam trap

The trap here is that candidates often assume reducing resources (Option B) or eliminating parallelism (Option E) will solve memory issues, when in fact these actions exacerbate the problem by increasing the data load per executor or removing the benefits of distributed processing.

24
MCQmedium

A company is using Amazon Kinesis Data Streams to ingest real-time clickstream data. The data is consumed by a Kinesis Data Analytics application that runs SQL queries. The application has been failing intermittently with 'ProvisionedThroughputExceededException' errors. Which action should be taken to resolve this issue?

A.Disable error logging in the Kinesis Data Analytics application.
B.Increase the record size in the Kinesis data stream.
C.Switch from Kinesis Data Analytics to Kinesis Data Firehose.
D.Increase the number of shards in the Kinesis data stream.
AnswerD

Correct: More shards increase read throughput capacity.

Why this answer

The 'ProvisionedThroughputExceededException' error indicates that the Kinesis Data Stream's read or write throughput limits have been exceeded. Increasing the number of shards in the stream directly increases the total provisioned throughput, allowing the Kinesis Data Analytics application to consume data without throttling.

Exam trap

The trap here is that candidates may confuse 'ProvisionedThroughputExceededException' with a data format or service selection issue, rather than recognizing it as a direct capacity scaling problem that requires increasing shard count.

How to eliminate wrong answers

Option A is wrong because disabling error logging does not resolve the underlying throughput issue; it only hides the error messages. Option B is wrong because increasing the record size does not increase the number of records per second or the total throughput; it may actually exacerbate throttling by consuming more capacity per record. Option C is wrong because switching to Kinesis Data Firehose does not address the throughput exception; Firehose is a delivery service that can also be throttled by the same stream limits and does not provide SQL query capabilities.

25
MCQeasy

A data engineering team needs to orchestrate a complex workflow that involves multiple AWS Glue jobs, Lambda functions, and S3 operations. The workflow must run on a schedule and allow monitoring of each step. Which AWS service should they use?

A.Amazon Simple Workflow Service (SWF)
B.AWS Step Functions
C.AWS Data Pipeline
D.Amazon CloudWatch Events
AnswerB

Step Functions provides state machines to orchestrate multi-step workflows.

Why this answer

AWS Step Functions is a serverless orchestration service that lets you coordinate multiple AWS services, including AWS Glue jobs, Lambda functions, and S3 operations, into a state machine workflow. It provides built-in scheduling via Amazon EventBridge (formerly CloudWatch Events) and offers visual monitoring, logging, and error handling for each step, making it the ideal choice for complex, multi-step workflows that require observability.

Exam trap

The trap here is that candidates often confuse AWS Step Functions with Amazon CloudWatch Events (EventBridge) because both can schedule tasks, but they fail to recognize that EventBridge only triggers a single target per rule and cannot orchestrate multi-step workflows with conditional logic, retries, or parallel execution.

How to eliminate wrong answers

Option A is wrong because Amazon Simple Workflow Service (SWF) is a legacy service designed for long-running, human-interactive workflows and does not natively integrate with modern AWS services like Glue or Lambda as seamlessly as Step Functions; it also lacks the built-in scheduling and visual monitoring capabilities required. Option C is wrong because AWS Data Pipeline is primarily a batch data processing and movement service focused on ETL jobs with predefined activities, not a general-purpose workflow orchestrator for arbitrary AWS services like Lambda or Glue jobs; it also does not provide step-level monitoring or retry logic for custom workflows. Option D is wrong because Amazon CloudWatch Events (now part of Amazon EventBridge) is a scheduling and event routing service that can trigger workflows but cannot orchestrate multiple steps with dependencies, error handling, or state management; it only initiates a single target per rule, not a multi-step sequence.

26
MCQmedium

A company uses AWS Glue ETL jobs to process data from multiple sources. The job fails with the error: 'An error occurred while calling o123.pyWriteDynamicFrame. Insufficient memory.' The job runs on a G.1X worker type with 10 workers. What should be changed to resolve this error?

A.Increase the number of workers to 20.
B.Enable the Spark UI to monitor the job.
C.Change the worker type to G.2X.
D.Reduce the number of partitions in the DynamicFrame.
AnswerA

More workers increase parallelism and reduce memory pressure per worker.

Why this answer

The error 'Insufficient memory' in AWS Glue ETL jobs typically indicates that the total memory across all executors is insufficient for the data being processed. Increasing the number of workers from 10 to 20 doubles the total memory and compute capacity available, allowing the job to handle larger datasets without running out of memory. This is the most direct and effective fix for a memory exhaustion error when using the G.1X worker type.

Exam trap

The trap here is that candidates often confuse 'insufficient memory' with a per-worker memory limit and choose to upgrade the worker type (G.2X), but the error is about total cluster memory, which is more effectively addressed by increasing the number of workers.

How to eliminate wrong answers

Option B is wrong because enabling the Spark UI only provides monitoring and debugging capabilities; it does not allocate additional memory or resolve the underlying memory shortage. Option C is wrong because changing the worker type to G.2X doubles the memory per worker (from 16 GB to 32 GB), but the error is about total memory insufficiency, and increasing the number of workers (option A) is a more scalable and cost-effective approach that directly addresses the error without requiring a change in worker type. Option D is wrong because reducing the number of partitions in the DynamicFrame would actually increase the data size per partition, potentially worsening memory pressure on individual executors, not resolving the overall memory shortage.

27
MCQmedium

A data engineering team is building a real-time clickstream analytics pipeline on AWS. They need to ingest millions of events per second from mobile apps and websites, process them with low latency, and store the results in Amazon S3 for downstream analysis. Which combination of AWS services should the team use to minimize operational overhead while meeting these requirements?

A.Use Amazon MQ to ingest streaming data, AWS Lambda to process each message, and save output to Amazon S3.
B.Use Amazon Kinesis Data Streams to ingest data, Amazon EMR to process with Spark Streaming, and save output to Amazon S3.
C.Use Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Analytics for real-time processing, and Amazon Kinesis Data Firehose to deliver results to Amazon S3.
D.Use AWS Glue to ingest data into Amazon RDS, then use AWS Glue ETL jobs to transform and load into Amazon S3.
AnswerC

This combination provides serverless, low-latency ingestion, processing, and delivery with minimal operational overhead.

Why this answer

Amazon Kinesis Data Streams scales to handle millions of events per second with low latency, Kinesis Data Analytics provides real-time processing without managing infrastructure, and Kinesis Data Firehose delivers processed data to Amazon S3 with automatic buffering and compression, minimizing operational overhead. Option A is wrong because Amazon MQ is a managed message broker for standard protocols (e.g., JMS) and does not offer the high-throughput, real-time streaming capabilities required for clickstream analytics. Option B is wrong because, while Kinesis Data Streams works for ingestion, using Amazon EMR with Spark Streaming adds operational overhead for cluster management and scaling, and is less suited for low-latency, serverless processing compared to Kinesis Data Analytics.

Option D is wrong because AWS Glue is a batch ETL service, not designed for real-time ingestion, and Amazon RDS is a relational database that cannot handle the throughput and streaming nature of clickstream data; Glue cannot directly ingest streaming data into RDS in real time.

28
Multi-Selecteasy

A data engineer needs to collect and analyze log data from multiple EC2 instances in real-time. The solution should be serverless and scalable. Which TWO AWS services should be used?

Select 2 answers
A.Amazon Kinesis Data Firehose
B.Amazon EMR
C.Amazon Athena
D.Amazon OpenSearch Service
E.Amazon S3
AnswersA, D

Firehose can ingest streaming data.

Why this answer

Amazon Kinesis Data Firehose is the correct choice because it is a fully managed, serverless service that can capture, transform, and load streaming log data from EC2 instances into destinations like Amazon S3 or Amazon OpenSearch Service in near real-time, with no infrastructure to manage. It automatically scales to handle high-throughput data streams, making it ideal for real-time log analytics.

Exam trap

The trap here is that candidates often choose Amazon S3 alone for storage, forgetting that a real-time ingestion layer like Kinesis Data Firehose is required to collect and stream the data from EC2 instances into a queryable destination.

29
MCQeasy

A data scientist needs to perform exploratory data analysis on a 100 GB CSV file stored in Amazon S3. The data is not sensitive. The scientist wants to use SQL queries to filter and aggregate the data without setting up a server or moving the data. Which service should be used?

A.AWS Glue
B.Amazon EMR
C.Amazon Athena
D.Amazon Redshift Spectrum
AnswerC

Athena is serverless and allows SQL queries on S3 data.

Why this answer

Amazon Athena is the correct choice because it is a serverless, interactive query service that allows you to run standard SQL directly on data stored in Amazon S3 without any infrastructure setup. For a 100 GB CSV file, Athena can handle the query workload efficiently by automatically scaling, and it charges only for the data scanned per query, making it ideal for ad-hoc exploratory analysis without moving or transforming the data.

Exam trap

AWS often tests the distinction between serverless query services (Athena) and services that require provisioning (EMR, Redshift), so the trap here is that candidates may choose Redshift Spectrum thinking it is serverless, but it actually requires an active Redshift cluster.

How to eliminate wrong answers

Option A is wrong because AWS Glue is primarily a serverless ETL (extract, transform, load) service used for data cataloging and preparing data for analytics, not for running interactive SQL queries directly on raw data in S3. Option B is wrong because Amazon EMR requires provisioning and managing a cluster of EC2 instances (even if transient), which contradicts the requirement of not setting up a server. Option D is wrong because Amazon Redshift Spectrum is a feature of Amazon Redshift that allows querying data in S3, but it requires an existing Redshift cluster (a provisioned data warehouse) to be running, which violates the 'no server setup' constraint.

30
MCQhard

An organization stores sensitive customer data in S3. A data pipeline uses AWS Glue to transform the data and load it into Amazon Redshift. The security team requires that data be encrypted at rest in S3 and in transit between S3 and Glue, and between Glue and Redshift. Which configuration meets these requirements?

A.Use S3 client-side encryption, and use VPC Peering between Glue and Redshift.
B.Use S3 default encryption with SSE-KMS, and use Network Load Balancer for Redshift.
C.Enable S3 server-side encryption with SSE-S3, and use SSL for both Glue connections.
D.Enable S3 default encryption with SSE-KMS, use a VPC endpoint for S3, and configure Glue to use SSL for Redshift connection.
AnswerD

Correct because SSE-KMS ensures encryption at rest in S3 with a customer-managed key, a VPC endpoint for S3 uses HTTPS/TLS for in-transit encryption between S3 and Glue, and configuring Glue to use SSL for Redshift encrypts data in transit between Glue and Redshift.

Why this answer

It ensures encryption at rest in S3 via SSE-KMS, encrypts data in transit between S3 and Glue by using a VPC endpoint (which enforces HTTPS/TLS), and encrypts data in transit between Glue and Redshift by configuring SSL for the Redshift connection. SSE-KMS provides envelope encryption with a customer-managed key, while the VPC endpoint and SSL satisfy the in-transit encryption requirements.

Exam trap

The trap here is that candidates often assume VPC Peering alone provides encryption in transit, but it only provides network isolation without encryption, and they may overlook that SSL must be explicitly configured for the Glue-to-Redshift connection.

How to eliminate wrong answers

Option A is wrong because client-side encryption does not guarantee server-side encryption at rest in S3 (the security team requires encryption at rest in S3, which is typically satisfied by server-side encryption), and VPC Peering alone does not enforce encryption in transit between Glue and Redshift (it only provides network connectivity, not TLS/SSL). Option B is wrong because a Network Load Balancer (NLB) for Redshift does not inherently encrypt traffic between Glue and Redshift; NLB operates at Layer 4 and does not terminate TLS unless explicitly configured with a TLS listener, which is not mentioned. Option C is wrong because SSE-S3 encrypts data at rest but does not provide encryption in transit between S3 and Glue (SSL must be explicitly enabled for the Glue connection to S3, and the option does not specify SSL for the S3-to-Glue leg).

31
MCQeasy

A company is using Amazon DynamoDB to store sensor data. The data is exported to Amazon S3 using DynamoDB Streams and AWS Lambda for long-term archival. Recently, the Lambda function has been failing due to 'ProvisionedThroughputExceededException' on the DynamoDB stream. What is the most likely cause?

A.The Lambda function is processing records too slowly, causing the stream to throttle.
B.The DynamoDB stream is disabled.
C.The DynamoDB table's write capacity is too low.
D.The Lambda function does not have enough memory allocated.
AnswerA

Correct: Slow processing can lead to throttling; increasing batch size or concurrency can help.

Why this answer

A is correct because the 'ProvisionedThroughputExceededException' on a DynamoDB stream indicates that the stream's read throughput is being throttled. When a Lambda function processes records too slowly, it cannot keep up with the rate of new records being written to the stream, causing the stream shards to throttle the Lambda consumer. This is a common issue when the Lambda function's processing time per record is high or when the function is invoked with a large batch size that exceeds its processing capacity.

Exam trap

A common misconception in AWS exams is that DynamoDB throttling errors are always related to table write capacity. However, the trap here is that the error is on the stream, not the table, and the root cause is the Lambda consumer's processing speed, not the table's provisioned throughput.

How to eliminate wrong answers

Option B is wrong because if the DynamoDB stream were disabled, the Lambda function would not be triggered at all, and the error would be a different one (e.g., 'ResourceNotFoundException' or no invocation), not a 'ProvisionedThroughputExceededException'. Option C is wrong because the error is on the DynamoDB stream, not on the table's write capacity; the table's write capacity affects writes to the table, but the stream's read throughput is independent and controlled by the stream's shard-level read limits. Option D is wrong because insufficient memory in the Lambda function would cause out-of-memory errors or timeouts, not a 'ProvisionedThroughputExceededException', which is a throttling error from the DynamoDB Streams API.

32
MCQhard

A company uses Amazon Kinesis Data Streams with a shard count of 5. The data producer sends 1000 records per second, each 1 KB in size. The consumer application reads from the stream using the Kinesis Client Library (KCL) and processes records. The consumer is experiencing high latency and falling behind. What is the most effective way to improve consumer throughput?

A.Switch to Kinesis Data Analytics for processing.
B.Use enhanced fan-out to dedicate read throughput to the consumer.
C.Increase the record size to 5 KB.
D.Increase the number of shards in the stream.
AnswerD

Correct: More shards provide more read capacity and allow parallel processing.

Why this answer

The consumer is falling behind because it cannot process records fast enough. The Kinesis Client Library (KCL) typically runs one consumer thread per shard, so with only 5 shards there are only 5 parallel consumers. To improve throughput, the number of shards should be increased to allow more parallel processing.

Options like enhanced fan-out would improve read throughput per consumer but do not increase parallelism; the bottleneck here is processing speed, not read throughput. Increasing shards directly increases the number of consumers and thus overall throughput.

Exam trap

The trap here is that candidates often confuse enhanced fan-out (which provides dedicated throughput per consumer) with solving throughput issues, but fail to realize that with only 5 shards, even dedicated throughput per shard is insufficient for high-volume consumption, making shard scaling the correct solution.

How to eliminate wrong answers

Option A is wrong because Kinesis Data Analytics is a service for running SQL or Apache Flink queries on streaming data in real time; it does not increase read throughput or resolve consumer backpressure. Option B is wrong because enhanced fan-out provides dedicated 2 MB/s read throughput per consumer per shard, but with only 5 shards the total read capacity is still limited to 10 MB/s (5 shards × 2 MB/s), which is insufficient to process 1000 records/second at 1 KB each (1 MB/s write, but consumer processing rate is constrained by shard-level read limits and record processing overhead). Option C is wrong because increasing record size to 5 KB would increase the data volume to 5 MB/s, worsening the consumer's latency and backpressure issue rather than solving it.

33
MCQeasy

A data engineer needs to transfer 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The on-premises network has a 1 Gbps connection to AWS. The transfer must be completed within 10 days. What is the MOST efficient approach?

A.Use AWS Snowball Edge to physically ship the data.
B.Use Amazon S3 Transfer Acceleration to speed up the upload.
C.Use AWS DataSync over the existing network connection.
D.Set up a VPN connection and use multi-part upload directly to S3.
AnswerA

Snowball Edge provides high-speed local transfer and avoids network bottlenecks.

Why this answer

Transferring 50 TB over a 1 Gbps connection would take approximately 5.6 days under ideal conditions (50 TB × 8 bits/byte / 1 Gbps / 86400 seconds/day ≈ 4.63 days), but real-world factors like network congestion, TCP overhead, and protocol inefficiencies typically reduce throughput to 50-70% of line rate, pushing the transfer beyond the 10-day window. AWS Snowball Edge provides a physical shipping alternative that bypasses network limitations entirely, making it the most efficient and reliable method for this volume and deadline.

Exam trap

The trap here is that candidates calculate the theoretical maximum transfer time (50 TB / 1 Gbps ≈ 4.6 days) and conclude it fits within 10 days, ignoring real-world network inefficiencies, protocol overhead, and the fact that sustained throughput rarely exceeds 50-70% of line rate, which pushes the actual time beyond the deadline.

How to eliminate wrong answers

Option B is wrong because S3 Transfer Acceleration only optimizes the last-mile path over AWS edge locations and does not increase the bandwidth of the 1 Gbps on-premises link; it cannot overcome the fundamental throughput bottleneck of the existing connection. Option C is wrong because AWS DataSync, while efficient for incremental transfers, still operates over the same 1 Gbps network and would face the same bandwidth constraint, making it impossible to complete 50 TB within 10 days given real-world overhead. Option D is wrong because setting up a VPN connection adds encryption overhead and further reduces effective throughput, and multi-part upload alone does not increase the available bandwidth; the transfer would still be limited by the 1 Gbps link.

34
MCQmedium

A data engineer needs to transform a large dataset stored in Amazon S3 using Apache Spark. The engineer wants to minimize costs and avoid managing infrastructure. Which AWS service should be used?

A.Amazon Athena
B.Amazon SageMaker
C.Amazon EMR
D.AWS Glue
AnswerD

AWS Glue provides serverless Spark execution, automatically scaling and costing only for active compute time.

Why this answer

AWS Glue is the optimal choice because it provides a serverless Apache Spark environment, fully managed, that allows the engineer to run Spark transformations without provisioning or managing clusters. This meets both requirements: using Apache Spark as specified, and minimizing costs and infrastructure management through its pay‑as‑you‑go, serverless model. Amazon Athena is a SQL query service and does not execute Apache Spark code.

Amazon EMR provides Spark but typically requires cluster management (unless using EMR Serverless, which is not as straightforward as Glue for serverless Spark). Amazon SageMaker is focused on machine learning, not general ETL transformations.

Exam trap

Candidates often select Amazon Athena because it is serverless and low‑cost, but overlook the key requirement: the question explicitly states ‘using Apache Spark.’ Only AWS Glue (serverless) and Amazon EMR (cluster‑based) natively support Spark. Between them, AWS Glue eliminates infrastructure management, making it the correct choice.

How to eliminate wrong answers

Option A is wrong because Amazon Athena is a serverless interactive query service that uses Presto/Trino SQL, not Apache Spark, and is designed for ad-hoc queries, not for running custom Spark transformations. Option B is wrong because Amazon SageMaker is a managed machine learning platform for building, training, and deploying models, not a Spark-based ETL service for general data transformation. Option C is wrong because Amazon EMR, while capable of running Apache Spark, requires the user to provision, configure, and manage EC2 clusters (even with managed scaling), incurring costs for idle cluster time and infrastructure overhead, which contradicts the requirement to minimize costs and avoid managing infrastructure.

35
MCQhard

A data engineer uses the IAM policy above for an AWS Lambda function that processes data in S3 and triggers an AWS Glue job. The Lambda function is unable to start the Glue job. What is the most likely cause?

A.The Glue job name in the resource ARN is misspelled.
B.The policy does not allow s3:PutObject on the bucket.
C.The policy does not include iam:PassRole permission.
D.The policy does not include s3:GetObject on the bucket.
AnswerC

To start a Glue job, Lambda must pass an execution role; iam:PassRole is required.

Why this answer

The Lambda function needs to pass an IAM role to AWS Glue when starting a job, which requires the `iam:PassRole` permission. Without this permission, the `StartJobRun` API call fails even if the Lambda has permissions to invoke Glue. The policy shown lacks this critical permission, making option C the correct answer.

Exam trap

The trap here is that candidates focus on S3 permissions (options B and D) because the Lambda processes S3 data, but the actual failure is the missing IAM permission required to delegate a role to AWS Glue, which is a subtle but critical detail in cross-service orchestration.

How to eliminate wrong answers

Option A is wrong because a misspelled Glue job name would cause a different error (e.g., 'Job not found'), not a permissions failure, and the question states the Lambda is 'unable to start' the job, implying an authorization issue. Option B is wrong because the Lambda function processes data in S3 (likely reading objects) and triggers a Glue job; the error is about starting the Glue job, not writing to S3, and `s3:PutObject` is not required for the Glue job start action. Option D is wrong because `s3:GetObject` is needed for reading data from S3, but the error is specifically about starting the Glue job, not about reading S3 objects; the Lambda may already have that permission or the error would manifest differently.

36
MCQhard

Refer to the exhibit. A data engineer runs an Athena query and gets a failure. What is the most likely cause?

A.The query result location uses a bucket with default encryption enabled.
B.The SQL query syntax is incorrect.
C.The IAM role used does not have permissions to write to S3.
D.The output S3 bucket specified in the query result configuration already exists.
AnswerC

The IAM role used by Athena must have s3:PutObject permission for the output bucket. Lack of permissions is a frequent and common cause of query failures.

Why this answer

The most likely cause for an Athena query failure is that the IAM role used does not have the necessary permissions to write query results to the specified S3 bucket. While syntax errors or encryption settings can cause issues, permissions are a frequent and common cause of failure in practice.

Exam trap

Candidates often assume that the bucket must be new or empty, but Athena can use any existing bucket. The real trap is forgetting that the IAM role must have write permissions to the output bucket.

How to eliminate wrong answers

Option A is wrong because default encryption on the S3 bucket does not cause Athena query failures; Athena can write to encrypted buckets as long as the IAM role has the necessary permissions (e.g., kms:GenerateDataKey). Option B is wrong because the question states the query fails due to the output location, not syntax; Athena provides specific syntax error messages if the SQL is incorrect. Option C is wrong because the IAM role lacking S3 write permissions would produce an access denied error, not a failure related to the output bucket already existing.

37
MCQmedium

A machine learning team is building a real-time inference pipeline using Amazon SageMaker. The input data is located in an S3 bucket, and the team needs to transform the data before inference using a custom Python script. The transformation should run on a serverless infrastructure and must be triggered automatically when new data arrives in S3. Which combination of services should the team use?

A.Use AWS Lambda functions triggered by S3 events to run the transformation, then invoke a SageMaker endpoint.
B.Use AWS Glue jobs triggered by S3 events.
C.Use Amazon SageMaker Processing jobs triggered by S3 events.
D.Use Amazon Kinesis Data Firehose to transform data and deliver to SageMaker.
AnswerA

Lambda provides serverless compute triggered by S3 events, and can call SageMaker endpoints.

Why this answer

AWS Lambda functions can be triggered directly by S3 events (e.g., ObjectCreated) to run a custom Python transformation script on the incoming data, and then invoke a SageMaker endpoint for real-time inference. This combination meets the serverless infrastructure requirement and provides automatic, event-driven processing without managing any servers.

Exam trap

The trap here is that candidates often confuse batch-oriented services like Glue or SageMaker Processing with real-time event-driven needs, or assume Kinesis Firehose can directly invoke a SageMaker endpoint without an intermediate Lambda function.

How to eliminate wrong answers

Option B is wrong because AWS Glue jobs are designed for batch ETL workloads, not real-time inference pipelines; they incur startup latency and are not triggered by S3 events in a serverless, low-latency manner. Option C is wrong because Amazon SageMaker Processing jobs are intended for large-scale, offline data processing and model evaluation, not for real-time, event-driven transformations before inference. Option D is wrong because Amazon Kinesis Data Firehose is a streaming ingestion service that buffers and delivers data, but it cannot directly invoke a SageMaker endpoint for inference; it would require additional Lambda or custom logic to call the endpoint.

38
MCQhard

A company runs a real-time analytics platform that ingests IoT sensor data from millions of devices. The data is sent to Amazon Kinesis Data Streams with 16 shards. A custom Java application using the Kinesis Client Library (KCL) processes the data and writes aggregated results to Amazon DynamoDB. The application runs on a fleet of EC2 instances in an Auto Scaling group. Recently, the team noticed that some records are being processed multiple times, resulting in duplicate entries in DynamoDB. The application uses the DynamoDB PutItem API to write records. The team needs to eliminate duplicates without significantly increasing latency. Which solution should the team implement?

A.Enable DynamoDB auto scaling to increase write capacity and reduce throttling, which causes retries and duplicates.
B.Use DynamoDB TransactWriteItems with a condition check that the record's Kinesis sequence number does not already exist in the table.
C.Place an Amazon SQS FIFO queue between the KCL application and DynamoDB to deduplicate messages.
D.Modify the application to use DynamoDB BatchWriteItem instead of PutItem to reduce the number of write requests.
AnswerB

Using a DynamoDB transaction with a condition check on the Kinesis sequence number ensures that each record is written only once.

Why this answer

Using a DynamoDB transaction with a condition check on the Kinesis sequence number ensures that each record is written only once. Option A is wrong because increasing write capacity does not address duplicate processing; duplicates arise from the KCL consumer processing records multiple times, not from throttling. Option C is wrong because while SQS FIFO provides deduplication at the queue level, it does not guarantee exactly-once processing downstream in DynamoDB; the consumer could still write duplicates if it fails after writing but before deleting the message.

Additionally, adding an extra queue increases latency and complexity. Option D is wrong because BatchWriteItem does not decrease duplicates; it only batches multiple put requests into one API call and still requires idempotency measures.

39
Matchingmedium

Match each AWS security service to its function in ML.

Drag a concept onto its matching description — or click a concept then click the description.

Concepts
Matches

Manage access to AWS resources

Encryption key management

Audit API calls

Isolate network resources

Discover and protect sensitive data

Why these pairings

In ML, AWS KMS handles encryption keys, IAM manages access, CloudTrail provides audit logs. Common confusions arise because these services often work together, but each has a distinct primary function.

40
MCQhard

A financial services company uses Amazon Kinesis Data Streams with 50 shards to ingest real-time stock trade data. The data is consumed by a custom Java application running on Amazon EC2 instances. Recently, the application has been experiencing high latency, and CloudWatch metrics show that the average iterator age is increasing. The application uses the Kinesis Client Library (KCL) with DynamoDB for lease tracking. The EC2 instances are in an Auto Scaling group with a minimum of 2 and maximum of 10 instances, and the current CPU utilization is below 50%. The team wants to reduce latency without increasing costs significantly. What should they do?

A.Increase the provisioned read capacity of the DynamoDB lease table
B.Enable enhanced fan-out on the Kinesis stream
C.Increase the number of shards in the Kinesis stream
D.Increase the maximum size of the Auto Scaling group and set a scaling policy based on iterator age
AnswerD

Increasing the maximum size of the Auto Scaling group and setting a scaling policy based on iterator age allows more EC2 instances to be added dynamically, increasing the number of consumers processing shards in parallel, which directly reduces iterator age without significant cost increase.

Why this answer

Increasing the number of consumers (EC2 instances) by raising the Auto Scaling group maximum and setting a scaling policy based on iterator age allows more shards to be processed concurrently, reducing the iterator age. Option A is incorrect because the DynamoDB lease table is not the bottleneck; lease operations are lightweight and the current read capacity is sufficient. Option B is incorrect because enhanced fan-out is designed for multiple consumer applications to get dedicated throughput, but here there is a single consumer group; it would not reduce latency for the existing consumer and would add cost.

Option C is incorrect because increasing shards would increase the stream's throughput capacity but also cost, whereas the current issue is consumer-side capacity, not stream capacity.

41
MCQmedium

A company is using AWS Glue Data Catalog as the metadata store for their data lake. They have multiple AWS accounts and want to share the catalog across accounts. Which feature should they use?

A.Amazon Athena Federated Query
B.AWS Lake Formation
C.AWS Resource Access Manager (RAM)
D.Amazon S3 Cross-Region Replication
AnswerC

RAM allows sharing Glue Data Catalog across accounts.

Why this answer

AWS Resource Access Manager (RAM) enables you to share AWS Glue Data Catalog databases and tables across multiple AWS accounts without needing to copy metadata. This allows a centralized catalog to be consumed by different accounts for querying and ETL operations, maintaining a single source of truth for the data lake.

Exam trap

The trap here is that candidates often confuse AWS Lake Formation's cross-account access capabilities with the actual sharing mechanism, but Lake Formation relies on AWS RAM to enable the sharing of Data Catalog resources.

How to eliminate wrong answers

Option A is wrong because Amazon Athena Federated Query allows querying data from external sources (e.g., CloudWatch, DynamoDB) using connectors, but it does not share the Glue Data Catalog across accounts. Option B is wrong because AWS Lake Formation provides fine-grained access control and data lake management, but cross-account catalog sharing is implemented via AWS RAM, not directly by Lake Formation (though Lake Formation can use RAM for sharing). Option D is wrong because Amazon S3 Cross-Region Replication replicates objects between S3 buckets in different regions, but it does not share the Glue Data Catalog metadata store across accounts.

42
MCQeasy

A data scientist needs to query a 2 TB dataset stored in Amazon S3 using Amazon Athena. The data is in CSV format and is used for exploratory analysis. Queries are currently slow and expensive. Which action will improve query performance and reduce cost?

A.Convert the data to JSON format to improve compression.
B.Increase the number of workers in the Athena query engine.
C.Convert the data to Parquet format and partition by a commonly filtered column.
D.Create a composite index on the data using Athena's index feature.
AnswerC

Parquet reduces data scanned due to columnar storage, and partitioning limits scan range.

Why this answer

Converting CSV data to Parquet (a columnar storage format) significantly reduces the amount of data scanned by Athena, as only the columns needed for the query are read. Partitioning by a commonly filtered column (e.g., date or region) further limits the data scanned to only relevant partitions, directly reducing both query cost (Athena charges per TB scanned) and query execution time.

Exam trap

The trap here is that candidates may think increasing compute resources (Option B) or adding indexes (Option D) works in Athena as it does in traditional databases, but Athena is serverless and index-free, relying on storage format and partitioning for optimization.

How to eliminate wrong answers

Option A is wrong because JSON is a row-based format that typically results in larger file sizes than CSV (due to repeated keys) and does not support columnar pruning or efficient compression for analytical queries, so it would not improve performance or reduce cost. Option B is wrong because Athena does not have a configurable 'number of workers' parameter; it automatically scales underlying resources based on query complexity, so this option reflects a misunderstanding of Athena's serverless architecture. Option D is wrong because Athena does not support creating composite indexes on data; it relies on partitioning, columnar formats, and data skipping (e.g., with Parquet) to optimize queries, not traditional database indexes.

43
Multi-Selectmedium

Which TWO options are valid ways to reduce the amount of data scanned by Amazon Athena queries, thereby reducing cost?

Select 2 answers
A.Use columnar storage formats like Parquet or ORC
B.Use LIMIT clause in SQL queries
C.Convert data to CSV format
D.Create materialized views in Athena
E.Partition the data by a frequently filtered column
AnswersA, E

Columnar formats allow reading only required columns.

Why this answer

A is correct because columnar storage formats like Parquet and ORC store data in a compressed, column-oriented layout. When Athena queries only a subset of columns, it can skip reading the entire row, drastically reducing the amount of data scanned from disk. This directly lowers the cost, as Athena charges based on the volume of data read per query.

Exam trap

The trap here is that candidates confuse the LIMIT clause with a query optimization technique, not realizing that Athena must still fully scan the underlying data to produce the limited result set, making it ineffective for cost reduction.

44
MCQmedium

An IAM policy attached to a SageMaker notebook role is shown. The data engineer tries to run an Athena query on a table in the 'my_database' Glue database. The query fails with an access denied error. What is the MOST likely cause?

A.The policy does not allow s3:PutObject on the query results location.
B.The policy does not allow glue:GetTable on the specific database.
C.The policy does not allow athena:StartQueryExecution on the Athena workgroup.
D.The policy does not allow s3:ListBucket on the bucket.
AnswerC

Correct because the IAM policy lacks the `athena:StartQueryExecution` action on the specific workgroup. This action is required to initiate an Athena query. Without it, any attempt to run a query will result in an access denied error, regardless of other permissions.

Why this answer

The IAM policy does not include the `athena:StartQueryExecution` action on the Athena workgroup, which is required to submit a query. Even if the role has permissions for Glue and S3, Athena will deny the request if the workgroup-level permission to start queries is missing, resulting in an access denied error.

Exam trap

The trap here is that candidates assume the error is due to missing S3 or Glue permissions because the query accesses those services, but the actual missing permission is the Athena-specific action required to initiate the query execution.

How to eliminate wrong answers

Option A is wrong because the error occurs at query submission, not at result writing; `s3:PutObject` on the query results location is needed only after the query runs successfully. Option B is wrong because the policy includes `glue:GetTable` on `my_database`, so the role can access the table metadata. Option D is wrong because `s3:ListBucket` is not required for Athena to read the table data; Athena uses `s3:GetObject` on the underlying data files, and the error is not about listing the bucket.

45
MCQeasy

A data engineer needs to transfer 50 TB of data from an on-premises HDFS cluster to Amazon S3. The data must be encrypted in transit and at rest. The on-premises network has a 1 Gbps connection to AWS. The transfer must complete within 5 days. Which solution is MOST cost-effective and meets the requirements?

A.Use S3 Transfer Acceleration to upload the data directly from HDFS to S3.
B.Use AWS DataSync with a DataSync agent installed on-premises to transfer the data to S3.
C.Order an AWS Snowball Edge device and copy the data to it, then ship it back.
D.Use AWS Glue to read from HDFS and write to S3 in a continuous ETL job.
AnswerB

DataSync can transfer over network with encryption and is optimized for speed.

Why this answer

(AWS DataSync). With a 1 Gbps connection, the maximum theoretical transfer in 5 days is about 54 TB (1 Gbps = 0.125 GB/s, 0.125 * 86400 * 5 = 54000 GB = 54 TB), so network transfer is feasible within the time limit. AWS DataSync uses a DataSync agent installed on-premises to transfer data from HDFS to S3, encrypting data in transit (TLS) and at rest (S3 server-side encryption).

This is the most cost-effective solution because it avoids the hardware and shipping costs of Snowball Edge (option C). Option A (S3 Transfer Acceleration) does not directly integrate with HDFS and is designed for speeding up uploads over public internet, not for encrypting data from HDFS. Option D (AWS Glue) is an ETL service, not a data transfer solution, and would require additional infrastructure and complexity.

46
MCQmedium

A data engineering team needs to ingest streaming data from thousands of IoT devices into a data lake on Amazon S3 for near-real-time analytics. The data must be partitioned by device ID and timestamp, and the team must minimize data loss during ingestion failures. Which solution is MOST appropriate?

A.Use Amazon Kinesis Data Streams with a Lambda function that writes to S3.
B.Use Amazon Kinesis Data Firehose to write directly to S3 with dynamic partitioning.
C.Use Amazon S3 Transfer Acceleration with direct uploads from devices.
D.Use AWS Lambda to receive data via API Gateway and write to S3.
AnswerB

Firehose provides automatic partitioning, retries, and near-real-time delivery to S3.

Why this answer

Amazon Kinesis Data Firehose with dynamic partitioning is the most appropriate solution because it natively supports partitioning incoming data by device ID and timestamp before writing to S3, and it provides built-in data buffering and retry logic to minimize data loss during ingestion failures. Unlike a Lambda-based approach, Firehose handles large-scale streaming ingestion without requiring custom code for partitioning or error handling, making it ideal for near-real-time analytics on IoT data.

Exam trap

The trap here is that candidates often choose Option A (Lambda with Kinesis Data Streams) because they think it offers more control, but they overlook Firehose’s native dynamic partitioning and managed retry capabilities, which are more reliable and cost-effective for high-volume streaming ingestion to S3.

How to eliminate wrong answers

Option A is wrong because using a Lambda function with Kinesis Data Streams introduces a scaling bottleneck and potential data loss if the Lambda fails or throttles, as Lambda has a maximum invocation concurrency limit and does not natively retry failed records to S3 without custom logic. Option C is wrong because S3 Transfer Acceleration is designed to speed up uploads over long distances, not to ingest streaming data or handle partitioning by device ID and timestamp, and it provides no built-in mechanism for near-real-time analytics or failure recovery. Option D is wrong because using API Gateway with Lambda to receive data directly from devices is not scalable for thousands of IoT devices, introduces latency from HTTP overhead, and lacks native streaming data buffering and retry capabilities, increasing the risk of data loss during failures.

47
MCQmedium

A company uses Amazon EMR to run Spark jobs on a cluster with 10 core nodes of type r5.xlarge. The jobs are I/O intensive and read large amounts of data from S3. The team notices high network throughput but low CPU utilization. Which configuration change would improve job performance at the same cost?

A.Change the instance type to m5.xlarge (general purpose) to balance resources.
B.Increase the number of core nodes to 20.
C.Replace the core nodes with r5d.xlarge instances that have local SSDs.
D.Use spot instances for the core nodes to save cost and reinvest in more nodes.
AnswerC

Local SSDs provide high I/O for caching, reducing network traffic.

Why this answer

R5d instances include local NVMe SSDs. These SSDs can be used for caching intermediate data during Spark jobs, reducing the need to read from and write to S3 over the network. This directly addresses the I/O bottleneck and high network throughput observed, improving job performance.

Option A is incorrect because moving to general-purpose m5 instances does not provide local SSDs and may not improve I/O. Option B is incorrect because doubling the number of core nodes would increase cost significantly without necessarily solving the I/O issue. Option D is incorrect because spot instances reduce cost but do not inherently improve I/O performance; they may even add instability.

48
Multi-Selecteasy

A data engineer is building a data pipeline using AWS Glue. The pipeline reads data from Amazon S3, transforms it, and writes it back to S3 in a different format. The engineer needs to handle schema evolution (new columns added over time). Which TWO features of AWS Glue can help manage schema evolution?

Select 2 answers
A.AWS Glue Data Catalog
B.AWS Glue DynamicFrame
C.AWS Lake Formation
D.Amazon Athena
E.Amazon S3 object tags
AnswersA, B

Data Catalog stores schema and can be updated as schema evolves.

Why this answer

AWS Glue Data Catalog is correct because it stores schema metadata and can be updated automatically or manually to reflect new columns added to source data, enabling schema evolution tracking. AWS Glue DynamicFrame is correct because it provides a flexible, schema-on-read structure that can accommodate varying schemas across records, allowing transformations to handle new columns without breaking the pipeline.

Exam trap

The trap here is that candidates may confuse AWS Lake Formation's data lake governance features with schema evolution capabilities, or assume Athena's query-time schema flexibility is equivalent to Glue's ETL-time schema handling.

49
MCQhard

A data scientist is building a training dataset from data stored in Amazon S3. The data consists of JSON files each containing a 'timestamp' field. The scientist wants to use AWS Glue to catalog the data and enable querying via Amazon Athena. However, Athena queries are returning zero results for time-range filters. What is the most likely cause?

A.The AWS Glue crawler does not have permissions to read the S3 bucket.
B.Athena cannot query nested JSON objects.
C.The JSON files are not in the correct format for Athena.
D.The 'timestamp' field is not defined as a partition column in the Glue table.
AnswerC

Correct. JSON files may have timestamps in an unsupported format or structural issues preventing proper parsing.

Why this answer

Athena supports querying JSON data, but the JSON files must have a schema that Athena can interpret. If the 'timestamp' field is in an unrecognized date/time format or the JSON structure is inconsistent, Athena may fail to parse the data correctly, resulting in zero rows for time-range filters. Option C is correct because the most likely cause is the JSON files not being in a format that Athena can parse properly for timestamp filtering.

Option A is wrong because permission issues would cause an access denied error. Option B is wrong because Athena supports nested JSON. Option D is wrong because even if the timestamp is not a partition column, filtering on it should still return results if the data matches the filter condition.

Exam trap

Candidates may assume that time-range filter failures are always due to missing partition columns, but often the issue is with the data format or timestamp parsing.

50
MCQhard

A company stores sensitive customer data in an S3 bucket. The security team requires that all data be encrypted at rest with a key that is automatically rotated every year. Which solution meets these requirements with the least operational overhead?

A.Use SSE-KMS with a customer-managed key and automatic rotation
B.Use SSE-C (customer-provided keys)
C.Use SSE-S3 (Amazon S3-managed keys)
D.Use SSE-KMS with a customer-managed key and manual rotation
AnswerC

SSE-S3 automatically rotates keys and requires no customer management.

Why this answer

SSE-S3 uses Amazon S3-managed keys (AES-256) that are automatically rotated annually by AWS, meeting the encryption-at-rest and automatic rotation requirements with zero operational overhead. This is the simplest option because no key management or rotation configuration is needed from the customer.

Exam trap

The trap here is that candidates often overthink and choose SSE-KMS with customer-managed keys because they associate 'customer-managed' with more control, but the question explicitly asks for the least operational overhead, which SSE-S3 provides by eliminating all key management tasks.

How to eliminate wrong answers

Option A is wrong because SSE-KMS with a customer-managed key requires you to enable automatic rotation (which is optional and only rotates the backing key, not the data key), adding operational overhead for key policy and permission management. Option B is wrong because SSE-C requires you to provide and manage your own encryption keys, including manual rotation, which incurs significant operational overhead and does not meet the automatic rotation requirement. Option D is wrong because manual rotation of a customer-managed key requires you to create new keys, update applications, and manage key aliases, which is high operational overhead and contradicts the 'least operational overhead' requirement.

51
MCQeasy

A data engineering team needs to ingest streaming data from thousands of IoT devices into Amazon S3 for near-real-time analytics. The data arrives in bursts and must be processed with minimal latency. Which AWS service is most appropriate for the ingestion layer?

A.Amazon Kinesis Data Streams
B.Amazon Kinesis Data Firehose
C.Amazon SQS
D.Amazon S3
AnswerA

Kinesis Data Streams provides low-latency, real-time data ingestion.

Why this answer

Amazon Kinesis Data Streams is the most appropriate ingestion layer because it is designed for real-time, low-latency data ingestion from thousands of sources, such as IoT devices. It can handle bursty traffic by scaling shards dynamically and provides sub-second to second-level latency for data to be available for processing, which meets the minimal latency requirement for near-real-time analytics.

Exam trap

The trap here is that candidates often confuse Kinesis Data Firehose with Kinesis Data Streams, assuming Firehose's direct S3 integration makes it faster, but they overlook the mandatory buffer delay that Firehose imposes, which violates the minimal latency requirement.

How to eliminate wrong answers

Option B (Amazon Kinesis Data Firehose) is wrong because it is a fully managed service for loading streaming data into S3, but it introduces a buffer interval (default 60 seconds or 1 MB) before writing to S3, which adds latency that does not meet the minimal latency requirement. Option C (Amazon SQS) is wrong because it is a message queue service designed for decoupling applications, not for real-time streaming analytics; it does not support sharding or parallel processing of high-throughput streams like IoT data bursts. Option D (Amazon S3) is wrong because it is an object storage service, not an ingestion layer; it cannot directly ingest streaming data in real-time and would require an intermediary service to collect and write data, adding latency and complexity.

52
MCQeasy

A data engineer needs to load data from a MySQL database to Amazon S3 daily. The database is 500 GB and the load window is 2 hours. The data must be extracted without impacting the source database performance. Which AWS service should be used to perform the extraction?

A.AWS Glue ETL job using a JDBC connection to read the full table.
B.AWS Database Migration Service (AWS DMS) with a full-load task to S3.
C.Amazon Athena with the MySQL federated query connector.
D.Amazon EMR with a Spark job reading from MySQL via JDBC.
AnswerB

DMS is designed for minimal impact migration and can load data directly to S3.

Why this answer

(AWS Database Migration Service). AWS DMS is specifically designed for migrating databases to AWS with minimal impact on the source. It can perform a full-load task to extract data from MySQL and write it to S3 efficiently within the 2-hour window.

Option A (AWS Glue ETL) uses JDBC and can cause higher overhead on the source, potentially impacting performance. Option C (Amazon Athena with MySQL federated query) is a query service, not an extraction tool, and may not handle 500 GB efficiently. Option D (Amazon EMR with Spark) is for big data processing and incurs overhead for setup and coordination, making it less suitable for direct daily extraction without impact.

53
MCQhard

A data scientist needs to run ad-hoc SQL queries on a large dataset stored in Amazon S3 (Parquet format, 2 TB). The queries are interactive and require sub-second response times. Which service should they use?

A.Amazon Redshift Spectrum
B.Amazon QuickSight
C.Amazon EMR with Spark SQL
D.Amazon Athena
AnswerD

Athena is serverless and optimized for interactive queries on S3 data.

Why this answer

Amazon Athena is the correct choice because it is a serverless, interactive query service designed for ad-hoc SQL queries on data stored in Amazon S3, with no infrastructure to manage. It natively supports Parquet format and can achieve sub-second response times on 2 TB datasets through columnar projection, predicate pushdown, and data partitioning, making it ideal for this use case.

Exam trap

The trap here is that candidates often confuse Amazon Athena with Amazon Redshift Spectrum, assuming both are equally serverless, but Spectrum still requires a provisioned Redshift cluster, whereas Athena is truly serverless and pay-per-query.

How to eliminate wrong answers

Option A is wrong because Amazon Redshift Spectrum requires an active Redshift cluster to be provisioned and running, adding latency and cost for ad-hoc queries, and it is not serverless like Athena. Option B is wrong because Amazon QuickSight is a business intelligence (BI) visualization tool, not a SQL query engine; it cannot run raw SQL queries directly on S3 data. Option C is wrong because Amazon EMR with Spark SQL involves provisioning and managing a cluster, which introduces startup delays and operational overhead, making it unsuitable for interactive sub-second queries that require instant response.

54
Multi-Selecteasy

Which TWO AWS services can be used to transform data in transit before storing it in Amazon S3? (Choose TWO.)

Select 2 answers
A.AWS Glue
B.Amazon Redshift Spectrum
C.AWS Data Pipeline
D.Amazon Kinesis Data Firehose
E.Amazon Athena
AnswersA, D

Glue can process streaming data with streaming ETL jobs.

Why this answer

AWS Glue is correct because it provides a serverless data integration service that can transform data in transit using its built-in transformation jobs (e.g., PySpark scripts) before writing the results to Amazon S3. This allows you to clean, enrich, or reshape streaming or batch data as it moves through the pipeline.

Exam trap

The trap here is that candidates often confuse query engines (like Athena or Redshift Spectrum) with transformation services, forgetting that in-transit transformation requires processing before the data reaches its final storage location.

55
Multi-Selecthard

A company uses Amazon Athena to query a data lake in Amazon S3. The data is partitioned by year, month, day, and hour. The team notices that queries are slow and expensive. The team wants to improve performance and reduce costs. Which THREE actions should the team take?

Select 3 answers
A.Ensure queries filter on partition columns (year, month, day, hour).
B.Increase the number of partitions by adding a partition for minute.
C.Convert data from CSV to Parquet format.
D.Use CSV format with GZIP compression.
E.Use S3 storage classes like S3 Intelligent-Tiering for cost savings.
AnswersA, C, E

Partition pruning reduces scanned data.

Why this answer

Athena charges based on the amount of data scanned per query. By filtering on partition columns (year, month, day, hour), Athena uses partition pruning to skip reading irrelevant S3 prefixes, drastically reducing the data scanned and thus lowering both cost and query latency.

Exam trap

The trap here is that candidates often think more granular partitions (e.g., minute) always improve performance, but in Athena, excessive partitions increase metadata overhead and can slow down queries due to the overhead of listing many small S3 prefixes.

56
MCQeasy

A Lambda function is triggered by S3 events. The event payload shown in the exhibit is received by the Lambda function. The function is supposed to process the CSV file and load it into DynamoDB. However, the function fails because it cannot read the file. What is the MOST likely cause?

A.The Lambda function lacks DynamoDB write permissions
B.The Lambda function's IAM role does not have s3:GetObject permission
C.The S3 bucket does not exist
D.The S3 event notification is misconfigured
AnswerB

Without read permission, the function cannot access the S3 object.

Why this answer

The Lambda function cannot read the file from S3 because its IAM role does not have the s3:GetObject permission. Option A is wrong because the failure is not due to DynamoDB write permissions; the function fails before writing. Option C is wrong because the S3 bucket exists, as indicated by the event triggering.

Option D is wrong because the event notification is correctly configured to trigger the Lambda function, as evidenced by the function receiving the event.

57
MCQhard

A company is designing a data pipeline to process log files from multiple sources. The logs are written to Amazon S3 every hour. The data is then transformed using AWS Glue ETL jobs and loaded into Amazon Redshift for analysis. The company needs to ensure that the data is available for analysis within 30 minutes of being written to S3. Currently, the Glue job is triggered hourly, but the company wants to reduce the latency. Which solution should the company implement?

A.Increase the frequency of the Glue crawler to run every 5 minutes
B.Use Amazon Redshift Spectrum to query the data directly from S3 without transformation
C.Use Amazon S3 event notifications to invoke an AWS Lambda function that starts the Glue job automatically
D.Reduce the Glue job trigger frequency to every 15 minutes
AnswerC

S3 events trigger Lambda immediately, which starts the Glue job with low latency.

Why this answer

Configuring an S3 event notification to invoke AWS Lambda, which starts the Glue job, allows near-real-time processing within minutes. Option A is wrong because hourly triggers do not reduce latency. Option B is wrong because increasing the crawler frequency does not trigger ETL jobs.

Option D is wrong because Redshift Spectrum does not transform data.

58
MCQeasy

A company needs to ingest real-time clickstream data from thousands of web servers into AWS for near-real-time analytics. The data volume varies and can spike during promotions. Which service should be used to capture and buffer the data before processing?

A.Amazon SQS
B.Amazon Kinesis Data Firehose
C.Amazon Kinesis Data Streams
D.Amazon MQ
AnswerC

Kinesis Data Streams provides a durable buffer for real-time data, enabling multiple consumers.

Why this answer

Amazon Kinesis Data Streams is the correct choice because it is designed for real-time data ingestion and buffering of large streams of data, such as clickstream events from thousands of web servers. It provides durable, low-latency storage (up to 365 days retention) and supports multiple consumers for near-real-time analytics, making it ideal for handling variable and spiky data volumes during promotions.

Exam trap

A common pitfall is confusing the buffering capabilities of Kinesis Data Streams versus Kinesis Data Firehose. Candidates often choose Firehose because they think 'buffer' implies a simple staging area, but Firehose lacks the multi-consumer and replay capabilities required for near-real-time analytics.

How to eliminate wrong answers

Option A is wrong because Amazon SQS is a message queue service for decoupling application components, not designed for high-throughput streaming data ingestion or near-real-time analytics; it lacks the ability to replay data and has a 256 KB message size limit. Option B is wrong because Amazon Kinesis Data Firehose is a fully managed service for loading streaming data into destinations like S3 or Redshift, but it does not provide a buffer for multiple consumers or allow custom processing logic; it is better suited for batch-oriented delivery rather than real-time analytics. Option D is wrong because Amazon MQ is a managed message broker service for Apache ActiveMQ and RabbitMQ, which is intended for traditional messaging patterns (e.g., JMS) and not optimized for high-velocity, real-time clickstream ingestion or replay capabilities.

59
MCQmedium

A data science team is building a real-time fraud detection system. Transactions are streamed via Amazon Kinesis Data Streams, and a Lambda function performs feature engineering and invokes an Amazon SageMaker endpoint for predictions. The team notices that the Lambda function is timing out and causing data loss. Which solution should the team implement to process the stream reliably and at low latency?

A.Use Amazon Kinesis Data Analytics for Apache Flink to consume the stream, perform feature engineering, and invoke the SageMaker endpoint with exactly-once processing.
B.Use the Kinesis Client Library (KCL) to process the stream in an Amazon EC2 instance, and store the predictions in Amazon DynamoDB.
C.Increase the Lambda function timeout to 15 minutes and allocate more memory to reduce processing time.
D.Configure Amazon Kinesis Firehose to deliver the stream to an Amazon S3 bucket, then trigger a Lambda function to process the data in batches.
AnswerA

Kinesis Data Analytics provides stateful stream processing with checkpointing, ensuring no data loss and low-latency integration with SageMaker.

Why this answer

Amazon Kinesis Data Analytics for Apache Flink provides a stateful, low-latency stream processing engine that can consume from Kinesis Data Streams, perform feature engineering in real-time, and invoke SageMaker endpoints with exactly-once processing semantics. This eliminates Lambda timeouts and data loss by using a long-running, scalable application instead of a short-lived function.

Exam trap

The trap here is that candidates often assume increasing Lambda resources (timeout/memory) or moving to a batch-based approach (Firehose/S3) can solve real-time streaming issues, but the exam tests the understanding that stateful, long-running stream processing engines like Flink are required for reliable, low-latency, exactly-once processing in production.

How to eliminate wrong answers

Option B is wrong because using the Kinesis Client Library (KCL) on an EC2 instance requires manual management of scaling, fault tolerance, and checkpointing, and does not natively integrate with SageMaker for low-latency predictions; it also adds operational overhead and potential for data loss if the instance fails. Option C is wrong because increasing the Lambda timeout to 15 minutes and allocating more memory only masks the underlying issue of Lambda's 15-minute maximum execution time and does not address the fundamental problem of stream processing at scale; Lambda is not designed for long-running, stateful stream processing and can still lose data if the function fails or throttles. Option D is wrong because Amazon Kinesis Firehose delivers data in batches to S3, which introduces significant latency (typically minutes) and is not suitable for real-time fraud detection; triggering a Lambda on S3 objects adds further delay and does not provide low-latency, per-record processing.

60
MCQhard

A data engineer has attached the above IAM policy to an IAM role used by an AWS Glue ETL job. The job reads from and writes to 'my-data-bucket'. The job is failing with an Access Denied error. What is the most likely cause?

A.The condition restricts access to a specific IP range that does not include the AWS Glue service IPs.
B.The IAM role needs to have s3:ListBucket permission.
C.The IAM role does not have permission to list the bucket.
D.The resource ARN should include the bucket itself, not just the objects.
AnswerA

The condition requires the request source IP to be in 10.0.0.0/24, but Glue's IPs are different.

Why this answer

The IAM policy includes a condition that restricts access to requests originating from a specific IP address range. AWS Glue ETL jobs run on ephemeral compute resources that use a dynamic pool of IP addresses, which are not guaranteed to fall within any fixed customer-managed IP range. Therefore, the condition causes the Access Denied error because the Glue service IPs are not within the allowed range.

Exam trap

The trap here is that candidates assume the IAM policy is missing a permission like s3:ListBucket, but the real issue is the IP condition that inadvertently blocks the Glue service because its source IPs are not within the specified range.

How to eliminate wrong answers

Option B is wrong because s3:ListBucket is not required for reading or writing objects; the error is Access Denied, not a missing permission, and the policy already includes s3:GetObject and s3:PutObject. Option C is wrong because the policy does not deny s3:ListBucket, and the error is not about listing the bucket; the job fails when trying to access objects, not when listing. Option D is wrong because the resource ARN 'arn:aws:s3:::my-data-bucket/*' correctly specifies objects within the bucket; including the bucket itself would be needed for bucket-level operations like ListBucket, but the job only needs object-level permissions.

61
MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. They notice that the data is delivered in 5-minute intervals even though they set the buffer interval to 60 seconds. What could be the cause?

A.The source Kinesis stream has insufficient shards.
B.The buffer size is set to a value larger than the incoming data rate.
C.The S3 bucket is in a different region.
D.The IAM role does not have permission to write to S3.
AnswerB

If the buffer size is large and data rate low, Firehose waits longer.

Why this answer

B is correct because Kinesis Data Firehose delivers data based on whichever condition is met first: the buffer interval (60 seconds) or the buffer size (e.g., 5 MB). If the incoming data rate is very low, the buffer size threshold may never be reached within 60 seconds, causing Firehose to wait longer—up to the maximum buffer interval of 900 seconds—before delivering. In this case, the data rate is so low that it takes 5 minutes to fill the buffer, overriding the 60-second interval setting.

Exam trap

The trap here is that candidates assume the buffer interval is a strict timer, but Firehose actually uses a 'first-trigger' model where the buffer size can override the interval, causing longer delivery delays than expected.

How to eliminate wrong answers

Option A is wrong because insufficient shards in the source Kinesis stream would cause throttling or data loss, not a delay in delivery intervals; Firehose reads from the stream independently of shard count. Option C is wrong because cross-region S3 buckets do not affect Firehose's buffer interval; they may add latency but not change the delivery frequency. Option D is wrong because if the IAM role lacked S3 write permissions, Firehose would fail to deliver data entirely, not deliver it at 5-minute intervals.

62
MCQmedium

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format, and the company wants to convert it to Parquet for efficient querying. Which configuration should be used?

A.Enable data transformation in Firehose using an AWS Lambda function to convert JSON to Parquet, and set the output format to Parquet.
B.Use an AWS Glue job to convert the JSON files in S3 to Parquet after delivery.
C.Use Amazon Kinesis Data Analytics to convert the stream to Parquet before sending to Firehose.
D.Configure Firehose to deliver data directly to Amazon Redshift, which automatically converts to Parquet.
AnswerA

Firehose can invoke a Lambda function for transformation and write Parquet to S3.

Why this answer

Amazon Kinesis Data Firehose supports data transformation via AWS Lambda, allowing you to convert incoming JSON records to Parquet format before delivery to S3. By enabling a Lambda function to perform the conversion and setting the output format to Parquet, Firehose handles the transformation in-stream, ensuring the data lands in S3 already in the optimized columnar format for efficient querying with services like Amazon Athena or Amazon Redshift Spectrum.

Exam trap

The trap here is that candidates often assume post-processing with AWS Glue (Option B) is the standard approach, overlooking Firehose’s built-in Lambda transformation capability for real-time format conversion, which is more efficient for streaming workloads.

How to eliminate wrong answers

Option B is wrong because running an AWS Glue job after delivery introduces latency and additional cost, as the data must first be stored as JSON in S3 and then reprocessed, which is less efficient than converting in-stream. Option C is wrong because Amazon Kinesis Data Analytics processes data using SQL or Apache Flink but does not natively output to Parquet; it can only output to destinations like Firehose or Lambda, and the conversion to Parquet would still require a downstream transformation. Option D is wrong because Amazon Redshift does not automatically convert data to Parquet; it stores data in its own columnar format, and while it can query Parquet files in S3 via Spectrum, direct delivery to Redshift bypasses the Parquet conversion requirement and does not produce Parquet files in S3.

63
MCQeasy

A data engineer needs to set up a data pipeline that ingests data from an Amazon RDS MySQL database into Amazon S3. The pipeline should run daily and capture incremental changes (inserts, updates, deletes) from the source database. Which AWS service should be used as the data ingestion tool?

A.AWS Database Migration Service (DMS) with continuous change data capture (CDC).
B.Amazon Kinesis Data Streams with a Lambda function.
C.AWS Data Pipeline with a SQL activity.
D.AWS Glue with a scheduled crawler.
AnswerA

Correct: DMS with CDC can capture incremental changes.

Why this answer

AWS DMS with continuous CDC is the correct choice because it is specifically designed to capture incremental changes (inserts, updates, deletes) from a relational database like Amazon RDS MySQL and replicate them to Amazon S3. DMS uses the MySQL binary log (binlog) to track row-level changes in near real-time, making it ideal for daily incremental pipelines. Other services either lack native CDC support or are not optimized for database-to-object-store incremental ingestion.

Exam trap

The trap here is that candidates often confuse AWS Glue crawlers or Data Pipeline SQL activities with CDC capabilities, but neither service natively captures incremental database changes from MySQL binlogs, which is the core requirement for this scenario.

How to eliminate wrong answers

Option B is wrong because Amazon Kinesis Data Streams is a real-time streaming service that requires a custom producer to capture database changes, and while a Lambda function can process records, it does not natively read MySQL binlogs or handle schema evolution for incremental database changes. Option C is wrong because AWS Data Pipeline with a SQL activity is designed for batch ETL jobs using SQL queries against databases, but it cannot capture incremental changes (especially deletes) without complex custom logic and does not support CDC from MySQL binlogs. Option D is wrong because AWS Glue with a scheduled crawler is used for schema discovery and metadata cataloging, not for capturing incremental data changes; a crawler only updates the Data Catalog and does not extract or replicate row-level inserts, updates, or deletes from a source database.

64
MCQeasy

A startup is building a data pipeline that ingests data from multiple sources into an Amazon S3 data lake. The data includes CSV files from legacy systems, JSON from web APIs, and Avro from mobile apps. The data must be transformed into Parquet format and cataloged for querying with Amazon Athena. The pipeline must be serverless and minimize operational overhead. The team has decided to use AWS Glue for ETL and cataloging. However, they are concerned about the cost of running Glue jobs continuously. The data arrives in small batches every 10 minutes. Which approach should the team use to minimize cost while meeting the requirements?

A.Use AWS Lambda functions to transform each file upon arrival and store as Parquet
B.Use Amazon Kinesis Data Firehose to stream data directly into S3 and use Glue to catalog it
C.Use scheduled Glue jobs to process the data every hour, consolidating multiple batches
D.Use a single daily Glue job to process all data at once
AnswerC

Hourly batch processing balances cost and latency.

Why this answer

Using scheduled Glue jobs every hour to process accumulated data reduces the number of job runs and associated costs, while still providing near-real-time processing (within the hour). Option A is wrong because Lambda functions have limited execution time and memory, making them unsuitable for large-scale transformations. Option B is wrong because Kinesis Data Firehose can directly deliver streaming data to S3, but it does not handle all source formats natively (e.g., CSV, Avro) and additional transformation may be needed.

Option D is wrong because a single daily Glue job introduces too much latency for batch arrivals every 10 minutes.

65
MCQeasy

A data engineer needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The company has a 1 Gbps internet connection. Which service would complete the transfer in the shortest time?

A.AWS Snowball
B.Amazon S3 Transfer Acceleration
C.AWS Direct Connect
D.AWS DataSync
AnswerA

Snowball can transfer 50 TB physically in days.

Why this answer

AWS Snowball is the correct choice because transferring 50 TB over a 1 Gbps internet connection would take approximately 5.5 days (50 TB × 1024 GB/TB × 8 bits/byte ÷ 1 Gbps ÷ 86400 seconds/day), assuming full utilization, which is unrealistic due to overhead and contention. Snowball provides a physical appliance that can be loaded with data locally and shipped to AWS, completing the transfer in a few days including shipping time, making it faster than any network-based method for this volume.

Exam trap

The trap here is that candidates underestimate the time required for large data transfers over a 1 Gbps link and overestimate the speed improvements of network acceleration services like S3 Transfer Acceleration or DataSync, which cannot overcome the fundamental bandwidth limitation.

How to eliminate wrong answers

Option B (Amazon S3 Transfer Acceleration) is wrong because it only optimizes the network path using AWS edge locations and does not increase bandwidth beyond the 1 Gbps internet connection, so the transfer would still take days. Option C (AWS Direct Connect) is wrong because even with a dedicated 1 Gbps connection, the theoretical minimum transfer time is still ~5.5 days, and provisioning a Direct Connect circuit typically takes weeks, adding significant delay. Option D (AWS DataSync) is wrong because it is a software agent that transfers data over the network and is still limited by the 1 Gbps internet bandwidth, offering no speed advantage over a raw network transfer for this volume.

66
MCQhard

A company uses AWS Glue to run ETL jobs on a daily schedule. The jobs are failing intermittently with 'OutOfMemory' errors. The data volume has grown 5x over the past month. Which is the MOST cost-effective fix?

A.Increase the number of partitions in the source S3 data
B.Increase the number of DPUs for the Glue job
C.Reduce the data volume by sampling
D.Switch from AWS Glue to Amazon EMR
AnswerB

More DPUs provide more memory and parallelism.

Why this answer

The 'OutOfMemory' errors in AWS Glue are caused by insufficient compute resources (DPUs) to process the 5x increased data volume. Increasing the number of DPUs allocates more memory and processing capacity to the job, directly addressing the memory shortage without changing the data or architecture. This is the most cost-effective fix because it scales resources incrementally rather than switching to a more expensive service like EMR.

Exam trap

The trap here is that candidates may assume the issue is data partitioning (Option A) or that a more powerful service like EMR (Option D) is always better, when in fact the simplest and most cost-effective solution is to adjust the Glue job's DPU allocation to match the increased workload.

How to eliminate wrong answers

Option A is wrong because increasing partitions in source S3 data does not directly increase the memory available to the Glue job; it may improve parallelism but does not resolve the OutOfMemory error caused by insufficient DPU allocation. Option C is wrong because reducing data volume by sampling would discard data and compromise the completeness of the ETL output, which is not a valid production fix for growing data. Option D is wrong because switching from AWS Glue to Amazon EMR is a more complex and costly solution that introduces cluster management overhead; it is not the most cost-effective fix when simply increasing DPUs in Glue can resolve the issue.

67
MCQeasy

A company wants to analyze historical data stored in Amazon S3 using Amazon Athena. The data is in CSV format and is partitioned by date. Which action will provide the best query performance and cost optimization?

A.Use AWS Glue to compress the CSV files with gzip
B.Create an S3 event notification to trigger a Lambda function that warms up Athena
C.Keep CSV format but ensure partitions are in the format year=YYYY/month=MM/day=DD
D.Convert the data to Parquet format and use the existing partition structure
AnswerD

Parquet is columnar and compressed, reducing scanned data and improving performance.

Why this answer

Converting data to Parquet and partitioning provides the best performance and cost savings because Athena can use predicate pushdown and column pruning, scanning less data. Option A (using Glue to gzip compress) still uses CSV which requires full scan. Option B (S3 event notification to warm up Athena) is not relevant because Athena caches results but doesn't need warming.

Option C (only partitioning) helps but CSV is still row-based and less efficient than Parquet.

68
Multi-Selecthard

A data engineer needs to set up a data lake on S3 that supports both batch and streaming ingestion. The data must be queryable by Athena, Redshift Spectrum, and EMR. Which TWO configurations are essential? (Choose two.)

Select 2 answers
A.Store data in columnar formats like Parquet or ORC.
B.Use the AWS Glue Data Catalog as a central metadata repository.
C.Enable S3 Select on the target buckets.
D.Enable S3 versioning on all buckets.
E.Set up Kinesis Data Firehose for streaming ingestion.
AnswersA, B

Columnar formats improve query performance and reduce scan costs for Athena and Redshift Spectrum.

Why this answer

Columnar formats like Parquet and ORC are optimized for analytical queries, reducing I/O by reading only the necessary columns. This is essential for Athena, Redshift Spectrum, and EMR, which all benefit from the efficient compression and predicate pushdown capabilities of these formats, enabling faster query performance and lower costs.

Exam trap

The trap here is that candidates may confuse the ingestion mechanism (e.g., Kinesis Data Firehose) with the essential data lake configuration, or assume that S3 Select is required for queryability, when in fact the core requirements are a unified metadata catalog and an efficient storage format.

69
Multi-Selectmedium

A data engineer is designing a data ingestion pipeline that will receive up to 5 GB of data per hour from thousands of IoT devices. The data must be stored in Amazon S3 and analyzed in near real-time. Which TWO services should be used together to meet these requirements? (Choose TWO.)

Select 2 answers
A.AWS Lambda
B.Amazon Kinesis Data Analytics
C.Amazon Athena
D.Amazon Kinesis Data Firehose
E.Amazon Simple Queue Service (Amazon SQS)
AnswersB, D

Kinesis Data Analytics can run SQL queries on streaming data for near real-time analysis.

Why this answer

Amazon Kinesis Data Firehose is the correct service because it can reliably ingest streaming data from thousands of IoT devices at up to 5 GB per hour, automatically buffer, compress, and deliver the data to Amazon S3 with near-real-time latency (typically 60 seconds). Amazon Kinesis Data Analytics is correct because it enables real-time SQL-based analytics on the streaming data before it is stored in S3, allowing the data engineer to derive insights as data arrives without needing to query the S3 bucket after storage.

Exam trap

The trap here is that candidates often confuse Amazon Kinesis Data Firehose with Amazon Kinesis Data Streams, or mistakenly think Amazon Athena can ingest streaming data because it can query S3 in near-real-time, but Athena is purely a query engine and cannot replace the ingestion and streaming analytics components required for this pipeline.

70
Multi-Selectmedium

A company is designing a data pipeline to ingest data from multiple sources into an Amazon S3 data lake. The data must be encrypted at rest and in transit. Which TWO actions should be taken to meet these requirements?

Select 2 answers
A.Enable Server-Side Encryption on the S3 bucket
B.Enable S3 Transfer Acceleration
C.Enforce HTTPS for all S3 API requests using bucket policy
D.Use client-side encryption before uploading
E.Use S3 VPC Endpoint
AnswersA, C

Encrypts objects at rest.

Why this answer

Enabling Server-Side Encryption (SSE-S3 or SSE-KMS) on the S3 bucket automatically encrypts data at rest when written to disk, using AES-256 encryption. This meets the requirement for encryption at rest without any client-side changes.

Exam trap

The trap here is that candidates often confuse S3 Transfer Acceleration or VPC Endpoints with encryption features, or mistakenly think client-side encryption is required alongside server-side encryption, when the simplest AWS-native pair is SSE + HTTPS enforcement.

71
MCQeasy

A data engineer needs to move 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The network bandwidth is limited to 100 Mbps. Which AWS service should be used to transfer the data most efficiently?

A.Amazon S3 Transfer Acceleration to speed up the transfer.
B.AWS Snowball Edge device to physically ship the data.
C.AWS Direct Connect to establish a dedicated network connection.
D.AWS Site-to-Site VPN to connect and copy data.
AnswerB

Snowball bypasses network limitations by shipping data physically.

Why this answer

Given 50 TB of data and a 100 Mbps network link, the theoretical minimum transfer time over the network is over 46 days (50 TB * 8 / 100 Mbps ≈ 4,000,000 seconds ≈ 46.3 days), not accounting for protocol overhead, retransmissions, or contention. AWS Snowball Edge is a physical appliance that bypasses the network bottleneck entirely, allowing you to copy data locally and ship it to AWS, making it the most efficient option for this volume over a constrained link.

Exam trap

The trap here is that candidates often overestimate the impact of acceleration or dedicated connections on large data volumes over low-bandwidth links, failing to calculate that even with perfect efficiency, a 100 Mbps link cannot transfer 50 TB in a reasonable time frame, making physical shipping the only viable option.

How to eliminate wrong answers

Option A is wrong because Amazon S3 Transfer Acceleration uses optimized network paths and edge locations but still relies on the same 100 Mbps internet link, so it cannot overcome the fundamental bandwidth limitation; the transfer would still take weeks. Option C is wrong because AWS Direct Connect provides a dedicated network connection with consistent bandwidth, but it does not increase the available 100 Mbps capacity—it would still require the same multi-week transfer time and involves significant setup cost and lead time. Option D is wrong because AWS Site-to-Site VPN encrypts traffic over the public internet but does not improve throughput; it adds overhead and still depends on the same 100 Mbps bottleneck, making it even slower than a direct transfer.

72
MCQmedium

A machine learning team needs to preprocess large volumes of clickstream data stored in Amazon S3 before training a model. The preprocessing includes data cleaning, feature engineering, and normalization. The team wants to use a serverless solution that minimizes operational overhead. Which combination of services should the team use?

A.Amazon SageMaker Notebooks with custom Python scripts.
B.Amazon EMR with Spark clusters.
C.AWS Glue ETL jobs reading from and writing to S3.
D.Amazon Athena with SQL queries.
AnswerC

AWS Glue is serverless and designed for ETL on data lakes.

Why this answer

AWS Glue ETL jobs are a serverless solution that automatically provisions and scales the underlying compute resources, making them ideal for preprocessing large volumes of clickstream data stored in S3. Glue can read directly from S3, perform data cleaning, feature engineering, and normalization using PySpark or Scala, and write the transformed data back to S3, all without managing any infrastructure. This minimizes operational overhead while handling the required preprocessing tasks at scale.

Exam trap

The trap here is that candidates often confuse 'serverless' with 'managed' — EMR is managed but not serverless, while Athena is serverless but lacks the flexibility for complex ETL transformations, leading them to incorrectly choose Athena or EMR.

How to eliminate wrong answers

Option A is wrong because Amazon SageMaker Notebooks run on EC2 instances that require manual provisioning, scaling, and lifecycle management, which introduces operational overhead and is not serverless. Option B is wrong because Amazon EMR with Spark clusters requires you to manage cluster provisioning, scaling, and termination, adding significant operational overhead compared to a serverless solution. Option D is wrong because Amazon Athena is primarily an interactive query service for ad-hoc analysis using SQL, not designed for complex ETL pipelines involving custom feature engineering and normalization logic that go beyond SQL capabilities.

73
MCQmedium

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The application reads from a Kinesis data stream and writes results to a sink. The application is failing with an 'OutOfMemoryError'. The application has parallelism set to 4 and uses 1 Kinesis Processing Unit (KPU). What is the MOST likely cause and solution?

A.The application is using too many operators; reduce parallelism to 2.
B.The heap memory per operator is too low; increase parallelism to 8.
C.The checkpoint interval is too short; increase it to 5 minutes.
D.The buffer timeout is too high; reduce it to 50 ms.
AnswerB

Higher parallelism allocates more total memory across tasks.

Why this answer

With parallelism set to 4 but only 1 KPU, each operator slot receives a fraction of the available heap memory, leading to an OutOfMemoryError. Increasing parallelism to 8 distributes the workload across more slots, but more importantly, it forces Kinesis Data Analytics to allocate additional KPUs (each KPU provides 4 GB of memory), thereby increasing the total heap memory available to the application.

Exam trap

The trap here is that candidates assume increasing parallelism always reduces per-operator memory, but in Kinesis Data Analytics, parallelism is tied to KPU allocation, so increasing parallelism can actually increase total memory by provisioning more KPUs.

How to eliminate wrong answers

Option A is wrong because reducing parallelism would further decrease the number of operator slots, concentrating memory usage and worsening the OutOfMemoryError. Option C is wrong because a short checkpoint interval can cause backpressure and increased memory usage, but the primary issue here is insufficient heap memory per operator, not checkpoint timing. Option D is wrong because buffer timeout affects latency and batching behavior, not heap memory allocation; reducing it would increase the number of small records processed, potentially increasing memory pressure.

74
MCQmedium

A company is migrating its on-premises Hadoop cluster to AWS. They have a large amount of historical data stored in HDFS. Which approach is the most efficient for transferring this data to Amazon S3?

A.Use AWS Snowball Edge devices.
B.Use AWS Direct Connect.
C.Use AWS DataSync over the internet.
D.Use S3 Transfer Acceleration.
AnswerA

Snowball is designed for large offline data transfers.

Why this answer

AWS Snowball Edge is ideal for large data transfers when network bandwidth is limited. AWS DataSync is for network transfers, but slower for huge datasets. S3 Transfer Acceleration improves speed but still network.

Direct Connect is network-based.

75
Multi-Selecteasy

A company stores IoT sensor data in Amazon S3 and uses Amazon Athena for ad-hoc queries. The data is partitioned by date, but queries are still slow and expensive. Which TWO actions can improve query performance and reduce cost? (Choose TWO.)

Select 2 answers
A.Use S3 lifecycle policies to compact small files into larger ones
B.Convert the data from CSV to Parquet format
C.Disable server-side encryption on the S3 bucket
D.Use AWS Glue instead of Athena for querying
E.Increase the number of partitions to hour-level granularity
AnswersA, B

Fewer, larger files reduce the overhead of opening many files in Athena.

Why this answer

Compacts small files into larger ones, reducing the number of objects and minimizing metadata overhead, which improves query performance. Option B converts data from CSV to Parquet, a columnar format that reduces the amount of data scanned by Athena, lowering cost and speeding up queries. Option C (disabling encryption) does not affect performance and is not recommended.

Option D (using Glue) is a different service and not a direct improvement for Athena queries. Option E (increasing partitions to hour-level) can create many small files, degrading performance.

Page 1 of 5 · 350 questions totalNext →

Ready to test yourself?

Try a timed practice session using only Data Engineering questions.