MLS-C01 Data Engineering Practice Test 3 — 15 Questions

Question 1

A data engineer needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The company has a 1 Gbps internet connection. Which service would complete the transfer in the shortest time?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS Snowball. AWS Snowball is the correct choice because transferring 50 TB over a 1 Gbps internet connection would take approximately 5.5 days (50 TB × 1024 GB/TB × 8 bits/byte ÷ 1 Gbps ÷ 86400 seconds/day), assuming full utilization, which is unrealistic due to overhead and contention. Snowball provides a physical appliance that can be loaded with data locally and shipped to AWS, completing the transfer in a few days including shipping time, making it faster than any network-based method for this volume.

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Direct Connect

Answer

AWS DataSync

Question 2

A company is building a data lake on Amazon S3. They need to enforce encryption at rest for all objects. Which combination of actions will achieve this? (Assume the bucket is versioned.)

Accepted Answer

Enable S3 default encryption and set a bucket policy to deny PutObject without encryption headers. Combining S3 default encryption with a bucket policy that denies PutObject requests lacking encryption headers ensures that every object stored in the bucket is encrypted at rest, even if the PutObject call does not include encryption parameters. Default encryption alone can be overridden by a client that explicitly sets encryption headers, but the bucket policy enforces encryption for all uploads, closing that loophole. This dual approach guarantees compliance with encryption-at-rest requirements for a versioned bucket.

Answer

Use AWS KMS with automatic key rotation

Answer

Enable S3 default encryption only

Answer

Enable S3 Block Public Access

Question 3

A data engineering team uses AWS Glue to run ETL jobs. They notice that jobs are taking longer to complete as data volume grows. They want to optimize performance without increasing cost significantly. Which THREE strategies should they consider?

Accepted Answer

Partition the input data in S3. Partitioning input data in S3 (B) allows AWS Glue to use partition pruning, reading only the relevant subsets of data instead of scanning the entire dataset. This reduces I/O and processing time, directly addressing the performance degradation caused by growing data volumes.

Answer

Remove partitioning from the output

Answer

Use Amazon EMR instead of Glue

Question 4

An IAM policy is attached to a group. A user in the group tries to read the object s3://data-lake-bucket/sensitive/file.txt from an IP address 192.168.1.1. What will happen?

Accepted Answer

The request is denied because of the Deny statement. IAM evaluates all applicable policies using an explicit-deny-wins model. When a Deny statement matches the request (here, the source IP 192.168.1.1 falls within the denied range), that Deny overrides any Allow, regardless of how permissive the Allow statement is. The user is a member of the group to which the policy is attached, so the policy applies and the s3:GetObject request is denied.

Answer

The request is allowed because the Allow statement grants s3:GetObject

Answer

The request is allowed because the Deny condition does not match

Answer

The request is denied because the policy has no explicit Allow for the sensitive prefix

Question 5

A data engineer needs to ingest streaming data from an on-premises Kafka cluster into Amazon S3 with minimal operational overhead. Which AWS service should be used to stream the data into S3 without managing servers?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service that can directly ingest streaming data from an on-premises Kafka cluster (via a Kinesis Data Firehose HTTP endpoint or a custom producer) and deliver it to Amazon S3 without requiring any server management. It handles scaling, buffering, and compression automatically, minimizing operational overhead.

Answer

Amazon Kinesis Data Streams

Answer

AWS Glue

Answer

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Question 6

A data science team uses Amazon SageMaker to train models on a dataset stored in Amazon S3. The dataset is 2 TB and is accessed by multiple training jobs. The team notices that training jobs are slow due to high S3 GET request latency. Which solution would provide the fastest and most cost-effective data access?

Accepted Answer

Mount an Amazon FSx for Lustre file system integrated with the S3 bucket. Amazon FSx for Lustre provides a high-performance, POSIX-compliant file system that can be directly linked to an S3 bucket, allowing training instances to access data with sub-millisecond latency instead of S3 GET request latency. This integration enables data to be read from the Lustre file system at up to hundreds of gigabytes per second of throughput, which is significantly faster than reading directly from S3, and it is cost-effective because you only pay for the storage and throughput you provision during training.

Answer

Place all training instances in a Cluster Placement Group

Answer

Enable S3 Transfer Acceleration on the bucket

Answer

Use Elastic Fabric Adapter (EFA) for training instances

Question 7

A company runs a daily ETL job that reads data from Amazon RDS, transforms it using AWS Glue, and writes the results to Amazon S3. The job started failing yesterday with the error: 'Rate exceeded'. What is the most likely cause and solution?

Accepted Answer

The RDS database is overwhelmed by the number of connections; reduce the Glue job's parallelism or increase RDS instance size. The 'Rate exceeded' error in an AWS Glue job reading from Amazon RDS typically indicates that the database is being overwhelmed by too many concurrent connections or queries. AWS Glue jobs can spawn multiple executors, each opening connections to RDS, and if the database's max_connections or IOPS limit is exceeded, RDS throttles requests. Reducing the Glue job's parallelism (e.g., setting the number of executors or DPUs lower) or scaling up the RDS instance (e.g., increasing instance size or provisioned IOPS) resolves the issue.

Answer

The Glue job is using too many DPUs; reduce the number of DPUs

Answer

The S3 bucket has reached its request rate limit; request a limit increase

Answer

Enable job bookmarks in the Glue job to process only new data

Question 8

A company uses AWS Lake Formation to manage permissions on a data lake stored in Amazon S3. A data analyst tries to query a table using Amazon Athena but receives an 'Access Denied' error. The analyst has SELECT permission on the table in Lake Formation. What is the most likely cause?

Accepted Answer

The S3 bucket is not registered with Lake Formation. When Lake Formation manages permissions on a data lake, it requires that the underlying S3 bucket be registered with Lake Formation. If the bucket is not registered, Lake Formation cannot enforce its fine-grained access controls, and Athena will fail with an 'Access Denied' error even if the analyst has SELECT permission on the table in Lake Formation. Registering the bucket allows Lake Formation to integrate with S3 and apply its permission model.

Answer

The S3 bucket is encrypted with a KMS key that the analyst does not have access to

Answer

The table does not have any partitions defined

Answer

The IAM role used by Athena does not have lakeformation:GetDataAccess permission

Question 9

A data pipeline uses Amazon Kinesis Data Streams with a Lambda consumer to process clickstream data. The Lambda function sometimes times out because of spikes in traffic. The team wants to buffer the data before processing to handle spikes. Which approach is most effective?

Accepted Answer

Send the data to an Amazon SQS queue and have Lambda poll from it. Amazon SQS can act as a buffer to handle traffic spikes. By sending data from Kinesis Data Streams to an SQS queue, Lambda can poll from the queue in batches, smoothing out the load and preventing timeouts. This decouples the ingestion from processing, allowing Lambda to process data at its own pace.

Answer

Increase the number of shards in the Kinesis stream

Answer

Use Kinesis Data Firehose as an intermediary that buffers data and delivers to Lambda in batches

Answer

Increase the Lambda function timeout to 15 minutes

Question 10

A company uses Amazon EMR to run Spark jobs on a large dataset stored in Amazon S3. The jobs are failing with 'OutOfMemoryError' in the executors. The data is not skewed. Which configuration change will most likely resolve the issue?

Accepted Answer

Increase the spark.executor.memoryOverhead setting. When Spark executors run out of memory during shuffle operations, the `spark.executor.memoryOverhead` setting is often the culprit. This parameter allocates off-heap memory for JVM overhead, internal metadata, and shuffle buffers. Increasing it provides more room for these operations without reducing the executor heap, directly addressing OutOfMemoryError in non-skewed data scenarios.

Answer

Enable Kryo serialization

Answer

Decrease the number of shuffle partitions

Answer

Increase the number of executor cores

Question 11

A data engineer is designing a data ingestion pipeline that will receive up to 5 GB of data per hour from thousands of IoT devices. The data must be stored in Amazon S3 and analyzed in near real-time. Which TWO services should be used together to meet these requirements? (Choose TWO.)

Accepted Answer

Amazon Kinesis Data Analytics. Amazon Kinesis Data Firehose is the correct service because it can reliably ingest streaming data from thousands of IoT devices at up to 5 GB per hour, automatically buffer, compress, and deliver the data to Amazon S3 with near-real-time latency (typically 60 seconds). Amazon Kinesis Data Analytics is correct because it enables real-time SQL-based analytics on the streaming data before it is stored in S3, allowing the data engineer to derive insights as data arrives without needing to query the S3 bucket after storage.

Answer

AWS Lambda

Answer

Amazon Athena

Answer

Amazon Simple Queue Service (Amazon SQS)

Question 12

A company needs to transfer 10 TB of data from an on-premises data center to Amazon S3. The network bandwidth is limited to 100 Mbps, and the transfer must complete within 5 days. Which TWO options are viable? (Choose TWO.)

Accepted Answer

Use AWS Snowball Edge device to ship the data. AWS Snowball Edge is a physical data transport solution designed for large-scale data transfers over slow or unreliable networks. With 10 TB of data and a 100 Mbps link, the theoretical transfer time is over 9 days, exceeding the 5-day window. Snowball Edge bypasses the network bottleneck entirely by shipping the data via courier. Option E is also correct because AWS Direct Connect provides a dedicated network connection that can offer higher and more consistent bandwidth than the existing internet connection. If provisioned with sufficient capacity (e.g., 1 Gbps), the transfer can complete within the 5-day window. Options A and B are not viable because they cannot overcome the physical bandwidth limitation of 100 Mbps. Option D is not viable because DataSync still relies on the existing internet connection, which is insufficient.

Answer

Use S3 Transfer Acceleration to speed up the transfer

Answer

Use S3 Multipart Upload to upload files in parallel

Answer

Use AWS DataSync over the existing internet connection

Question 13

A company is using Amazon Kinesis Data Analytics for Apache Flink to process real-time data. The data source is a Kinesis data stream, and the output is written to an S3 bucket. Recently, the processing latency has increased significantly. The team suspects that the Flink application is encountering backpressure. Which metric should the team monitor to confirm backpressure?

Accepted Answer

busyTimeMsPerSecond. The correct metric to confirm backpressure in a Flink application is `busyTimeMsPerSecond`. This metric measures the percentage of time a task is actively processing data versus waiting for input. A high `busyTimeMsPerSecond` value (close to 1000ms) indicates that the task is fully utilized and cannot keep up with the incoming data rate, which is the direct symptom of backpressure. Other metrics like `currentLowWatermark` relate to event time progress, not backpressure.

Answer

currentLowWatermark

Answer

numberOfFailedCheckpoints

Answer

numRecordsInPerSecond

Question 14

A company is building a data pipeline to process sensitive customer data. The pipeline uses AWS Glue for ETL and stores results in Amazon S3. The security team requires that all data be encrypted at rest in S3 using customer-managed AWS KMS keys. Additionally, the Glue job must be able to write encrypted data to S3. What should the data engineer do to meet these requirements?

Accepted Answer

Attach a policy to the Glue job's IAM role that includes kms:GenerateDataKey and kms:Decrypt actions for the KMS key.. AWS Glue jobs use an IAM role to interact with AWS services. To write encrypted data to S3 using a customer-managed AWS KMS key, the IAM role must have permissions for `kms:GenerateDataKey` (to request a data key for encryption) and `kms:Decrypt` (to decrypt the data key when reading or writing). This allows the Glue job to encrypt objects at rest in S3 with the specified KMS key, meeting the security team's requirement.

Answer

Use S3 server-side encryption with customer-provided keys (SSE-C).

Answer

Use S3 server-side encryption with SSE-S3, which is enabled by default.

Answer

Configure an S3 bucket policy to enforce encryption and attach it to the Glue job's IAM role.

Question 15

A large e-commerce company is using Amazon DynamoDB as the source for real-time analytics. The data is streamed to Amazon Kinesis Data Streams using DynamoDB Streams and then processed by an AWS Lambda function. The Lambda function writes the data to an Amazon Elasticsearch Service cluster for search and visualization. Recently, the Lambda function has been failing with throttling errors from the Elasticsearch cluster. What is the MOST effective way to handle this?

Accepted Answer

Configure a Dead Letter Queue (DLQ) on the Lambda function to capture failed records and implement retry logic.. Using a Dead Letter Queue (DLQ) allows the Lambda function to capture records that fail due to Elasticsearch throttling, so they can be retried later without blocking the function's processing of other records. This prevents data loss and handles backpressure effectively. Option A is incorrect because increasing Lambda concurrency would increase the rate of writes to the Elasticsearch cluster, worsening the throttling. Option B is incorrect because increasing Kinesis shards would increase the throughput of data arriving at the Lambda function, again exacerbating the throttling. Option C is incorrect because decreasing the Kinesis retention period does not reduce the data volume; it only changes how long data is stored in the stream, and it would not solve the throttling issue.

Answer

Increase the Lambda function's reserved concurrency to handle more invocations.

Answer

Increase the number of shards in the Kinesis data stream.

Answer

Decrease the Kinesis stream's retention period to reduce the data volume.