MLS-C01 Data Engineering Practice Test 13 — 15 Questions

Question 1

A company needs to move 50 TB of data from an on-premises data center to Amazon S3. The company has a limited internet bandwidth of 100 Mbps. The data transfer must be completed within 10 days. Which TWO services should the company use together to meet these requirements?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Amazon S3 as the destination. Option A (Amazon S3 as the destination) is correct because the requirement is to move the data into Amazon S3, so S3 must be the target storage service that Snowball Edge imports into. Option D (AWS Snowball Edge) is correct because transferring 50 TB over a 100 Mbps link would take roughly 46+ days (50 TB ≈ 400,000,000 Mb ÷ 100 Mbps), far exceeding the 10-day deadline, whereas a physical Snowball Edge device ships the data offline and can complete the job within the window. Option B (AWS Direct Connect) is not appropriate because provisioning a dedicated circuit typically takes weeks and still wouldn't provide enough bandwidth in time. Option C (AWS Site-to-Site VPN) is incorrect because it runs over the same limited internet connection and cannot accelerate the transfer. Option E (AWS DataSync over the internet) is incorrect because it also depends on the constrained 100 Mbps bandwidth and would not finish within 10 days.

Answer

AWS Direct Connect

Answer

AWS Site-to-Site VPN

Answer

AWS DataSync over the internet

Question 2

Refer to the exhibit. A data engineer is creating an IAM policy for an AWS Glue ETL job that reads encrypted objects from an S3 bucket, transforms them, and writes the results back to the same bucket. The bucket uses SSE-KMS encryption with the KMS key specified. The ETL job is failing with an "Access Denied" error when trying to write data. What is the likely cause?

Accepted Answer

The policy is missing the kms:Encrypt permission. The IAM policy must include the kms:Encrypt permission for the AWS Glue ETL job to write encrypted objects to the S3 bucket using SSE-KMS. The policy likely includes s3:PutObject, kms:Decrypt, and kms:GenerateDataKey, but kms:Encrypt is required for the write operation. Options A, B, and C are incorrect because the necessary permissions (kms:Decrypt, s3:PutObjectAcl, and s3:PutObject) are either already present or not required for writing.

Answer

The policy is missing the kms:Decrypt permission

Answer

The policy is missing the s3:PutObjectAcl permission

Answer

The policy is missing the s3:PutObject permission

Question 3

A company runs a streaming data pipeline using Amazon Kinesis Data Streams with 10 shards. The pipeline ingests sensor data from thousands of devices. Each device sends a JSON payload every 5 seconds. The payload size is approximately 2 KB. The data is consumed by a fleet of EC2 instances running a custom Java application that uses the Kinesis Client Library (KCL). Over the past week, the company has observed that the consumer application is experiencing increased latency, and the Kinesis stream's 'GetRecords.IteratorAgeMilliseconds' CloudWatch metric is consistently above 10 seconds. The company has verified that the EC2 instances have sufficient CPU and memory resources. The KCL application is configured with 10 workers, one per shard. The application processes each record by performing a simple transformation and writing to Amazon DynamoDB. The DynamoDB table has sufficient write capacity and is not throttling. The company wants to reduce the iterator age to under 2 seconds. Which action should the company take?

Accepted Answer

Increase the number of shards in the Kinesis stream to 20. The KCL application is already running one worker per shard (10 workers for 10 shards), so parallelism is maxed out at the shard level. IteratorAgeMilliseconds above 10 seconds with adequate CPU/memory indicates the consumers cannot keep up with the read throughput per shard. Increasing shards to 20 doubles the parallel read capacity and allows KCL to rebalance workers across the new shards, reducing iterator age.

Answer

Replace Kinesis Data Streams with Amazon Kinesis Data Firehose

Answer

Increase the write capacity of the DynamoDB table

Answer

Increase the number of KCL workers to 20

Question 4

A data engineering team uses Amazon EMR with Spark to transform large datasets in S3. The team notices that the Spark jobs on the EMR cluster are failing with out-of-memory errors. The cluster uses instance types with moderate memory. Which configuration change would MOST effectively reduce memory pressure without increasing cost?

Accepted Answer

Enable shuffle compression and set spark.shuffle.compress to true.. Enabling shuffle compression (spark.shuffle.compress=true) reduces the amount of data written to disk and stored in memory during shuffle operations, thereby lowering memory pressure without changing the cluster size or cost. Option A is incorrect because adding more instances increases cost and does not directly reduce memory per task. Option B is incorrect because increasing executor cores may increase parallelism but can also increase memory usage per task if not properly configured, and does not directly reduce memory pressure. Option C is incorrect because while Kryo serialization reduces object serialization overhead, it does not significantly reduce shuffle data size; shuffle compression is more effective for reducing memory usage during shuffle.

Answer

Use a larger EMR cluster with more instances of the same type.

Answer

Increase the number of executor cores to improve parallelism.

Answer

Switch from Java serialization to Kryo serialization.

Question 5

A company uses Amazon Kinesis Data Firehose to deliver data to an Amazon S3 bucket. The data is organized by year/month/day/hour. The team needs to ensure that all data is encrypted at rest in S3 using an AWS KMS customer managed key (CMK). Which configuration should the team implement?

Accepted Answer

In the Firehose delivery stream configuration, enable S3 destination encryption and select the customer managed KMS key.. The correct approach. Kinesis Data Firehose can be configured to encrypt data at rest in S3 using AWS KMS. In the Firehose delivery stream configuration, under S3 destination settings, you can enable encryption and select a customer managed KMS key. This ensures all data written to S3 is encrypted with that key. Option A is incorrect because S3 default encryption applies to objects uploaded directly to S3, but Firehose writes objects using its own IAM role and can override default encryption; configuring encryption in Firehose is the recommended way. Option B is incorrect because using a Lambda function to encrypt after delivery adds unnecessary complexity and latency; Firehose can encrypt natively. Option D is incorrect because a bucket policy denying PutObject without the correct KMS key would work but is not the simplest or most straightforward configuration; Firehose can handle encryption directly.

Answer

Configure the S3 bucket's default encryption to use the customer managed KMS key.

Answer

Use an AWS Lambda function to encrypt the data after it is delivered to S3.

Answer

Add a bucket policy that denies PutObject unless the request includes the correct KMS key.

Question 6

A company runs a nightly batch job that reads data from Amazon RDS for PostgreSQL, transforms it using AWS Glue, and writes the output to Amazon S3 in Parquet format. The job takes 2 hours to complete, but the data volume has grown, and the job now takes 4 hours, exceeding the allowed window. The team needs to reduce the job duration without increasing cost. Which action is MOST effective?

Accepted Answer

Partition the output S3 data by a high-cardinality column used in filtering during transformation.. Increasing DPUs would increase cost, which violates the requirement to not increase cost. Option B is incorrect because job bookmarks are designed for incremental processing, but the job runs a full batch load nightly, so bookmarks do not skip data in a full load. Option C is incorrect because converting from Parquet to CSV increases file size and write time, likely lengthening the job. Option D is correct: partitioning the output by a high-cardinality column used in filtering during transformation can reduce write time by enabling parallel writes and can improve downstream query performance, but more importantly it may allow the transformation to leverage partition pruning if the job reads the output back or if the transformation logic can use the partition column to reduce data shuffling. However, the primary benefit is that it optimizes storage and retrieval without additional cost, making it the best choice among the options.

Answer

Increase the number of Glue DPUs (Data Processing Units) allocated to the job.

Answer

Enable AWS Glue job bookmark to skip already processed data.

Answer

Change the output format from Parquet to CSV to reduce write time.

Question 7

A company uses Amazon S3 to store historical transaction data in CSV format. The data is partitioned by transaction_date. A data analyst runs Amazon Athena queries that frequently filter on customer_id and transaction_date. The queries are slow and expensive. The team needs to improve query performance and reduce cost. Which combination of actions should the team take? (Choose TWO.)

Accepted Answer

Convert the data from CSV to Parquet format.. Option C is correct because converting CSV to Parquet, a columnar format, lets Athena read only the columns referenced in a query (customer_id and transaction_date) instead of scanning entire rows, drastically reducing data scanned and thus cost and latency. Option D is correct because adding customer_id as a partition key (with transaction_date) allows Athena's partition pruning to skip irrelevant S3 prefixes when queries filter on customer_id, further cutting the amount of data scanned. Option A is wrong because S3 Select pushdown is not a general Athena feature that reduces transfer for these queries; Athena already pushes predicate evaluation to the storage layer via Parquet/ORC statistics, and S3 Select is a separate S3 API. Option B is wrong because JSON is a row-based text format that is generally slower and more expensive to scan than columnar Parquet. Option E is wrong because Athena is serverless and does not expose configurable 'query workers'; performance is governed by data format, partitioning, and file layout, not worker count.

Answer

Enable S3 Select pushdown in Athena to reduce data transfer.

Answer

Convert the data to JSON format for better query performance.

Answer

Increase the number of Athena query workers.

Question 8

A data engineer needs to transfer 50 TB of data from an on-premises HDFS cluster to Amazon S3. The data must be encrypted in transit and at rest. The on-premises network has a 1 Gbps connection to AWS. The transfer must complete within 5 days. Which solution is MOST cost-effective and meets the requirements?

Accepted Answer

Use AWS DataSync with a DataSync agent installed on-premises to transfer the data to S3.. AWS DataSync with an on-premises agent is the most cost-effective solution that meets all requirements. DataSync provides built-in encryption in transit (TLS) and at rest (S3 SSE), automates the transfer of large datasets over the existing 1 Gbps link, and can complete 50 TB within 5 days at sustained throughput. It avoids the higher cost and logistics of Snowball Edge while being far more efficient than Transfer Acceleration or Glue for a one-time bulk migration.

Answer

Use S3 Transfer Acceleration to upload the data directly from HDFS to S3.

Answer

Order an AWS Snowball Edge device and copy the data to it, then ship it back.

Answer

Use AWS Glue to read from HDFS and write to S3 in a continuous ETL job.

Question 9

A company uses Amazon Athena to analyze data stored in S3. The data is in CSV format and is partitioned by year/month/day. Queries that filter on a specific day are slow. The team wants to improve query performance without changing the data format. Which action should the team take?

Accepted Answer

Ensure that the S3 folder structure follows the Hive-style partition naming convention (e.g., year=2023/month=01/day=01).. Athena uses Hive-style partitioning to prune partitions based on the partition columns in the WHERE clause. If the S3 folder structure does not follow the Hive-style naming convention (e.g., year=2023/month=01/day=01), Athena cannot infer the partitions correctly, leading to full table scans and slow queries. Ensuring the correct folder structure allows Athena to skip irrelevant partitions, improving performance.

Answer

Use a larger number of small files to increase parallelism.

Answer

Increase the number of partitions by adding hour and minute levels.

Answer

Convert the data to Parquet format.

Question 10

Refer to the exhibit. A data engineer has attached this IAM policy to an IAM role used by an AWS Glue ETL job. The job reads from an S3 bucket (data-bucket) that is encrypted with SSE-KMS using the key arn:aws:kms:us-east-1:123456789012:key/abc123, transforms the data, and writes the result to a different S3 bucket (output-bucket) encrypted with a different KMS key (arn:aws:kms:us-east-1:123456789012:key/xyz789). When the job runs, it fails with an access denied error. What is the cause?

Accepted Answer

The policy does not grant kms:Encrypt permission for the output bucket's KMS key.. The AWS Glue ETL job writes to an S3 bucket encrypted with SSE-KMS using a specific KMS key. To write to that bucket, the job's IAM role must have permissions to use that KMS key for encryption (kms:Encrypt, and often kms:GenerateDataKey). The policy lacks these permissions for the output bucket's KMS key, causing an access denied error when the job attempts to write encrypted data.

Answer

The policy does not include s3:GetObject permission for the output bucket.

Answer

The policy does not include glue:CreateTable permission.

Answer

The policy does not include s3:PutObject permission for the output bucket.

Question 11

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data must be transformed before delivery using AWS Lambda. The Lambda function adds a timestamp field. The Firehose stream receives up to 10,000 records per second. The transformation currently takes 500 ms per record. What should the team do to ensure the transformation can keep up with the incoming data without data loss?

Accepted Answer

Increase the Lambda concurrency limit for the function to handle parallel invocations.. Increasing the Lambda concurrency limit allows more parallel invocations, enabling the function to process the high throughput of 10,000 records per second (each taking 500 ms) without falling behind. Option A is incorrect because shards are a concept for Kinesis Data Streams, not Firehose. Option B is incorrect: placing the Lambda function in a VPC typically adds network latency and does not improve performance for this simple transformation. Option D is incorrect: increasing S3 buffer size/interval may delay data delivery but does not increase transformation capacity.

Answer

Increase the number of shards in the Kinesis stream.

Answer

Place the Lambda function in a VPC to improve network performance.

Answer

Increase the S3 buffer size and buffer interval in the Firehose delivery stream.

Question 12

A company runs a critical data pipeline using Apache Spark on Amazon EMR. The pipeline reads data from Amazon S3, performs complex transformations, and writes results back to S3. The job runs every hour and must complete within 30 minutes. Recently, the job has been taking longer and occasionally failing due to executor losses. The team suspects memory pressure. Which action should the team take to improve stability and performance without increasing cost?

Accepted Answer

Enable Spark dynamic allocation to adjust executors based on workload.. Enabling Spark dynamic allocation allows the cluster to scale executors up and down based on workload, which improves stability by adding executors when tasks are pending and removing idle ones, reducing memory pressure and executor loss without over-provisioning. This is a no-cost change because it optimizes existing cluster resources rather than adding nodes or increasing per-executor memory, which would raise cost. Dynamic allocation directly addresses the symptom of executor losses under memory pressure by better matching resources to demand.

Answer

Increase the spark.executor.memory setting to allocate more memory per executor.

Answer

Increase the number of core nodes in the EMR cluster.

Answer

Decrease the number of shuffle partitions (spark.sql.shuffle.partitions) to reduce overhead.

Question 13

A company runs a real-time analytics platform that ingests IoT sensor data from millions of devices. The data is sent to Amazon Kinesis Data Streams with 16 shards. A custom Java application using the Kinesis Client Library (KCL) processes the data and writes aggregated results to Amazon DynamoDB. The application runs on a fleet of EC2 instances in an Auto Scaling group. Recently, the team noticed that some records are being processed multiple times, resulting in duplicate entries in DynamoDB. The application uses the DynamoDB PutItem API to write records. The team needs to eliminate duplicates without significantly increasing latency. Which solution should the team implement?

Accepted Answer

Use DynamoDB TransactWriteItems with a condition check that the record's Kinesis sequence number does not already exist in the table.. KCL guarantees at-least-once delivery, so duplicates are expected when the application reprocesses records after checkpoints or retries. Using DynamoDB TransactWriteItems with a condition check on the Kinesis sequence number ensures idempotent writes: if the sequence number already exists, the write is rejected, eliminating duplicates without adding significant latency.

Answer

Enable DynamoDB auto scaling to increase write capacity and reduce throttling, which causes retries and duplicates.

Answer

Place an Amazon SQS FIFO queue between the KCL application and DynamoDB to deduplicate messages.

Answer

Modify the application to use DynamoDB BatchWriteItem instead of PutItem to reduce the number of write requests.

Question 14

A company uses AWS Glue to run ETL jobs that process data from Amazon RDS for PostgreSQL and load it into Amazon Redshift. The Glue job runs nightly and takes 6 hours to complete. The Redshift cluster is a single dc2.large node. The team needs to reduce the load time to under 3 hours. The data volume is 200 GB per night. The team is considering using Amazon Redshift Spectrum to query data directly from S3 instead of loading it. However, the data transformation logic is complex and requires multiple joins and aggregations that are currently performed in Glue. Which approach should the team recommend to meet the time requirement?

Accepted Answer

Increase the Redshift cluster to a multi-node cluster with dc2.8xlarge nodes to improve COPY and query performance.. The bottleneck is the single dc2.large node, which has limited CPU, memory, and I/O for both COPY ingestion and the complex joins/aggregations. Scaling to a multi-node cluster with dc2.8xlarge nodes distributes the COPY workload and parallelizes query execution across slices, which is the most direct way to cut the 6-hour job below 3 hours while preserving the existing Glue transformation logic. Redshift's MPP architecture means adding nodes and larger instance types linearly improves both load and query throughput.

Answer

Use Redshift Spectrum to create external tables and run the transformations directly in Redshift, bypassing the Glue job.

Answer

Split the Glue job into multiple parallel jobs that each load a portion of the data into separate Redshift tables, then use UNION ALL views.

Answer

Stage the data in S3 in Parquet format and use a COPY command with the PARQUET option to load data faster.

Question 15

A company uses Amazon S3 to store log files from various applications. The logs are in JSON format and are appended to existing files every few minutes. A data analyst wants to run SQL queries on the logs using Amazon Athena. However, queries return incomplete results because Athena does not support modifying data. The team needs to enable querying of the latest log data with minimal changes to the existing ingestion process. Which solution should the team implement?

Accepted Answer

Create an Athena table using the Hive JSON SerDe that reads the logs directly from the existing S3 bucket.. Athena can query JSON data in place using the Hive JSON SerDe without modifying the source files or ingestion pipeline. Since the logs are appended to existing S3 objects, Athena reads the current object contents at query time, so the latest appended data is visible. This satisfies the 'minimal changes' requirement because no ETL or format conversion is needed.

Answer

Convert the logs to Parquet format using a scheduled AWS Glue job and store them in a separate S3 bucket.

Answer

Stream the logs to Amazon Kinesis Data Firehose, which writes the data to S3 in Parquet format.

Answer

Use AWS Glue to load the JSON logs into Amazon Redshift and query using Redshift.