MLS-C01 Data Engineering Practice Test 27 — 15 Questions

Question 1

A company is migrating its on-premises Apache Hadoop cluster to AWS. The cluster processes large datasets using Spark jobs. The company wants to minimize operational overhead and use native AWS services. Which combination of services should the company use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Amazon EMR with Spark and Amazon S3. Amazon EMR is the native AWS big-data service that runs Apache Spark without managing cluster nodes, and Amazon S3 provides durable, decoupled storage for the datasets. This combination minimizes operational overhead because EMR handles provisioning, scaling, and patching, while S3 decouples compute from storage so clusters can be transient. It is the direct lift-and-shift of a Hadoop/Spark workload to a managed service.

Answer

Amazon Redshift with Spectrum and Amazon S3

Answer

Amazon Athena and AWS Glue

Answer

Amazon EC2 instances with Apache Spark installed and Amazon S3

Question 2

Refer to the exhibit. A data engineer runs the AWS CLI command to check an object in an S3 bucket. The bucket is part of a data lake and is configured with versioning enabled. However, the output shows "VersionId": null. What is the most likely reason for this?

Accepted Answer

The object was uploaded before versioning was enabled. When versioning is enabled on a bucket, objects uploaded after that point receive a unique VersionId. Objects that existed before versioning was enabled have a VersionId of null — they are treated as the 'null version.' Therefore, an object showing 'VersionId': null was most likely uploaded before versioning was turned on.

Answer

The object is encrypted using SSE-S3, which hides the version ID

Answer

The command must include the --version-id parameter to display the version ID

Answer

Versioning is not enabled on the bucket

Question 3

A company is streaming e-commerce events to Amazon Kinesis Data Streams. The data science team needs to join events from multiple shards in near real-time and then store the joined results in Amazon S3. Which solution would meet these requirements with the LEAST operational overhead?

Accepted Answer

Use Amazon Kinesis Data Analytics for Apache Flink to read from the Kinesis stream, perform a join operation using Flink SQL, and write the results to S3 using a sink connector.. Amazon Kinesis Data Analytics for Apache Flink is a fully managed service that can read from Kinesis Data Streams, perform stateful stream processing (including joins across shards), and write results to S3 using a built-in sink connector. It handles shard enumeration, checkpointing, and scaling automatically, minimizing operational overhead. This directly meets the requirements of near real-time join and S3 storage without managing infrastructure.

Answer

Use AWS Lambda functions with Kinesis triggers to process each record, join across shards using a DynamoDB table for state, and write to S3.

Answer

Use Amazon Kinesis Data Firehose to buffer the data and write to S3, then use Amazon Athena to join the data after it is stored.

Answer

Use AWS Glue ETL jobs that read from the Kinesis stream via the Kinesis connector and write the joined results to S3.

Question 4

A data engineer needs to load data from a MySQL database to Amazon S3 daily. The database is 500 GB and the load window is 2 hours. The data must be extracted without impacting the source database performance. Which AWS service should be used to perform the extraction?

Accepted Answer

AWS Database Migration Service (AWS DMS) with a full-load task to S3.. AWS Database Migration Service (AWS DMS) with a full-load task to S3 is the correct choice because it is designed to migrate data from databases to S3 with minimal impact on the source. DMS reads from the source in a controlled manner and can be configured to avoid performance degradation.

Answer

AWS Glue ETL job using a JDBC connection to read the full table.

Answer

Amazon Athena with the MySQL federated query connector.

Answer

Amazon EMR with a Spark job reading from MySQL via JDBC.

Question 5

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The stream has 8 shards. A Lambda function processes each record and writes to Amazon DynamoDB. The Lambda function sometimes fails due to DynamoDB write throttling, causing duplicate processing of records after retries. The data engineering team needs to ensure exactly-once processing semantics for the DynamoDB writes. What should the team do?

Accepted Answer

Use DynamoDB conditional writes with the Kinesis sequence number as a unique attribute to make writes idempotent.. Exactly-once semantics for DynamoDB writes from Kinesis-triggered Lambda require idempotency, since Lambda retries on failure can reprocess the same record. Using DynamoDB conditional writes with the Kinesis sequence number as a unique attribute ensures that a duplicate write is rejected if the sequence number already exists, making the operation idempotent and achieving exactly-once effect.

Answer

Use an Amazon SQS FIFO queue between Kinesis and Lambda to deduplicate records.

Answer

Configure the Lambda event source mapping with a maximum retry count of 0 and a DLQ.

Answer

Increase the DynamoDB write capacity units to avoid throttling.

Question 6

A data pipeline uses AWS Glue to crawl an S3 bucket and create a table in the AWS Glue Data Catalog. The data is in Parquet format with partitions by date. After a new partition is added to S3, the crawler runs but the new partition is not reflected in the table. What is the most likely cause?

Accepted Answer

The new partition folder does not follow the Hive-style partition naming convention expected by the crawler.. AWS Glue crawlers rely on Hive-style partition naming (e.g., year=2024/month=01/day=15/) to detect and register partitions in the Data Catalog. If the new partition folder is named differently — such as 2024/01/15/ without the key=value format — the crawler will not recognize it as a partition and the table will not be updated. This is the most common cause of missing partitions after a crawler run.

Answer

The crawler requires an AWS Lambda trigger to be configured for new partitions.

Answer

The Parquet schema in the new partition does not match the existing table schema.

Answer

The S3 bucket has too many partitions, exceeding the Glue crawler limit.

Question 7

A company uses Amazon Kinesis Data Streams to collect IoT sensor data. The stream has 4 shards. A consumer application reads from the stream using the Kinesis Client Library (KCL). The application processes records and stores them in Amazon DynamoDB. Recently, the data volume has increased, and the consumer is falling behind. Which action should the team take to increase the processing throughput?

Accepted Answer

Increase the number of shards in the Kinesis stream.. Kinesis Data Streams throughput is bounded by the number of shards: each shard supports up to 1 MB/s or 1,000 records/s for writes and 2 MB/s for reads (shared among consumers). With only 4 shards, the stream's total read capacity caps the KCL consumer's throughput regardless of how many instances are running. Increasing the shard count (resharding) raises the stream's parallelism and lets KCL assign more shards to more workers, directly increasing processing throughput.

Answer

Deploy additional consumer instances using the same application name.

Answer

Increase the write capacity of the DynamoDB table.

Answer

Increase the data retention period of the stream to 7 days.

Question 8

A company uses Amazon Redshift to run analytics on sales data. The data is loaded daily from S3 using COPY commands. The team notices that the COPY command performance degrades over time due to table bloat. The team needs to maintain query performance and reduce storage costs. Which combination of maintenance operations should the team perform regularly? (Choose THREE.)

Accepted Answer

Run the VACUUM command to reclaim space and re-sort rows.. Option C is correct because VACUUM in Amazon Redshift reclaims space from deleted rows and re-sorts rows according to the table's sort keys, directly addressing table bloat and restoring query performance. Option D is correct because a DEEP COPY recreates the table by copying data to a new table with optimal physical storage, fully removing bloat and applying the current sort and distribution settings, which is useful when VACUUM alone is insufficient. Option E is correct because ANALYZE updates table statistics used by the query planner, ensuring efficient query plans after data changes and maintenance operations. Option A is not appropriate because UNLOAD followed by reload is a manual, disruptive workaround rather than a standard maintenance operation for bloat. Option B is not correct because changing the distribution style to KEY is a design decision that does not reclaim space or resolve bloat.

Answer

Run the UNLOAD command to export data to S3 and then reload.

Answer

Change the distribution style of the table to KEY.

Question 9

A company uses AWS Glue to run ETL jobs on a daily basis. The jobs read from Amazon RDS and write to Amazon S3. The data volume has grown, and the jobs are taking longer to complete. The team wants to optimize the jobs for cost and performance. Which combination of techniques should the team implement? (Choose THREE.)

Accepted Answer

Use a larger Glue worker type, such as G.2X, for more memory per worker.. Option A is correct because moving to a larger Glue worker type such as G.2X provides more memory and vCPUs per worker, which helps memory-intensive or shuffle-heavy ETL jobs complete faster and can reduce the total number of workers (and thus cost) needed. Option B is correct because Glue job bookmarks track previously processed data so each daily run only reads new or changed records from Amazon RDS, cutting the volume of data scanned and processed, which directly improves runtime and cost. Option E is correct because pushdown predicates in the JDBC connection push filtering logic to the source database, so only the required rows are read from Amazon RDS instead of pulling the entire table into Glue, reducing I/O and DPU usage. Option C is not correct because increasing output S3 partitions affects downstream read parallelism, not the performance of the Glue ETL job itself, and can even create small-file overhead. Option D is not correct because simply raising the maximum DPU count to 100 does not optimize the job and can substantially increase cost without addressing data volume or filtering inefficiencies.

Answer

Increase the number of partitions in the output S3 data to improve parallelism.

Answer

Increase the maximum number of DPUs for the job to 100.

Question 10

A company uses Amazon EMR to run Spark jobs on a cluster with 10 core nodes of type r5.xlarge. The jobs are I/O intensive and read large amounts of data from S3. The team notices high network throughput but low CPU utilization. Which configuration change would improve job performance at the same cost?

Accepted Answer

Replace the core nodes with r5d.xlarge instances that have local SSDs.. R5d instances include local NVMe SSDs. These SSDs can be used for caching intermediate data during Spark jobs, reducing the need to read from and write to S3 over the network. This directly addresses the I/O bottleneck and high network throughput observed, improving job performance. Option A is incorrect because moving to general-purpose m5 instances does not provide local SSDs and may not improve I/O. Option B is incorrect because doubling the number of core nodes would increase cost significantly without necessarily solving the I/O issue. Option D is incorrect because spot instances reduce cost but do not inherently improve I/O performance; they may even add instability.

Answer

Change the instance type to m5.xlarge (general purpose) to balance resources.

Answer

Increase the number of core nodes to 20.

Answer

Use spot instances for the core nodes to save cost and reinvest in more nodes.

Question 11

A company wants to build a data lake on Amazon S3. The data lake will store raw data in its original format and also transformed data in Parquet. The data is generated by various sources and must be cataloged for discovery. Which service should the company use to automatically discover, catalog, and make the data searchable?

Accepted Answer

AWS Glue Data Catalog. AWS Glue Data Catalog is a fully managed, persistent metadata store that automatically discovers and catalogs data stored in Amazon S3. It uses AWS Glue crawlers to scan data sources, infer schemas, and create table definitions in a central Hive-compatible metastore. This makes the data searchable and immediately queryable by services like Amazon Athena, Amazon Redshift Spectrum, and Amazon EMR. The Data Catalog is the foundational component for building a data lake on AWS because it provides a unified metadata repository across diverse data sources and formats.

Answer

Amazon S3

Answer

Amazon Athena

Answer

Amazon EMR

Question 12

A data engineering team is building a real-time clickstream analytics pipeline on AWS. They need to ingest millions of events per second from mobile apps and websites, process them with low latency, and store the results in Amazon S3 for downstream analysis. Which combination of AWS services should the team use to minimize operational overhead while meeting these requirements?

Accepted Answer

Use Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Analytics for real-time processing, and Amazon Kinesis Data Firehose to deliver results to Amazon S3.. Kinesis Data Streams handles high-throughput ingestion (millions of events per second), Kinesis Data Analytics (now Managed Service for Apache Flink) provides serverless real-time processing with SQL or Flink, and Kinesis Data Firehose delivers results to S3 without managing infrastructure. This combination is fully managed, scales automatically, and minimizes operational overhead while meeting low-latency and S3 delivery requirements.

Answer

Use Amazon MQ to ingest streaming data, AWS Lambda to process each message, and save output to Amazon S3.

Answer

Use Amazon Kinesis Data Streams to ingest data, Amazon EMR to process with Spark Streaming, and save output to Amazon S3.

Answer

Use AWS Glue to ingest data into Amazon RDS, then use AWS Glue ETL jobs to transform and load into Amazon S3.

Question 13

A company uses AWS Glue to run ETL jobs that transform data from Amazon RDS for MySQL to Amazon S3. The current job runs daily and takes 3 hours to process 100 GB of data. The company expects data volume to grow 10x in the next year. They need to reduce job runtime and cost. Which approach should they take?

Accepted Answer

Use parallel reads with pushdown predicates in the Glue job's source connection, and write the output in columnar format (Parquet) partitioned by date.. Using parallel reads with pushdown predicates reduces the amount of data transferred from RDS to Glue by filtering at the database level, which lowers extraction time and load on the source. Writing output in columnar format (Parquet) reduces storage size and improves query performance for downstream analytics. Partitioning by date enables efficient pruning. Option A is incorrect because S3 Select is used for server-side filtering of data already in S3, not for tuning extraction from RDS. Option C is incorrect because increasing DPUs alone does not solve the bottleneck from the source database; pushdown predicates are more effective. Option D is incorrect because Redshift Spectrum is used for querying data in S3, not for performing transformations in ETL jobs.

Answer

Use S3 Select with Glue to filter data before transformation.

Answer

Increase the number of Glue DPUs to 100 and enable job bookmarking.

Answer

Use Amazon Redshift Spectrum to perform transformations in place on S3.

Question 14

You are a data engineer at a fintech company. The company processes real-time stock market data from multiple exchanges. The data is ingested via Amazon Kinesis Data Streams with 50 shards. Each record is about 1 KB, and the ingestion rate is 5,000 records per second. The data is consumed by a Java application running on Amazon ECS that performs real-time analytics and stores results in Amazon DynamoDB. Recently, the application has been experiencing high latency, and some records are stuck in the shards for minutes before being consumed. The CloudWatch metrics show that the application's CPU utilization is low, but the iterator age is increasing. The application uses the Kinesis Client Library (KCL) with a single worker. What is the most likely cause and how should it be fixed?

Accepted Answer

Scale the number of KCL workers to match the number of shards (e.g., 50 workers) to process shards in parallel.. The Kinesis Client Library (KCL) assigns one shard per worker thread/process; with a single worker and 50 shards, only one shard is processed at a time, causing iterator age to climb while CPU stays idle. Scaling to 50 KCL workers (one per shard) allows parallel consumption across all shards, eliminating the backlog. This matches the symptom of low CPU with rising iterator age — a classic under-provisioned consumer signature.

Answer

Increase the number of shards to 200 to provide more throughput.

Answer

Increase the CPU capacity of the ECS task by moving to a larger instance type.

Answer

Move the destination from DynamoDB to Amazon RDS to reduce write latency.

Question 15

A data engineer is responsible for managing a data lake on Amazon S3. The data lake contains CSV files from various sources, totaling 10 TB. The engineer needs to make this data queryable using Amazon Athena. However, Athena queries are currently taking a long time and scanning large amounts of data. The engineer has noticed that the CSV files are not partitioned, and there are no indexes. The engineer wants to improve query performance and reduce costs. The data is accessed frequently for the last 30 days, but older data is rarely queried. The engineer also wants to minimize the amount of data scanned by Athena. What should the engineer do?

Accepted Answer

Convert the CSV files to Parquet format and partition the data by date.. Converting CSV to Parquet reduces data scanned because Parquet is columnar and compressed, and partitioning by date enables partition pruning so Athena only reads the last 30 days of frequently accessed data. Together these dramatically improve query performance and lower cost.

Answer

Convert the CSV files to JSON format and use Athena to query them.

Answer

Create indexes on the S3 objects using AWS Glue.

Answer

Convert the CSV files to ORC format and create a view in Athena.