MLS-C01 Data Engineering Practice Test 23 — 15 Questions

Question 1

Refer to the exhibit. A data engineer runs the AWS CLI command to check an object in an S3 bucket. The bucket is part of a data lake and is configured with versioning enabled. However, the output shows "VersionId": null. What is the most likely reason for this?

Accepted Answer

The object was uploaded before versioning was enabled. Option D is correct because the head-object command does not have the --version-id parameter, so it retrieves the latest version, but if versioning is enabled, the latest version will have a version ID. However, the output shows null, which indicates that the object is not versioned. This happens when the bucket has versioning enabled but the object was uploaded before versioning was enabled. Option A is wrong because the command does not require versioning parameter to show version ID. Option B is wrong because versioning is enabled at the bucket level, not object. Option C is wrong because SSE does not affect versioning.

Answer

The object is encrypted using SSE-S3, which hides the version ID

Answer

The command must include the --version-id parameter to display the version ID

Answer

Versioning is not enabled on the bucket

Question 2

A company runs a streaming data pipeline using Amazon Kinesis Data Streams with 10 shards. The pipeline ingests sensor data from thousands of devices. Each device sends a JSON payload every 5 seconds. The payload size is approximately 2 KB. The data is consumed by a fleet of EC2 instances running a custom Java application that uses the Kinesis Client Library (KCL). Over the past week, the company has observed that the consumer application is experiencing increased latency, and the Kinesis stream's 'GetRecords.IteratorAgeMilliseconds' CloudWatch metric is consistently above 10 seconds. The company has verified that the EC2 instances have sufficient CPU and memory resources. The KCL application is configured with 10 workers, one per shard. The application processes each record by performing a simple transformation and writing to Amazon DynamoDB. The DynamoDB table has sufficient write capacity and is not throttling. The company wants to reduce the iterator age to under 2 seconds. Which action should the company take?

Accepted Answer

Increase the number of shards in the Kinesis stream to 20. Option A is correct because the consumer is bottlenecked by the number of shards; increasing shards allows more parallelism and reduces latency. Option B is wrong because the issue is not DynamoDB capacity. Option C is wrong because switching to Firehose would change the architecture and may not support the custom transformation. Option D is wrong because increasing worker count beyond the number of shards does not help due to KCL's design.

Answer

Replace Kinesis Data Streams with Amazon Kinesis Data Firehose

Answer

Increase the write capacity of the DynamoDB table

Answer

Increase the number of KCL workers to 20

Question 3

A company is streaming e-commerce events to Amazon Kinesis Data Streams. The data science team needs to join events from multiple shards in near real-time and then store the joined results in Amazon S3. Which solution would meet these requirements with the LEAST operational overhead?

Accepted Answer

Use Amazon Kinesis Data Analytics for Apache Flink to read from the Kinesis stream, perform a join operation using Flink SQL, and write the results to S3 using a sink connector.. Option C is correct because Amazon Kinesis Data Analytics for Apache Flink can read from a Kinesis stream, perform stateful joins across shards using Flink's SQL or DataStream API, and write results to S3. Option A is wrong because while Glue ETL can process data, it is batch-oriented and not designed for near real-time streaming joins. Option B is wrong because Lambda with Kinesis triggers processes each shard independently; joining across shards would require external state management and is not a typical pattern. Option D is wrong because Kinesis Data Firehose cannot perform joins; it only writes data to destinations.

Answer

Use AWS Lambda functions with Kinesis triggers to process each record, join across shards using a DynamoDB table for state, and write to S3.

Answer

Use Amazon Kinesis Data Firehose to buffer the data and write to S3, then use Amazon Athena to join the data after it is stored.

Answer

Use AWS Glue ETL jobs that read from the Kinesis stream via the Kinesis connector and write the joined results to S3.

Question 4

A data engineer needs to load data from a MySQL database to Amazon S3 daily. The database is 500 GB and the load window is 2 hours. The data must be extracted without impacting the source database performance. Which AWS service should be used to perform the extraction?

Accepted Answer

AWS Database Migration Service (AWS DMS) with a full-load task to S3.. Option D is correct because AWS Database Migration Service (DMS) can perform continuous or scheduled full-load and change-data-capture tasks from MySQL to S3 without impacting source performance when using appropriate task settings and replication instance. Option A is wrong because Amazon Athena is a query service, not an extraction tool. Option B is wrong because AWS Glue can connect to JDBC sources but may cause higher overhead on the source; DMS is purpose-built. Option C is wrong because Amazon EMR is for big data processing, not direct extraction from MySQL to S3.

Answer

AWS Glue ETL job using a JDBC connection to read the full table.

Answer

Amazon Athena with the MySQL federated query connector.

Answer

Amazon EMR with a Spark job reading from MySQL via JDBC.

Question 5

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The stream has 8 shards. A Lambda function processes each record and writes to Amazon DynamoDB. The Lambda function sometimes fails due to DynamoDB write throttling, causing duplicate processing of records after retries. The data engineering team needs to ensure exactly-once processing semantics for the DynamoDB writes. What should the team do?

Accepted Answer

Use DynamoDB conditional writes with the Kinesis sequence number as a unique attribute to make writes idempotent.. Option B is correct because DynamoDB transactions allow conditional writes. Using the Kinesis sequence number as a condition check ensures idempotency: if a record with that sequence number already exists, the write is skipped. Option A is wrong because increasing write capacity may reduce throttling but does not guarantee exactly-once; duplicates can still occur. Option C is wrong because SQS delays are for visibility timeout, not idempotency. Option D is wrong because FIFO queues support exactly-once delivery but the Lambda would still need to deduplicate within the batch.

Answer

Use an Amazon SQS FIFO queue between Kinesis and Lambda to deduplicate records.

Answer

Configure the Lambda event source mapping with a maximum retry count of 0 and a DLQ.

Answer

Increase the DynamoDB write capacity units to avoid throttling.

Question 6

A data pipeline uses AWS Glue to crawl an S3 bucket and create a table in the AWS Glue Data Catalog. The data is in Parquet format with partitions by date. After a new partition is added to S3, the crawler runs but the new partition is not reflected in the table. What is the most likely cause?

Accepted Answer

The new partition folder does not follow the Hive-style partition naming convention expected by the crawler.. Option A is correct because the Glue crawler may be configured to only crawl new folders if the crawler's configuration is set to 'Crawl new folders only' but the partition path may not match the expected pattern, or the crawler's 'Schema updates' setting might be set to 'Ignore'. More commonly, the crawler's 'Update the table definition' option is set to 'Add new columns only' or 'Ignore the change and don't update the table'; the correct setting to add partitions is 'Add new columns only' or 'Add new partitions only'. However, the most typical issue is that the crawler is set to 'Crawl all folders each time' and still not picking up because the partition path is not in a recognized Hive-style format (e.g., date=2023-01-01/). Option A points to the partition path format. Option B is wrong because Glue can handle Parquet. Option C is wrong because the crawler does not need Lambda triggers. Option D is wrong because the crawler can handle up to many partitions.

Answer

The crawler requires an AWS Lambda trigger to be configured for new partitions.

Answer

The Parquet schema in the new partition does not match the existing table schema.

Answer

The S3 bucket has too many partitions, exceeding the Glue crawler limit.

Question 7

A data engineering team uses Amazon EMR with Spark to transform large datasets in S3. The team notices that the Spark jobs on the EMR cluster are failing with out-of-memory errors. The cluster uses instance types with moderate memory. Which configuration change would MOST effectively reduce memory pressure without increasing cost?

Accepted Answer

Enable shuffle compression and set spark.shuffle.compress to true.. Option C is correct because enabling compression reduces the amount of data shuffled over the network and stored in memory, thus reducing memory usage. Option A is wrong because increasing the number of executor cores may increase parallelism but does not directly reduce memory per task. Option B is wrong because using more instances would increase cost. Option D is wrong because Kryo serialization reduces object size but is not as effective as compression for shuffle data.

Answer

Use a larger EMR cluster with more instances of the same type.

Answer

Increase the number of executor cores to improve parallelism.

Answer

Switch from Java serialization to Kryo serialization.

Question 8

A company uses Amazon Kinesis Data Firehose to deliver data to an Amazon S3 bucket. The data is organized by year/month/day/hour. The team needs to ensure that all data is encrypted at rest in S3 using an AWS KMS customer managed key (CMK). Which configuration should the team implement?

Accepted Answer

In the Firehose delivery stream configuration, enable S3 destination encryption and select the customer managed KMS key.. Option A is correct because Kinesis Data Firehose can be configured to use server-side encryption with AWS KMS (SSE-KMS) for the S3 destination. Option B is wrong because S3 default encryption is applied at the bucket level, but Firehose can override it; the recommended approach is to configure encryption in Firehose. Option C is wrong because the bucket policy with Deny is not needed if encryption is configured properly. Option D is wrong because client-side encryption is not suitable for this pipeline.

Answer

Configure the S3 bucket's default encryption to use the customer managed KMS key.

Answer

Use an AWS Lambda function to encrypt the data after it is delivered to S3.

Answer

Add a bucket policy that denies PutObject unless the request includes the correct KMS key.

Question 9

A company runs a nightly batch job that reads data from Amazon RDS for PostgreSQL, transforms it using AWS Glue, and writes the output to Amazon S3 in Parquet format. The job takes 2 hours to complete, but the data volume has grown, and the job now takes 4 hours, exceeding the allowed window. The team needs to reduce the job duration without increasing cost. Which action is MOST effective?

Accepted Answer

Partition the output S3 data by a high-cardinality column used in filtering during transformation.. Option D is correct because partitioning the Parquet output by a frequently filtered column reduces the amount of data processed downstream and can speed up the Glue job if the transformation includes filtering. However, the question asks to reduce job duration: increasing parallelism by increasing the number of DPUs (option C) would increase cost, but the team does not want to increase cost. Option A is wrong because Glue's job bookmark can help incremental loads but does not speed up the full load. Option B is wrong because converting to CSV may increase size and processing time. Option D is the best: partitioning the output can improve write performance and reduce data volume for subsequent steps, but note that it might not drastically reduce the job duration itself. Actually, the most effective without increasing cost is to optimize the transformation logic. But among given options, D is plausible. Let's reconsider: Option C increases cost. Option D partitions output, which may slightly reduce write time but not transformation time. Option A and B are not effective. Perhaps the best is to use a larger instance with same cost? Not possible. Option D is the only one that does not increase cost and may help. I'll stick with D.

Answer

Increase the number of Glue DPUs (Data Processing Units) allocated to the job.

Answer

Enable AWS Glue job bookmark to skip already processed data.

Answer

Change the output format from Parquet to CSV to reduce write time.

Question 10

A company uses Amazon S3 to store historical transaction data in CSV format. The data is partitioned by transaction_date. A data analyst runs Amazon Athena queries that frequently filter on customer_id and transaction_date. The queries are slow and expensive. The team needs to improve query performance and reduce cost. Which combination of actions should the team take? (Choose TWO.)

Accepted Answer

Convert the data from CSV to Parquet format.. Option B and D are correct. Converting to Parquet reduces data scanned due to columnar storage and compression. Partitioning by customer_id (which is frequently filtered) improves partition pruning. Option A is wrong because increasing workers is not applicable to Athena (serverless). Option C is wrong because converting to JSON increases size. Option E is wrong because using S3 Select may not integrate with Athena directly.

Answer

Enable S3 Select pushdown in Athena to reduce data transfer.

Answer

Convert the data to JSON format for better query performance.

Answer

Increase the number of Athena query workers.

Question 11

A company uses Amazon Kinesis Data Streams to collect IoT sensor data. The stream has 4 shards. A consumer application reads from the stream using the Kinesis Client Library (KCL). The application processes records and stores them in Amazon DynamoDB. Recently, the data volume has increased, and the consumer is falling behind. Which action should the team take to increase the processing throughput?

Accepted Answer

Increase the number of shards in the Kinesis stream.. Option C is correct because increasing the number of shards increases the stream's throughput and allows more concurrent consumers. Option A is wrong because increasing the retention period does not affect throughput. Option B is wrong because adding more KCL workers without increasing shards will cause them to idle. Option D is wrong because increasing DynamoDB write capacity may reduce throttling but does not increase the consumer's reading throughput.

Answer

Deploy additional consumer instances using the same application name.

Answer

Increase the write capacity of the DynamoDB table.

Answer

Increase the data retention period of the stream to 7 days.

Question 12

A company uses Amazon Redshift to run analytics on sales data. The data is loaded daily from S3 using COPY commands. The team notices that the COPY command performance degrades over time due to table bloat. The team needs to maintain query performance and reduce storage costs. Which combination of maintenance operations should the team perform regularly? (Choose THREE.)

Accepted Answer

Run the VACUUM command to reclaim space and re-sort rows.. Option A, C, and D are correct. VACUUM reclaims space and re-sorts rows (if sort keys defined). ANALYZE updates statistics for query planning. DEEP COPY recreates the table to eliminate bloat completely. Option B is wrong because UNLOAD exports data, not maintenance. Option E is wrong because changing distribution style is a schema change, not regular maintenance.

Answer

Run the UNLOAD command to export data to S3 and then reload.

Answer

Change the distribution style of the table to KEY.

Question 13

A data engineer needs to transfer 50 TB of data from an on-premises HDFS cluster to Amazon S3. The data must be encrypted in transit and at rest. The on-premises network has a 1 Gbps connection to AWS. The transfer must complete within 5 days. Which solution is MOST cost-effective and meets the requirements?

Accepted Answer

Use AWS DataSync with a DataSync agent installed on-premises to transfer the data to S3.. Option C is correct. With 1 Gbps, the maximum theoretical transfer in 5 days is about 54 TB (1 Gbps = 0.125 GB/s, 0.125 * 86400 * 5 = 54000 GB = 54 TB). So it is feasible. AWS DataSync can transfer data from HDFS via a private endpoint using the DataSync agent, with encryption in transit (TLS) and at rest (S3 SSE). Option A is wrong because S3 Transfer Acceleration only speeds up uploads over public internet, not from HDFS directly. Option B is wrong because Snowball Edge would be faster but more expensive for this volume that can fit in the time window. Option D is wrong because AWS Glue is for ETL, not data transfer.

Answer

Use S3 Transfer Acceleration to upload the data directly from HDFS to S3.

Answer

Order an AWS Snowball Edge device and copy the data to it, then ship it back.

Answer

Use AWS Glue to read from HDFS and write to S3 in a continuous ETL job.

Question 14

A company uses Amazon Athena to analyze data stored in S3. The data is in CSV format and is partitioned by year/month/day. Queries that filter on a specific day are slow. The team wants to improve query performance without changing the data format. Which action should the team take?

Accepted Answer

Ensure that the S3 folder structure follows the Hive-style partition naming convention (e.g., year=2023/month=01/day=01).. Option D is correct because partitioning pruning is most effective when the partition columns are in a Hive-style format (e.g., year=2023/month=01/day=01). If the folders are not in that format, Athena may not prune partitions properly. Option A is wrong because increasing the number of partitions would further slow queries. Option B is wrong because converting to Parquet changes the format. Option C is wrong because using a large number of small files hurts performance.

Answer

Use a larger number of small files to increase parallelism.

Answer

Increase the number of partitions by adding hour and minute levels.

Answer

Convert the data to Parquet format.

Question 15

A company uses AWS Glue to run ETL jobs on a daily basis. The jobs read from Amazon RDS and write to Amazon S3. The data volume has grown, and the jobs are taking longer to complete. The team wants to optimize the jobs for cost and performance. Which combination of techniques should the team implement? (Choose THREE.)

Accepted Answer

Use a larger Glue worker type, such as G.2X, for more memory per worker.. Option A, B, and D are correct. Using job bookmarks enables incremental processing, reducing the amount of data read. Using pushdown predicates filters data at the source, reducing data transfer. Using a larger number of worker type (e.g., G.1X or G.2X) increases memory and CPU per worker, improving performance. Option C is wrong because adding more partitions after the job does not speed up the job. Option E is wrong because increasing the number of DPUs increases cost linearly and may not be as effective as using larger workers.

Answer

Increase the number of partitions in the output S3 data to improve parallelism.

Answer

Increase the maximum number of DPUs for the job to 100.