MLS-C01 Data Engineering Practice Test 2 — 15 Questions

Question 1

A company uses Amazon Athena to analyze data stored in S3. The data is in CSV format and is partitioned by year/month/day. Queries that filter on a specific day are slow. The team wants to improve query performance without changing the data format. Which action should the team take?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Ensure that the S3 folder structure follows the Hive-style partition naming convention (e.g., year=2023/month=01/day=01).. Partitioning pruning is most effective when the partition columns are in a Hive-style format (e.g., year=2023/month=01/day=01). If the folders are not in that format, Athena may not prune partitions properly. Option A is wrong because increasing the number of partitions would further slow queries. Option B is wrong because converting to Parquet changes the format. Option C is wrong because using a large number of small files hurts performance.

Answer

Use a larger number of small files to increase parallelism.

Answer

Increase the number of partitions by adding hour and minute levels.

Answer

Convert the data to Parquet format.

Question 2

A company wants to build a data lake on Amazon S3. The data lake will store raw data in its original format and also transformed data in Parquet. The data is generated by various sources and must be cataloged for discovery. Which service should the company use to automatically discover, catalog, and make the data searchable?

Accepted Answer

AWS Glue Data Catalog. AWS Glue Data Catalog is a central metadata repository that can automatically crawl S3 data sources to discover, catalog, and make data searchable. It is the correct choice for building a data lake with raw and transformed data in Parquet. Option B (Amazon S3) is storage, not a catalog. Option C (Amazon Athena) is a query engine, not a catalog. Option D (Amazon EMR) is a big data processing framework, not a catalog.

Answer

Amazon S3

Answer

Amazon Athena

Answer

Amazon EMR

Question 3

A data engineer is tasked with building a pipeline to process streaming data from IoT devices. The devices send data in JSON format every second. The pipeline must aggregate data in 5-minute windows and store the results in Amazon S3. The engineer needs to handle late-arriving data (up to 1 hour) and ensure exactly-once semantics. Which combination of AWS services should they use?

Accepted Answer

Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Analytics for windowed aggregations, and Amazon Kinesis Data Firehose to write to Amazon S3.. Amazon Kinesis Data Analytics supports windowed aggregations via SQL or Apache Flink, can handle late-arriving data using watermarking, and when used with Kinesis Data Streams as the source, provides exactly-once processing semantics. Option B is incorrect because AWS Glue Streaming ETL is based on Spark Streaming; while it can perform windowed aggregations, it does not natively guarantee exactly-once semantics with Kinesis Data Streams and is more complex to configure for late data handling. Option C is incorrect because Amazon SQS is not designed for high-throughput streaming ingestion (IoT devices sending data every second), and AWS Lambda invocations are at-least-once, not exactly-once, making it unsuitable for the required exactly-once semantics. Option D is incorrect because Amazon Kinesis Data Firehose is a delivery service that does not support custom windowed aggregations; it delivers data after a minimum buffer interval of 60 seconds and cannot handle late-arriving data with exactly-once guarantees.

Answer

Amazon Kinesis Data Streams for ingestion, AWS Glue Streaming ETL for aggregation, and Amazon S3 for storage.

Answer

Amazon SQS for ingestion, AWS Lambda for aggregation, and Amazon S3 for storage.

Answer

Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Firehose for transformation, and Amazon S3 for storage.

Question 4

A data engineer needs to transfer 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The company has a 100 Mbps internet connection and the data must be transferred within 5 days. Which AWS service is best suited for this task?

Accepted Answer

AWS Snowball Edge. AWS Snowball Edge is a physical device designed for large-scale data transfer, ideal for moving 50 TB of data when network bandwidth (100 Mbps) is insufficient to meet the 5-day deadline. Option A is wrong because AWS DataSync is a network-based service; transferring 50 TB over 100 Mbps would take approximately 50 days, far exceeding 5 days. Option C is wrong because Amazon S3 Transfer Acceleration improves transfer speeds over the internet but cannot overcome the 100 Mbps bandwidth limitation; it would still take too long. Option D is wrong because AWS Storage Gateway is for hybrid cloud storage with a local cache, not for one-time bulk data migration.

Answer

AWS DataSync

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Storage Gateway

Question 5

A company is migrating on-premises data to AWS. The data includes both structured and unstructured files, totaling 200 TB. The company has a 1 Gbps dedicated network connection to AWS. They want to minimize migration time and cost. Which TWO AWS services or features should they use together? (Choose two.)

Accepted Answer

AWS Snowball Edge. Correct options: A and D. AWS DataSync can efficiently transfer data over the network to S3, and Snowball Edge can be used for the largest files to reduce network load. Option B (S3 Transfer Acceleration) is for speeding up internet transfers, not needed over a dedicated connection. Option C (AWS Glue) is used for ETL, not for bulk data transfer. Option E (AWS Direct Connect) is already implied by the 1 Gbps dedicated connection; it's not an additional service.

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Glue

Answer

AWS Direct Connect

Question 6

A data engineering team is designing a streaming data pipeline that ingests 10,000 events per second. Each event is 2 KB. The pipeline must process events with a latency of less than 1 second. The team is considering using Amazon Kinesis Data Streams with 10 shards. Which TWO additional configurations should the team implement to meet the latency requirement? (Choose two.)

Accepted Answer

Enable auto-scaling of shards based on throughput.. Correct options: D and E. Enhanced fan-out (E) provides dedicated read throughput of 2 MB/second per shard per consumer, eliminating contention and ensuring low latency (<1 second) even with multiple consumers. Auto-scaling shards (D) dynamically adjusts shard count based on incoming throughput, which is critical because the current 10 shards can only ingest 10 MB/sec (1 MB/sec write per shard) but the input is 20 MB/sec (10,000 events × 2 KB each). Option A (record aggregation) reduces PUT request count but does not affect read latency. Option B (Kinesis Firehose to S3) is for data delivery, not latency. Option C (increasing shards to 100) provides excess capacity but is costly and less efficient than auto-scaling.

Answer

Use record aggregation to reduce the number of records.

Answer

Configure Kinesis Data Firehose to deliver data to Amazon S3.

Answer

Increase the number of shards to 100.

Question 7

A company uses AWS Glue to run ETL jobs that transform data from Amazon S3 to Amazon Redshift. The Glue job writes data to Redshift using the JDBC connection. Recently, the job has been failing with connection timeout errors when writing to Redshift. The Redshift cluster is a 2-node dc2.large cluster. The Glue job processes about 50 GB of data per run. The errors occur sporadically, and the job succeeds after a few retries. The data engineer needs to resolve the issue to prevent job failures. What should the engineer do?

Accepted Answer

Modify the Glue job to write data to S3 first, then use the Redshift COPY command to load data.. Writing to Redshift via JDBC is inefficient for large volumes and prone to timeouts. The recommended approach is to write data to S3 first and then use the Redshift COPY command, which is optimized for bulk loading and more reliable. Option A is incorrect because increasing the cluster size addresses capacity but not the inherent inefficiency of JDBC writes. Option C is incorrect because increasing the timeout only masks the issue without resolving the underlying problem. Option D is incorrect because AWS DMS is designed for database migration, not for ETL job processing within Glue.

Answer

Increase the Redshift cluster size to 4 nodes.

Answer

Increase the Glue job timeout to 24 hours.

Answer

Use AWS Database Migration Service (DMS) to load data into Redshift.

Question 8

A data engineer is building a data pipeline that ingests streaming data from Amazon Kinesis Data Streams, transforms the data using AWS Lambda, and stores the results in Amazon S3. The engineer needs to ensure that each record is processed exactly once and in order. Which TWO approaches should the engineer consider? (Choose TWO.)

Accepted Answer

Configure the Lambda function with a reserved concurrency of 1 and a batch size of 1 to process records sequentially.. To achieve exactly-once processing and in-order delivery from Kinesis Data Streams with Lambda, the engineer should consider:
- Option A: Configuring reserved concurrency of 1 and batch size of 1 ensures that each Lambda invocation processes one record at a time and that records within a shard are processed sequentially, maintaining ordering.
- Option B: Using the Kinesis Producer Library (KPL) (or relying on the sequence number already assigned to each record) allows the consumer Lambda to implement idempotent processing by checking the sequence number against a deduplication store (e.g., DynamoDB with conditional writes), thereby achieving exactly-once semantics.
Combining these approaches helps meet both requirements.

Option C is not correct because integrating SQS FIFO queues between Kinesis and Lambda adds unnecessary complexity and does not directly preserve the ordering across shards without additional mapping; also, Kinesis Data Streams does not natively fan-out to SQS FIFO.
Option D is incorrect because S3 Event Notifications are asynchronous and do not address ordering or exactly-once processing in the pipeline.
Option E is incorrect because Kinesis Data Firehose is designed for near-real-time delivery and can introduce duplicate records; it does not guarantee exactly-once processing.

Answer

Use Amazon SQS FIFO queues to decouple Kinesis and Lambda, ensuring ordering and exactly-once delivery.

Answer

Enable S3 Event Notifications to trigger Lambda for each object.

Answer

Use Kinesis Data Firehose to buffer and deliver data to S3, then use Lambda to process.

Question 9

A company runs a data lake on Amazon S3 with AWS Glue for ETL. The data science team needs to train machine learning models on historical data, but they are concerned about data quality issues such as missing values, duplicates, and outliers. The team wants to build a data quality monitoring solution that automatically detects anomalies and alerts the data engineering team. Which THREE steps should the team take to implement this solution? (Choose THREE.)

Accepted Answer

Use AWS Glue DataBrew to create data quality rules that check for missing values, duplicates, and outliers, and schedule them to run regularly.. AWS Glue DataBrew provides built-in data quality checks and profiling. Option C is correct because Amazon CloudWatch alarms can be set on data quality metrics to trigger SNS notifications when thresholds are breached. Option D is correct because the Deequ library can be implemented on Amazon EMR to compute data quality metrics and store them in CloudWatch. Option B is incorrect because Amazon Kinesis Data Analytics is used for real-time streaming analytics, not for batch data quality checks on historical data. Option E is incorrect because Amazon SageMaker Processing jobs are for model training and data processing, not specifically for data quality monitoring.

Answer

Use Amazon Kinesis Data Analytics to continuously monitor streaming data for data quality issues.

Answer

Use Amazon SageMaker Processing jobs to run custom data quality scripts and store results in SageMaker Experiments.

Question 10

A data science team uses Amazon SageMaker to train models on a large dataset stored in S3. The dataset is 500 GB in CSV format and is updated daily. The team wants to optimize data loading for training jobs to reduce I/O wait time. Which data ingestion strategy is MOST effective?

Accepted Answer

Use SageMaker Pipe input mode to stream data directly from S3.. SageMaker Pipe input mode streams data directly from S3 to the training algorithm without writing to the instance's EBS volume, eliminating disk I/O bottlenecks. This is especially effective for large datasets (500 GB) that are updated daily, as it reduces startup time and avoids the need to download the entire dataset before training begins.

Answer

Use SageMaker File input mode and increase the EBS volume size to 1 TB.

Answer

Convert the CSV files to Parquet format and use File input mode.

Answer

Load the data into an Amazon EFS file system and mount it to the training instance.

Question 11

A company uses Amazon Kinesis Data Streams to ingest real-time clickstream data from a website. The data is consumed by a Lambda function that writes records to an S3 bucket. Recently, the number of shards was increased from 2 to 4 to handle higher throughput. After the change, the Lambda function started processing records with increased latency and some records were being written out of order. What is the MOST likely cause?

Accepted Answer

The partition key used by the producer does not ensure that related records go to the same shard after resharding.. After resharding from 2 to 4 shards, the mapping of partition keys to shards changes. If the producer does not use a partition key that ensures related records (e.g., same user session) are routed to the same shard, records that were previously ordered within a shard may now be split across multiple shards. Since the Lambda consumer processes shards independently, records from the same logical sequence can arrive out of order, and the increased shard count can also cause higher latency if the consumer is not properly parallelized.

Answer

The S3 bucket is not configured with versioning, causing overwrites.

Answer

The Lambda function is reading from the oldest sequence number, causing high IteratorAgeSeconds.

Answer

The Lambda function’s reserved concurrency is too low for the increased shard count.

Question 12

A data engineer needs to transform large CSV files stored in S3 into Parquet format and load them into a data warehouse for analysis. The transformation must be cost-effective and serverless. Which AWS service should be used?

Accepted Answer

AWS Glue. AWS Glue is the correct choice because it provides a fully managed, serverless ETL service that can automatically convert CSV files from S3 into Parquet format using its built-in Spark engine. It is cost-effective as you only pay for the resources consumed during the job execution, and it integrates directly with data warehouses like Amazon Redshift for loading transformed data.

Answer

Amazon Athena

Answer

Amazon EMR with Spark

Answer

AWS Data Pipeline

Question 13

A company uses Amazon Kinesis Data Firehose to deliver streaming data to an S3 bucket. The data is JSON and must be partitioned by year, month, and day. The delivery stream is configured with a buffer interval of 60 seconds and buffer size of 5 MB. The data producer sends about 1 MB per second. The data is arriving in S3 but the partitions are not being created as expected. What is the MOST likely reason?

Accepted Answer

The delivery stream does not have dynamic partitioning enabled with the appropriate custom prefix.. Kinesis Data Firehose requires dynamic partitioning to be explicitly enabled and configured with a custom prefix (e.g., 'year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/') to automatically partition data by year, month, and day. Without this setting, Firehose writes all data to a single S3 prefix, ignoring the desired partition structure.

Answer

The data is encrypted with AWS KMS and Firehose cannot write to encrypted buckets.

Answer

The buffer interval is too short for the data volume, causing incomplete records.

Answer

The S3 bucket has versioning enabled, which prevents partitioning.

Question 14

An ML team is building a recommendation system. The training data includes user-item interactions stored in Amazon DynamoDB. The team wants to export this data to S3 in Parquet format for use with Amazon SageMaker. The export should be incremental (only new or changed records) and run daily. Which approach meets these requirements with MINIMAL operational overhead?

Accepted Answer

Use DynamoDB Streams with AWS Lambda to write changes to S3 in Parquet format.. DynamoDB Streams capture every change (insert, update, delete) in near real-time, and AWS Lambda can process these events to write only the changed records to S3 in Parquet format. This approach provides incremental, daily exports with minimal operational overhead, as it is fully serverless and requires no infrastructure management.

Answer

Use the DynamoDB Export to S3 feature and schedule it daily with AWS Glue.

Answer

Use a script that scans the DynamoDB table and filters by last updated timestamp.

Answer

Set up an Amazon EMR cluster running Spark jobs to read DynamoDB and write to S3.

Question 15

A data scientist uses Amazon SageMaker to train a model. The training dataset is 10 GB and stored in S3. The training job uses a ml.m5.large instance. The data must be available on the local file system during training. Which input mode should be used?

Accepted Answer

File input mode. File input mode is correct because it downloads the entire training dataset from S3 to the local file system of the ml.m5.large instance before training begins, ensuring the data is available locally as required. This mode is suitable for datasets up to 10 GB, as the instance's local storage (typically 8 GB for ml.m5.large) may be insufficient, but SageMaker uses the instance's Amazon EBS volume (up to 512 GB) for file input mode, making it viable.

Answer

Local input mode

Answer

Batch input mode

Answer

Pipe input mode