MLS-C01 Data Engineering Practice Test 14 — 15 Questions

Question 1

A company is using Amazon Kinesis Data Streams to ingest clickstream data. The data is consumed by a fleet of EC2 instances running a custom consumer application. The consumer is falling behind and the shard iterator age is increasing. Which TWO actions should the data engineer take to improve consumer performance? (Choose TWO.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the number of shards in the stream. Increasing the number of shards in the stream directly increases the total read capacity of the Kinesis Data Stream. Each shard provides a fixed read throughput of 2 MB/s (or 5 read transactions per second), so adding shards allows the consumer fleet to parallelize processing across more data partitions, reducing the backlog and shard iterator age.

Answer

Decrease the data retention period

Answer

Use an AWS Lambda function to process the data

Answer

Switch to the Kinesis Client Library (KCL)

Question 2

A data engineer is designing a data pipeline to process streaming data from Amazon Kinesis Data Streams and store the results in Amazon S3 in Parquet format. The data must be available for querying in Amazon Athena within minutes of arrival. Which THREE services should be used together? (Choose THREE.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose can directly ingest streaming data from Kinesis Data Streams, convert it to Parquet, and deliver to S3 with low latency. Amazon Kinesis Data Analytics can process and analyze the stream in real-time (e.g., aggregations or filtering) before sending to Firehose. AWS Glue provides a data catalog for the S3 data, making it queryable by Athena. Together, these three services enable near-real-time querying of streaming data in Athena.

Answer

Amazon EMR

Answer

Amazon Redshift

Question 3

A company wants to centralize logging from multiple AWS accounts and on-premises servers. The logs must be stored cost-effectively and be searchable. Which TWO services should be used? (Choose TWO.)

Accepted Answer

Amazon Athena. Amazon S3 (option E) provides a cost-effective, durable storage layer for centralized logs, while Amazon Athena (option C) enables serverless SQL-based searching directly on the log data stored in S3 without needing to load it into a separate database. Together, they allow you to store logs cheaply and query them on demand, meeting the requirements of cost-effectiveness and searchability.

Answer

Amazon CloudWatch Logs

Answer

Amazon Redshift

Answer

Amazon Kinesis Data Streams

Question 4

A data engineer is troubleshooting an AWS Glue job that reads from an S3 bucket and writes to another S3 bucket. The job fails with an 'Access Denied' error when trying to write to the output bucket. The IAM policy attached to the Glue service role is shown. What is the MOST likely cause of the failure?

Accepted Answer

The output bucket is not listed in the Resource of the IAM policy. The IAM policy attached to the Glue service role explicitly lists the output bucket in the Resource field. If the output bucket is not listed, the Glue job will receive an 'Access Denied' error when attempting to write to it, because the policy does not grant the necessary s3:PutObject permission for that bucket. This is the most direct cause of the failure.

Answer

The user who runs the job does not have S3 permissions

Answer

The Glue job role does not have permissions to start a job run

Answer

The S3 bucket policy denies access to the Glue service

Question 5

A data engineer runs the above CLI command and sees that the bucket contains many small Parquet files (1 MB each) under the prefix. When querying this data with Athena, the query performance is poor and costs are high. Which approach would MOST improve performance and reduce cost?

Accepted Answer

Consolidate the small files into fewer, larger Parquet files. C is correct because consolidating many small Parquet files into fewer, larger files (e.g., 128–256 MB each) reduces the overhead of Amazon Athena's file listing and metadata operations, and improves compression and predicate pushdown efficiency. Parquet is a columnar format optimized for analytics, so keeping it while reducing file count directly addresses the root cause of poor performance and high cost.

Answer

Convert the files to JSON format

Answer

Convert the files to CSV format

Answer

Add more partitions by including hour in the prefix

Question 6

A company runs a data pipeline using AWS Glue ETL jobs that process about 10 TB of data daily from Amazon S3. The jobs are triggered by a schedule and write results to a separate S3 bucket. Recently, the jobs have been taking longer to complete, and the data engineering team has observed that the number of files in the source bucket has increased significantly, from thousands to millions of small files (each about 100 KB). The Glue jobs are configured to use the 'Group Files' option, but performance is still poor. The team needs to improve the job performance without changing the source data generation process. Which course of action should the team take?

Accepted Answer

Create a separate Glue job that runs before the main job to consolidate small files into larger ones in the source bucket. The main performance bottleneck is the large number of small files, which causes high overhead in reading metadata and opening files. Option D addresses this by creating a separate Glue job that consolidates small files into larger files (e.g., 100 MB) before the main ETL job runs, reducing the file count and improving read performance. Option A is incorrect because increasing DPUs may provide more parallelism but does not solve the underlying small-file problem; the overhead of opening millions of files remains. Option B is incorrect because switching to Amazon EMR with Spark would still encounter the same small-file issue unless additional measures (like coalesce or file compaction) are taken, which Option D already provides. Option C is incorrect because AWS Lambda has limitations on execution duration and memory, making it impractical to pre-process millions of small files efficiently.

Answer

Increase the number of DPUs allocated to the existing Glue job

Answer

Switch the ETL processing to Amazon EMR with Spark

Answer

Use AWS Lambda to pre-process the files and combine them

Question 7

An e-commerce company uses Amazon Kinesis Data Firehose to deliver clickstream data to an Amazon S3 bucket. The data is then queried using Amazon Athena. The marketing team wants to run daily reports that aggregate click events by product ID. However, the reports are slow because Athena scans the entire dataset each time. The data is partitioned by date (e.g., s3://bucket/clickstream/2023/01/01/). The product ID is a column within the data. The data engineering team wants to improve query performance without moving the data to another service. Which approach should the team take?

Accepted Answer

Repartition the data by product ID in addition to date. Repartition the data by product ID in addition to date. This adds a partition level for product ID, so queries that filter on product ID will only scan the relevant partitions. Option A (convert to Parquet) reduces data scanned due to columnar storage and compression, but without partition pruning on product ID, Athena would still scan all partitions for each query. Option B (Redshift Spectrum) would still require scanning data, and involves additional service complexity. Option C (create a view) does not change physical storage; it only provides a logical filter, but Athena still scans all underlying data. Therefore, repartitioning by product ID provides the most direct improvement for queries filtering by product ID.

Answer

Convert the data from JSON to Parquet format

Answer

Use Amazon Redshift Spectrum to query the data

Answer

Create a view in Athena that filters by product ID

Question 8

A startup is building a data pipeline that ingests data from multiple sources into an Amazon S3 data lake. The data includes CSV files from legacy systems, JSON from web APIs, and Avro from mobile apps. The data must be transformed into Parquet format and cataloged for querying with Amazon Athena. The pipeline must be serverless and minimize operational overhead. The team has decided to use AWS Glue for ETL and cataloging. However, they are concerned about the cost of running Glue jobs continuously. The data arrives in small batches every 10 minutes. Which approach should the team use to minimize cost while meeting the requirements?

Accepted Answer

Use scheduled Glue jobs to process the data every hour, consolidating multiple batches. Using scheduled Glue jobs every hour to process accumulated data reduces the number of job runs and associated costs, while still providing near-real-time processing (within the hour). Option A is wrong because Lambda functions have limited execution time and memory, making them unsuitable for large-scale transformations. Option B is wrong because Kinesis Data Firehose can directly deliver streaming data to S3, but it does not handle all source formats natively (e.g., CSV, Avro) and additional transformation may be needed. Option D is wrong because a single daily Glue job introduces too much latency for batch arrivals every 10 minutes.

Answer

Use AWS Lambda functions to transform each file upon arrival and store as Parquet

Answer

Use Amazon Kinesis Data Firehose to stream data directly into S3 and use Glue to catalog it

Answer

Use a single daily Glue job to process all data at once

Question 9

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams and AWS Lambda. The Lambda function processes records and writes results to Amazon S3. The engineer notices that the Lambda function is experiencing throttling and some records are being dropped. Which TWO actions should the engineer take to improve the reliability of the pipeline?

Accepted Answer

Increase the number of shards in the Kinesis data stream.. Increasing the number of shards in the Kinesis data stream directly increases the stream's throughput capacity. Each shard supports up to 1 MB/s write and 2 MB/s read, so more shards allow the stream to handle higher data volumes, reducing the likelihood of throttling and dropped records at the stream level.

Answer

Add a Dead Letter Queue to the Lambda function to capture failed records.

Answer

Decrease the batch size in the Lambda event source mapping.

Answer

Increase the Kinesis stream's retention period to 7 days.

Question 10

A machine learning team is using Amazon SageMaker to train a model on a dataset stored in S3. The training job reads data from S3 using Pipe input mode, but the training is slow. The team wants to improve data throughput. Which THREE actions should they take?

Accepted Answer

Mount the S3 bucket using an S3 file system and use File mode with a larger instance type.. Mounting an S3 bucket using an S3 file system (e.g., via mount-s3 or s3fs) and switching to File mode allows the training instance to access data as local files, eliminating the overhead of streaming decompression and per-record parsing inherent in Pipe mode. Using a larger instance type provides more network bandwidth and CPU resources to handle the file I/O, directly improving data throughput for large datasets.

Answer

Enable S3 Transfer Acceleration on the bucket.

Answer

Use Amazon ElastiCache to cache the training data.

Question 11

A data engineer is designing a data lake on Amazon S3. The data is collected from IoT devices and is highly variable in volume. The engineer needs to ensure that the data is ingested reliably and can be processed in near real-time. Which AWS service should be used to ingest the data into the data lake?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to reliably load streaming data into data lakes on Amazon S3 with near-real-time latency (typically 60 seconds). It automatically handles scaling to accommodate highly variable IoT data volumes, provides built-in data transformation and compression, and requires no manual shard management or consumer code, making it ideal for ingestion into S3-based data lakes.

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Answer

Amazon Simple Queue Service (SQS)

Question 12

A data engineer has attached the above IAM policy to an IAM role used by an AWS Glue ETL job. The job reads from and writes to 'my-data-bucket'. The job is failing with an Access Denied error. What is the most likely cause?

Accepted Answer

The condition restricts access to a specific IP range that does not include the AWS Glue service IPs.. The IAM policy includes a condition that restricts access to requests originating from a specific IP address range. AWS Glue ETL jobs run on ephemeral compute resources that use a dynamic pool of IP addresses, which are not guaranteed to fall within any fixed customer-managed IP range. Therefore, the condition causes the Access Denied error because the Glue service IPs are not within the allowed range.

Answer

The IAM role needs to have s3:ListBucket permission.

Answer

The IAM role does not have permission to list the bucket.

Answer

The resource ARN should include the bucket itself, not just the objects.

Question 13

A machine learning engineer is using Amazon SageMaker to train a model. The training dataset is 2 TB and is stored in Amazon S3. The engineer wants to reduce the training time by improving data loading performance. Which data ingestion mode should be used?

Accepted Answer

Pipe mode. Pipe mode is the correct choice because it streams data directly from Amazon S3 to the training container via a Unix named pipe, bypassing disk writes and reducing I/O latency. For a 2 TB dataset, this eliminates the bottleneck of downloading data to the training instance's Amazon Elastic Block Store (EBS) volume, significantly improving data loading performance and reducing overall training time.

Answer

Incremental mode

Answer

File mode

Answer

Fast file mode

Question 14

A data engineer needs to transform a large dataset stored in Amazon S3 using Apache Spark. The engineer wants to minimize costs and avoid managing infrastructure. Which AWS service should be used?

Accepted Answer

AWS Glue. AWS Glue is the optimal choice because it provides a serverless Apache Spark environment, fully managed, that allows the engineer to run Spark transformations without provisioning or managing clusters. This meets both requirements: using Apache Spark as specified, and minimizing costs and infrastructure management through its pay‑as‑you‑go, serverless model. Amazon Athena is a SQL query service and does not execute Apache Spark code. Amazon EMR provides Spark but typically requires cluster management (unless using EMR Serverless, which is not as straightforward as Glue for serverless Spark). Amazon SageMaker is focused on machine learning, not general ETL transformations.

Answer

Amazon Athena

Answer

Amazon SageMaker

Answer

Amazon EMR

Question 15

A data engineer is investigating a slow Athena query on a partitioned table. The table is partitioned by year, month, and day, and the data is stored in S3 with the prefix pattern 'raw/YYYY/MM/DD/'. The engineer runs the above CLI command and sees that there are many small files. Which action would most improve query performance?

Accepted Answer

Use S3DistCp to coalesce files into fewer, larger files.. The core issue is that the Athena query is slow due to many small files, which increases the overhead of S3 LIST operations and task scheduling in the Presto/Trino engine underlying Athena. Coalescing these small files into fewer, larger files with S3DistCp reduces the number of S3 GET requests and minimizes the scheduling overhead, directly improving query throughput. This is a classic small-files problem, not a data format or partitioning issue.

Answer

Convert the data to columnar format like Parquet or ORC.

Answer

Increase the number of partitions in the Athena DDL.

Answer

Add more partitions to reduce the amount of data scanned per query.