MLS-C01 Data Engineering Practice Test 13 — 15 Questions

Question 1

A company is using Amazon Kinesis Data Streams to ingest clickstream data. The data is consumed by a fleet of EC2 instances running a custom consumer application. The consumer is falling behind and the shard iterator age is increasing. Which TWO actions should the data engineer take to improve consumer performance? (Choose TWO.)

Accepted Answer

Increase the number of shards in the stream. Increasing the number of shards increases parallelism, and using enhanced fan-out allows each consumer to have its own read throughput. Option B is wrong because decreasing retention period does not improve consumer performance. Option D is wrong because using a Lambda function may not help if the bottleneck is shard throughput. Option E is wrong because using KCL (Kinesis Client Library) is already standard; not a direct fix.

Answer

Decrease the data retention period

Answer

Use an AWS Lambda function to process the data

Answer

Switch to the Kinesis Client Library (KCL)

Question 2

A data engineer is designing a data pipeline to process streaming data from Amazon Kinesis Data Streams and store the results in Amazon S3 in Parquet format. The data must be available for querying in Amazon Athena within minutes of arrival. Which THREE services should be used together? (Choose THREE.)

Accepted Answer

Amazon Kinesis Data Firehose. Kinesis Data Firehose can write data to S3 in Parquet format with near-real-time delivery. AWS Glue provides the Data Catalog for table metadata, and Athena queries the data. Option A (Kinesis Data Analytics) is for real-time analytics on streams, not for storage. Option C (EMR) is for batch processing, not streaming. Option E (Redshift) is for data warehousing, not immediate S3 querying.

Answer

Amazon EMR

Answer

Amazon Redshift

Answer

Amazon Kinesis Data Analytics

Question 3

A company wants to centralize logging from multiple AWS accounts and on-premises servers. The logs must be stored cost-effectively and be searchable. Which TWO services should be used? (Choose TWO.)

Accepted Answer

Amazon Athena. Amazon S3 is a cost-effective storage for log archives, and Amazon Athena allows querying the logs directly on S3 without loading into a database. Option A (CloudWatch Logs) is for real-time monitoring, not long-term storage. Option C (Redshift) is more expensive. Option E (Kinesis) is for streaming, not storage.

Answer

Amazon CloudWatch Logs

Answer

Amazon Redshift

Answer

Amazon Kinesis Data Streams

Question 4

A data engineer is troubleshooting an AWS Glue job that reads from an S3 bucket and writes to another S3 bucket. The job fails with an 'Access Denied' error when trying to write to the output bucket. The IAM policy attached to the Glue service role is shown. What is the MOST likely cause of the failure?

Accepted Answer

The output bucket is not listed in the Resource of the IAM policy. The policy only allows s3:GetObject and s3:PutObject on 'example-bucket', but the job is writing to a different bucket. The Glue job needs permissions on the output bucket as well. Option A is wrong because the job role has permissions for Glue actions. Option B is wrong because S3 permissions are for a specific bucket. Option D is wrong because the job role is separate from the user role.

Answer

The user who runs the job does not have S3 permissions

Answer

The Glue job role does not have permissions to start a job run

Answer

The S3 bucket policy denies access to the Glue service

Question 5

A data engineer runs the above CLI command and sees that the bucket contains many small Parquet files (1 MB each) under the prefix. When querying this data with Athena, the query performance is poor and costs are high. Which approach would MOST improve performance and reduce cost?

Accepted Answer

Consolidate the small files into fewer, larger Parquet files. Consolidating small files into larger files (e.g., 100 MB) reduces the overhead of reading many small files in Athena. Option A is wrong because CSV is not columnar and would not improve performance. Option B is wrong because converting to JSON is similar. Option D is wrong because adding more partitions may create even more files.

Answer

Convert the files to JSON format

Answer

Convert the files to CSV format

Answer

Add more partitions by including hour in the prefix

Question 6

A company runs a data pipeline using AWS Glue ETL jobs that process about 10 TB of data daily from Amazon S3. The jobs are triggered by a schedule and write results to a separate S3 bucket. Recently, the jobs have been taking longer to complete, and the data engineering team has observed that the number of files in the source bucket has increased significantly, from thousands to millions of small files (each about 100 KB). The Glue jobs are configured to use the 'Group Files' option, but performance is still poor. The team needs to improve the job performance without changing the source data generation process. Which course of action should the team take?

Accepted Answer

Create a separate Glue job that runs before the main job to consolidate small files into larger ones in the source bucket. Using an AWS Glue job to periodically compact small files into larger files (e.g., 100 MB) before the main ETL job runs will reduce the overhead of reading millions of small files. Option B is wrong because increasing DPUs may help but does not address the root cause of many small files. Option C is wrong because using Spark on EMR may still suffer from small file issue. Option D is wrong because Lambda has limitations on processing large numbers of files.

Answer

Increase the number of DPUs allocated to the existing Glue job

Answer

Switch the ETL processing to Amazon EMR with Spark

Answer

Use AWS Lambda to pre-process the files and combine them

Question 7

An e-commerce company uses Amazon Kinesis Data Firehose to deliver clickstream data to an Amazon S3 bucket. The data is then queried using Amazon Athena. The marketing team wants to run daily reports that aggregate click events by product ID. However, the reports are slow because Athena scans the entire dataset each time. The data is partitioned by date (e.g., s3://bucket/clickstream/2023/01/01/). The product ID is a column within the data. The data engineering team wants to improve query performance without moving the data to another service. Which approach should the team take?

Accepted Answer

Repartition the data by product ID in addition to date. Partitioning by product ID would allow Athena to prune partitions for queries filtering by product ID. Option A is wrong because converting to Parquet alone may improve but does not eliminate full scan. Option C is wrong because creating a view does not change physical storage. Option D is wrong because Redshift Spectrum still requires scanning.

Answer

Convert the data from JSON to Parquet format

Answer

Use Amazon Redshift Spectrum to query the data

Answer

Create a view in Athena that filters by product ID

Question 8

A startup is building a data pipeline that ingests data from multiple sources into an Amazon S3 data lake. The data includes CSV files from legacy systems, JSON from web APIs, and Avro from mobile apps. The data must be transformed into Parquet format and cataloged for querying with Amazon Athena. The pipeline must be serverless and minimize operational overhead. The team has decided to use AWS Glue for ETL and cataloging. However, they are concerned about the cost of running Glue jobs continuously. The data arrives in small batches every 10 minutes. Which approach should the team use to minimize cost while meeting the requirements?

Accepted Answer

Use scheduled Glue jobs to process the data every hour, consolidating multiple batches. Triggering Glue jobs on schedule (e.g., every hour) to process accumulated data reduces the number of job runs and cost, while still meeting near-real-time needs. Option A is wrong because continuous streaming with Firehose may not handle all source formats. Option C is wrong because using Lambda for transformation is limited by timeout and memory. Option D is wrong because running a single daily job may introduce too much latency.

Answer

Use AWS Lambda functions to transform each file upon arrival and store as Parquet

Answer

Use Amazon Kinesis Data Firehose to stream data directly into S3 and use Glue to catalog it

Answer

Use a single daily Glue job to process all data at once

Question 9

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams and AWS Lambda. The Lambda function processes records and writes results to Amazon S3. The engineer notices that the Lambda function is experiencing throttling and some records are being dropped. Which TWO actions should the engineer take to improve the reliability of the pipeline?

Accepted Answer

Increase the number of shards in the Kinesis data stream.. Increasing the number of shards in the Kinesis stream increases throughput and reduces throttling. Setting the Lambda function's reserved concurrency ensures it has sufficient capacity to handle the incoming records.

Answer

Add a Dead Letter Queue to the Lambda function to capture failed records.

Answer

Decrease the batch size in the Lambda event source mapping.

Answer

Increase the Kinesis stream's retention period to 7 days.

Question 10

A machine learning team is using Amazon SageMaker to train a model on a dataset stored in S3. The training job reads data from S3 using Pipe input mode, but the training is slow. The team wants to improve data throughput. Which THREE actions should they take?

Accepted Answer

Mount the S3 bucket using an S3 file system and use File mode with a larger instance type.. Using Amazon S3 with a VPC endpoint improves network performance. Using Amazon EFS as a data source can provide higher throughput for sequential access. Using SageMaker's File mode with a larger instance type with more network bandwidth can also improve throughput.

Answer

Enable S3 Transfer Acceleration on the bucket.

Answer

Use Amazon ElastiCache to cache the training data.

Question 11

A data engineer is designing a data lake on Amazon S3. The data is collected from IoT devices and is highly variable in volume. The engineer needs to ensure that the data is ingested reliably and can be processed in near real-time. Which AWS service should be used to ingest the data into the data lake?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the best choice for reliably ingesting streaming data into S3 with near real-time delivery, automatic scaling, and optional transformations.

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Answer

Amazon Simple Queue Service (SQS)

Question 12

A data engineer has attached the above IAM policy to an IAM role used by an AWS Glue ETL job. The job reads from and writes to 'my-data-bucket'. The job is failing with an Access Denied error. What is the most likely cause?

Accepted Answer

The condition restricts access to a specific IP range that does not include the AWS Glue service IPs.. The policy restricts access to requests originating from the IP range 10.0.0.0/24. AWS Glue jobs run in a VPC that uses private IPs, but the source IP condition is evaluated based on the IP address of the Glue service principal, which is not within that range. The condition should be removed or modified to allow access from the Glue service.

Answer

The IAM role needs to have s3:ListBucket permission.

Answer

The IAM role does not have permission to list the bucket.

Answer

The resource ARN should include the bucket itself, not just the objects.

Question 13

A machine learning engineer is using Amazon SageMaker to train a model. The training dataset is 2 TB and is stored in Amazon S3. The engineer wants to reduce the training time by improving data loading performance. Which data ingestion mode should be used?

Accepted Answer

Pipe mode. SageMaker Pipe mode streams data directly from S3 to the training algorithm, which can reduce training time by overlapping data loading and training, especially for large datasets.

Answer

Incremental mode

Answer

File mode

Answer

Fast file mode

Question 14

A data engineer needs to transform a large dataset stored in Amazon S3 using Apache Spark. The engineer wants to minimize costs and avoid managing infrastructure. Which AWS service should be used?

Accepted Answer

AWS Glue. AWS Glue is a serverless Spark ETL service that can process large datasets stored in S3 without requiring infrastructure management.

Answer

Amazon Athena

Answer

Amazon SageMaker

Answer

Amazon EMR

Question 15

A data engineer is investigating a slow Athena query on a partitioned table. The table is partitioned by year, month, and day, and the data is stored in S3 with the prefix pattern 'raw/YYYY/MM/DD/'. The engineer runs the above CLI command and sees that there are many small files. Which action would most improve query performance?

Accepted Answer

Use S3DistCp to coalesce files into fewer, larger files.. Athena performs best with larger files. Consolidating small files into fewer, larger files (e.g., using S3DistCp or Glue ETL) reduces the overhead of reading many small files and improves query performance.

Answer

Convert the data to columnar format like Parquet or ORC.

Answer

Increase the number of partitions in the Athena DDL.

Answer

Add more partitions to reduce the amount of data scanned per query.