MLS-C01 Data Engineering Practice Test 16 — 15 Questions

Question 1

A company ingests application logs into an Amazon S3 bucket under the prefix logs/year=2024/month=03/day=15/. Analysts run Athena queries that almost always filter on a single day and a specific application name. Queries currently scan the entire month of data. The team wants Athena to read only the relevant day and application without changing the raw log files. What should the data engineer do?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Partition the table in the AWS Glue Data Catalog by year, month, day, and application, and include those columns in the WHERE clause of the queries.. Partitioning is the primary mechanism for reducing Athena scan volume on S3 data. When partition keys are registered in the Data Catalog and referenced in the query predicate, Athena lists and reads only the matching prefixes. Because the existing key layout already encodes year, month, and day, registering partitions is inexpensive and immediately restricts scans to the requested day and application.

Answer

Enable S3 Select on the bucket so Athena pushes the day filter down to the S3 storage layer.

Answer

Create an Athena workgroup with a per-query data usage control limit that matches the expected daily scan size.

Answer

Convert the log files from text to ORC and rely on ORC predicate pushdown alone to skip irrelevant days.

Question 2

A machine learning team needs to prepare a large dataset for SageMaker training. The raw data consists of millions of small JSON files in Amazon S3, and training jobs are slow because of high per-file overhead. The team wants to consolidate the data into fewer, larger Parquet files while preserving all records and minimizing cost. Which two actions should the data engineer take to meet these requirements? (Choose two.)

Accepted Answer

Use Amazon EMR with Apache Spark to read the JSON objects, coalesce the partitions, and write Parquet to a new S3 prefix.. Both AWS Glue and Amazon EMR run Apache Spark, which can read the many small JSON files in parallel, repartition or coalesce the data, and write Parquet output. That consolidates files and converts format, reducing per-file overhead for SageMaker training while preserving all records. Transfer Acceleration, lifecycle transitions, and Athena CTAS do not reliably achieve controlled file consolidation.

Answer

Use Amazon Athena to run a CREATE TABLE AS SELECT statement that writes the JSON data into a single Parquet file in S3.

Answer

Create an S3 Lifecycle policy that transitions the JSON objects to S3 Glacier Instant Retrieval after 30 days to reduce storage cost.

Answer

Enable S3 Transfer Acceleration on the source bucket and run an AWS Glue crawler to catalog the JSON files before training.

Question 3

A machine learning engineer is using Amazon SageMaker Ground Truth to label a dataset of images. The labeling job is configured with a private workforce. The engineer notices that some workers are submitting labels that are inconsistent with the majority. The engineer wants to improve the quality of the labels by identifying and removing low-quality workers from the workforce without affecting the entire labeling job. Which action should the engineer take?

Accepted Answer

Enable the 'Worker quality' feature in the labeling job, which automatically identifies and removes workers whose accuracy falls below a specified threshold based on gold standard annotations.. To automatically identify and remove low-quality workers in a SageMaker Ground Truth labeling job, the engineer should use the built-in worker quality feature based on gold standard annotations. By including gold standard data, Ground Truth can evaluate each worker's accuracy and automatically remove those who fall below a configured threshold. This ensures high-quality labels without manual intervention, improving the overall dataset quality efficiently.

Answer

Use Amazon A2I with a human review workflow to review all labels submitted by workers and reject those that are incorrect, then block the workers from further tasks.

Answer

Use the Amazon SageMaker Ground Truth console to view the worker statistics and manually remove the low-quality workers from the private workforce.

Answer

Configure the labeling job to use annotation consolidation with a majority vote, and set up an Amazon CloudWatch alarm to notify when worker accuracy drops below a threshold, then manually remove workers.

Question 4

A data scientist is using Amazon SageMaker to train a model on a dataset stored in Amazon S3. The dataset is 500 GB and is stored as a single CSV file. The training job is taking a long time to start because SageMaker is downloading the entire dataset to the training instance's Amazon EBS volume before training begins. The data scientist wants to reduce the startup time without changing the data format. Which solution will meet this requirement?

Accepted Answer

Use Amazon FSx for Lustre as the data source for the training job.. Amazon FSx for Lustre provides a POSIX-compliant file system that can be linked to an S3 bucket. SageMaker training jobs can mount the FSx for Lustre file system, allowing data to be read on demand without downloading the entire dataset to EBS. This significantly reduces startup time for large datasets, especially when the data format remains unchanged.

Answer

Increase the size of the Amazon EBS volume attached to the training instance.

Answer

Use Pipe mode instead of File mode.

Answer

Split the CSV file into multiple smaller files in S3.

Question 5

A data engineer is storing large volumes of semi-structured JSON data in Amazon S3. The data will be queried using Amazon Athena. The engineer wants to optimize query performance and reduce costs. Which action should the engineer take?

Accepted Answer

Convert the JSON data to Apache Parquet format and partition the data by commonly queried fields.. For large datasets queried by Athena, converting data to a columnar format like Parquet and partitioning by commonly filtered columns significantly improves performance and reduces cost. Parquet's columnar storage allows Athena to read only the columns needed, and partitioning limits the data scanned. This combination is a standard optimization technique for Athena. Other options either do not improve performance or are not applicable to Athena.

Answer

Store the JSON data in a single large file to minimize the number of S3 objects.

Answer

Enable S3 Transfer Acceleration to speed up data retrieval by Athena.

Answer

Use Amazon S3 Select to query the JSON data directly without converting formats.

Question 6

A data engineer is building a data pipeline that ingests data from an Amazon Kinesis Data Stream. The engineer needs to store the raw data in Amazon S3 for archival and also process it in real time using an AWS Lambda function. The Lambda function must be able to process each record exactly once, even if the function fails and retries. Which configuration should the engineer use to achieve exactly-once processing?

Accepted Answer

Configure the Lambda event source mapping to use the 'Latest' starting position, set a batch size of 1, and enable the 'ReportBatchItemFailures' feature to only retry failed records.. Achieving exactly-once processing with Lambda and Kinesis is challenging because Lambda automatically retries failed batches, which can cause duplicates. The best approach is to minimize batch size to 1 and use the ReportBatchItemFailures feature, which allows the function to indicate which specific records failed, so only those are retried. This reduces the scope of duplicates. However, true exactly-once requires the function to be idempotent, but among the given options, this configuration is the most aligned with the goal.

Answer

Use the Kinesis Client Library (KCL) to process the stream with a Lambda function, and enable checkpointing after each record is processed.

Answer

Configure the Lambda event source mapping to start reading from the TRIM_HORIZON position, and enable a dead-letter queue (DLQ) for failed records.

Answer

Configure the Lambda function as a Kinesis Data Streams consumer with a batch size of 1 and set the maximum retry attempts to 0.

Question 7

A machine learning team stores training data in an Amazon S3 bucket that contains 4 million small JSON objects, each under 100 KB. A SageMaker training job that reads the objects directly from S3 is spending most of its time on per-object request overhead, and the team wants to reduce the number of S3 GET requests without changing the data content. Which solution will meet these requirements with the LEAST operational effort?

Accepted Answer

Run an AWS Glue ETL job that reads the JSON objects and writes them out as larger Parquet files in a new S3 prefix, then point the training job at that prefix.. Reading millions of tiny S3 objects forces one GET request per object, which dominates runtime. Consolidating the objects into a smaller number of large, columnar Parquet files removes that overhead and reduces bytes scanned by the training job. A managed AWS Glue ETL job performs the conversion without standing up servers, which keeps operational effort low while directly addressing the request-count bottleneck.

Answer

Create an Amazon EFS file system, copy the objects onto it, and mount it into the SageMaker training job.

Answer

Use Amazon S3 Batch Operations to copy the objects into a single large object, then have the training job read that object.

Answer

Enable S3 Transfer Acceleration on the bucket and have the training job continue reading the individual objects.

Question 8

A data engineer is using AWS Glue to process a large dataset stored in Amazon S3. The dataset is partitioned by date in the format s3://bucket/data/year=YYYY/month=MM/day=DD/. The engineer needs to run a nightly ETL job that processes only the previous day's data. The job must be efficient and minimize data scanned. Which approach should the engineer take?

Accepted Answer

Create an AWS Glue crawler to catalog the entire S3 bucket, then use AWS Glue ETL with a filter on the partition columns to process only the previous day's data.. To efficiently process only the previous day's data, the engineer should first catalog the partitioned data using a Glue crawler, which creates a table with partition keys. Then, in the ETL job, filtering on those partition columns enables partition pruning, so only the relevant S3 prefix is read. This minimizes data scanned and improves performance. Other options either scan unnecessary data or introduce additional services and complexity.

Answer

Configure the AWS Glue ETL job to use a pushdown predicate in the DynamicFrame reader to filter on the partition columns.

Answer

Use AWS Glue ETL to read the entire dataset and then apply a filter transform to select only the previous day's records.

Answer

Use Amazon Athena to query the previous day's data and then use AWS Glue to transform the results.

Question 9

A machine learning team stores raw training data in an Amazon S3 bucket with versioning enabled. A data engineer must create a reproducible, point-in-time snapshot of the entire bucket for a model audit that will occur six months from now. The snapshot must be immutable, retained for seven years, and cost-optimized for data that is rarely accessed after the initial audit. Which combination of actions should the engineer take?

Accepted Answer

Create an S3 Object Lock compliance-mode retention rule on the bucket, then use S3 Batch Operations with the S3 Glacier Deep Archive storage class for long-term retention.. The audit requires an immutable point-in-time copy retained for seven years at low cost. S3 Object Lock in compliance mode prevents deletion or overwrite of protected versions for the retention period, which no other S3 feature guarantees. Combining Object Lock with S3 Batch Operations to set the Glacier Deep Archive storage class keeps the data durable and cheap to retain for years while still allowing a restore when the audit occurs.

Answer

Enable S3 Cross-Region Replication to a second bucket and apply a lifecycle rule to transition objects to S3 Glacier Deep Archive after 30 days.

Answer

Use S3 Storage Lens to record a metrics snapshot of the bucket, then configure an S3 Lifecycle rule to expire noncurrent versions after seven years.

Answer

Configure an S3 Lifecycle rule to transition objects to S3 Standard-Infrequent Access and enable MFA delete on the bucket.

Question 10

A company is using AWS Glue to perform ETL on data stored in Amazon S3. The data is in CSV format and needs to be transformed into Parquet. The Glue job must handle schema evolution and ensure that new columns added to the source data are automatically included in the output. The company also wants to minimize the number of Glue jobs and simplify maintenance. Which two actions should the data engineer take to meet these requirements? (Choose two.)

Accepted Answer

Use the AWS Glue ETL job to read from the Data Catalog table and write to Parquet with the updateBehavior parameter set to UPDATE_IN_DATABASE.. To handle schema evolution in AWS Glue, the crawler should be configured to update the Data Catalog with new columns as they appear. Then, the Glue job can read the latest schema from the catalog. Additionally, when writing to the Data Catalog, using updateBehavior=UPDATE_IN_DATABASE ensures the catalog is updated with any schema changes from the job. Together, these actions automate schema evolution and reduce maintenance overhead.

Answer

Use the AWS Glue DynamicFrame with the resolveChoice transform to handle schema changes.

Answer

Enable job bookmarks to track processed data and automatically update the schema.

Answer

Manually update the Glue job script to add new columns whenever the source schema changes.

Question 11

A data engineering team is using AWS Glue to transform data stored in Amazon S3. The team needs to ensure that the Glue ETL job can handle schema changes in the source data, such as new columns being added or data types changing, without failing. The team also wants to minimize the need to manually update the job script when schemas evolve. Which two actions should the team take to meet these requirements? (Choose two.)

Accepted Answer

Use the AWS Glue DynamicFrame API instead of DataFrame API, and leverage the 'ResolveChoice' and 'ApplyMapping' transforms to handle schema inconsistencies.. To handle schema evolution in AWS Glue without manual script updates, the team should use DynamicFrames with ResolveChoice to manage type inconsistencies, and configure the job to update the Data Catalog schema automatically. DynamicFrames are inherently flexible and can accommodate new columns and type changes. ResolveChoice resolves conflicts, while updating the Data Catalog keeps metadata current. Together, these actions allow the job to process evolving data robustly.

Answer

Enable the 'Group small files' option in the Glue job to combine multiple small files into larger ones, reducing the number of partitions and improving schema inference.

Answer

Configure the Glue job to use the 'Detect schema' option and set the job to update the Data Catalog table with the new schema automatically.

Answer

Enable the 'Job bookmark' feature to track processed data and avoid reprocessing, which also helps in handling schema changes by skipping incompatible records.

Question 12

A data engineering team runs an AWS Glue extract, transform, and load job that reads semi-structured JSON from Amazon S3, flattens nested arrays, and writes partitioned Parquet to another S3 prefix. The job previously completed in 20 minutes but now takes over three hours and occasionally fails with executor out-of-memory errors after the source data grew from 50 GB to 2 TB. The team wants to reduce runtime and eliminate the memory failures with minimal code changes. Which action should the team take?

Accepted Answer

Repartition the DynamicFrame on a high-cardinality key and set the spark.sql.shuffle.partitions property to a larger value.. The job slowdown and executor memory errors point to data skew and insufficient shuffle parallelism after the dataset grew fortyfold. Repartitioning the DynamicFrame on a high-cardinality key redistributes records evenly, and increasing spark.sql.shuffle.partitions creates more, smaller shuffle tasks. Together these changes let the existing transform logic run in parallel without rewriting the job, meeting the minimal-code-change constraint.

Answer

Enable AWS Glue job metrics and switch the worker type from Standard to G.1X.

Answer

Convert the source JSON to Parquet with a separate job, then apply a partition pushdown predicate in the main job.

Answer

Increase the number of Glue Data Processing Units and enable the job bookmark.

Question 13

A machine learning engineer is preparing a dataset for training a model using Amazon SageMaker. The dataset is stored in Amazon S3 in CSV format, and it is 500 GB in size. The engineer needs to preprocess the data by removing duplicate rows and normalizing numerical features. The preprocessing must be performed efficiently and cost-effectively, and the resulting dataset should be stored back in S3 for training. Which approach should the engineer take?

Accepted Answer

Use an Amazon SageMaker Processing job with a custom Python script that uses Apache Spark to read the CSV data, perform deduplication and normalization using Spark MLlib, and write the output to S3 in Parquet format.. For large-scale preprocessing in SageMaker, a Processing job with Apache Spark is ideal. Spark can handle 500 GB datasets by distributing the workload across multiple nodes, avoiding memory issues. Spark MLlib offers StandardScaler for normalization and DataFrame operations for deduplication. The job writes output to S3 in Parquet, which is efficient for training. This approach is cost-effective and integrated with SageMaker, making it the best choice.

Answer

Use an Amazon SageMaker Processing job with a custom Python script that uses pandas to load the entire dataset into memory, remove duplicates, normalize features, and write the output to S3.

Answer

Use AWS Glue to create an ETL job that reads the CSV data, uses the DropDuplicates and StandardScaler transforms, and writes the output in Parquet format to S3.

Answer

Use an Amazon EMR cluster with Apache Spark to process the data, then use AWS Data Pipeline to copy the output to S3.

Question 14

A financial services company must build a feature store that serves both batch training jobs and low-latency online inference for a fraud detection model. The online store must return features within single-digit milliseconds, and the batch store must support petabyte-scale historical point-in-time queries for model retraining. The team wants to use managed AWS services and avoid operating its own database clusters. Which two actions should the team take? (Choose two.)

Accepted Answer

Use Amazon SageMaker Feature Store with an online store backed by Amazon DynamoDB and an offline store backed by Amazon S3.. The requirements are low-latency online reads, petabyte-scale offline history, managed services, and no self-operated clusters. SageMaker Feature Store natively combines a DynamoDB-backed online store with an S3-backed offline store and adds point-in-time correctness. The DynamoDB plus DynamoDB Streams to S3 pattern also satisfies latency, scale, and managed-service constraints, though it requires the team to implement point-in-time logic. Both keep the online and offline paths in AWS-managed infrastructure.

Answer

Use Amazon SageMaker Feature Store with the online store disabled and query the offline store directly for inference.

Answer

Deploy an Amazon ElastiCache for Redis cluster as the online store and an Amazon Redshift cluster as the offline store.

Answer

Store features in an Amazon Aurora PostgreSQL cluster with a read replica and export snapshots to Amazon S3 nightly.

Question 15

A data engineer is configuring an AWS Glue crawler to catalog a large dataset stored in Amazon S3. The data is partitioned by year, month, and day in a Hive-style layout, with Parquet files in each leaf prefix. The crawler currently takes several hours because it scans every partition to infer the schema. The engineer wants the crawler to finish quickly while still producing a correct table definition that includes the partition columns. What should the engineer do?

Accepted Answer

Set the crawler to crawl only the root prefix of the dataset instead of all partitions, and rely on the Hive-style prefix structure for partition discovery.. Crawler runtime on a deeply partitioned dataset is dominated by enumerating and sampling every leaf prefix. Crawling only the root prefix lets the crawler infer the data schema from a sample of files and derive the partition columns from the Hive-style year, month, and day path segments. The table still exposes those partition columns, and the crawler completes far faster because it does not walk the entire partition tree.

Answer

Set the crawler to use a classifier that recognizes Parquet and disable the 'Update all new and existing partitions with metadata from the table' option.

Answer

Increase the crawler's data sampling rate to 100 percent so it reads fewer files per partition.

Answer

Configure the crawler to use the 'Create a single schema for each S3 path' option and set a schedule to run daily.