MLS-C01 Data Engineering Practice Test 31 — 15 Questions

Question 1

A company runs a nightly Apache Spark job on Amazon EMR that reads 5 TB of JSON from Amazon S3, joins it with a 2 GB reference table, and writes Parquet. The job has run reliably for months, but after the reference table grew, executors now fail with out-of-memory errors during the join stage. The engineer must reduce memory pressure without changing the output. Which approach is MOST appropriate?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Broadcast the reference table to all executors so the join avoids a shuffle.. When one join side is small enough to fit in executor memory, broadcasting it converts a shuffle join into a map-side join, removing the need to materialize the large side per partition. This directly relieves the out-of-memory condition caused by the grown reference table, as long as the broadcast fits within Spark's configured threshold.

Answer

Convert the reference table from Parquet to CSV to reduce its in-memory footprint.

Answer

Repartition the large JSON dataset by the join key before the join to increase parallelism.

Answer

Increase the number of executors and reduce each executor's memory allocation proportionally.

Question 2

A machine learning team stores 40 TB of historical training features in an Amazon S3 bucket as uncompressed CSV. A SageMaker training job reads the full dataset at the start of every run using File mode, and each run now spends over two hours just downloading data before training begins. The team wants to reduce the startup time of each training job with minimal changes to the training script. What should a data engineer do?

Accepted Answer

Convert the CSV objects to Apache Parquet and use Pipe mode with the SageMaker training job.. Pipe mode streams records from S3 on demand rather than copying the whole dataset to local storage before training, eliminating the large upfront download characteristic of File mode. Pairing it with Parquet reduces the bytes transferred because only needed columns are read. Together these changes cut startup time substantially while requiring at most a small change to the input configuration, not a rewrite of the training logic.

Answer

Move the dataset into an Amazon EFS file system mounted on the training instances and keep File mode.

Answer

Increase the SageMaker training instance's attached Amazon EBS volume size and keep File mode.

Answer

Enable S3 Transfer Acceleration on the bucket and keep File mode with CSV input.

Question 3

A data scientist has a 900 GB Parquet dataset on Amazon S3 partitioned by date. A nightly SageMaker training job in File mode currently takes 6 hours, and CloudWatch shows GPU utilization near 20 percent while the data channel is saturated. The team cannot change the algorithm but can change how data is fed to the job. Which change most improves training throughput?

Accepted Answer

Increase the number of training instances and enable Pipe mode so the algorithm streams shards directly from S3.. Low GPU utilization with a saturated data channel is the classic File mode symptom: the container waits for the dataset to download before and during training. Pipe mode streams shards so compute starts immediately and never waits on a full local copy, and distributing across instances multiplies available ingestion bandwidth, so both changes together raise throughput.

Answer

Switch the training job to use a larger instance type with more vCPUs but the same single instance count.

Answer

Enable SageMaker Debugger and add a rule that monitors GPU utilization during training.

Answer

Attach an Amazon EFS file system to the training job and copy the dataset there before training.

Question 4

A machine learning team stores training data in Amazon S3 and runs nightly SageMaker training jobs that read the same 2 TB dataset from many instances across multiple epochs. Training is I/O bound and throughput to S3 varies, extending job runtimes. The team wants to reduce data transfer time and provide consistent read performance. Which approach is MOST appropriate?

Accepted Answer

Attach an Amazon FSx for Lustre file system linked to the S3 bucket and use it as the training input channel.. FSx for Lustre is purpose-built for high-performance parallel access and integrates directly with S3, so a linked file system presents the dataset as a fast, shared POSIX namespace. SageMaker training jobs can mount it as an input channel, giving many instances consistent throughput across repeated epochs. EFS, EBS copies, and Transfer Acceleration do not deliver the same combination of shared access and sustained throughput.

Answer

Create an Amazon EFS file system and mount it to the training instances as the input data source.

Answer

Enable S3 Transfer Acceleration on the bucket and continue reading directly from S3.

Answer

Copy the dataset to the instance's Amazon EBS volume before each epoch using a lifecycle script.

Question 5

A data engineer needs to ingest a continuously growing stream of application logs into Amazon S3 for later machine learning training. The logs arrive at variable rates, sometimes spiking to several megabytes per second, and the team wants minimal operational overhead with automatic scaling. The data should be delivered in near real time and compressed. Which service should the engineer use?

Accepted Answer

Amazon Kinesis Data Firehose with S3 as the destination and GZIP compression enabled.. Amazon Kinesis Data Firehose is the managed service for delivering streaming data to destinations such as S3. It scales automatically with throughput, buffers and compresses records, and requires no server management, which fits the variable-rate log ingestion and near-real-time compressed delivery requirements.

Answer

AWS Database Migration Service with S3 as the target.

Answer

AWS Glue crawler running on a schedule to pull logs from the application servers.

Answer

Amazon S3 Transfer Acceleration on a bucket that receives log uploads from the application.

Question 6

A company runs an AWS Glue extract, transform, and load job that reads semi-structured JSON from Amazon S3, flattens nested arrays, and writes Parquet to a curated bucket. The job has grown to 200 DPU-hours and fails intermittently with out-of-memory errors on executors. A data engineer must reduce runtime and memory pressure without changing the output schema. (Choose two.)

Accepted Answer

Enable the Glue job bookmark and process only new S3 objects on each run.. Job bookmarks restrict each run to newly arrived objects, shrinking the data volume that flows through the pipeline, while additional workers with auto scaling spread partitions across more executors to lower per-executor memory use and shorten runtime. Both changes affect throughput and resource distribution only; they leave the transformation logic and the curated Parquet schema exactly as before.

Answer

Convert the job to use the AWS Glue Python shell job type instead of Spark.

Answer

Apply a ResolveChoice transform to cast all nested fields to string before the flatten step.

Answer

Set the job's maximum concurrency to a high value and schedule overlapping runs.

Question 7

A machine learning engineer is using Amazon SageMaker to train a model on a large dataset stored in Amazon S3. The dataset is in CSV format and is 1 TB in size. The training job is taking a long time to start because the data is being downloaded to the training instance's Amazon EBS volume. The engineer wants to reduce the startup time and improve training efficiency. Which solution should the engineer implement?

Accepted Answer

Convert the dataset to the RecordIO format and use Pipe mode in the SageMaker training job.. Pipe mode in SageMaker streams training data directly from Amazon S3 to the training algorithm, bypassing the need to download the entire dataset to the EBS volume. This drastically reduces startup time, especially for large datasets like 1 TB. Converting to RecordIO format is recommended because it supports efficient streaming and is compatible with Pipe mode. Other options either do not address the download bottleneck or are unrelated to performance.

Answer

Use SageMaker's File mode with a larger instance type that has higher network bandwidth.

Answer

Enable SageMaker's network isolation and use a VPC endpoint to accelerate the S3 download.

Answer

Increase the size of the Amazon EBS volume attached to the training instance to accommodate the dataset faster.

Question 8

A data engineer is building a feature pipeline that must replay the last 7 days of events from an Amazon Kinesis Data Stream into a new consumer application. The stream currently retains records for 24 hours, and the engineer must not change the retention period or affect existing consumers. Which approach allows the new consumer to read records from the past 7 days?

Accepted Answer

Point the new consumer at an Amazon S3 destination where the events were archived, such as a Firehose delivery stream or Kinesis Data Analytics output, and replay from there.. Kinesis Data Streams only permits reads within the configured retention window, so a 24-hour stream cannot serve seven-day-old records no matter which starting position or consumer type is used. Because retention must remain unchanged, the historical events must come from a durable store. Replaying from an S3 archive created by a delivery or analytics consumer provides the missing six days without affecting current readers.

Answer

Increase the stream's retention period to 168 hours, then read from TRIM_HORIZON, and leave it at 168 hours permanently.

Answer

Enable enhanced fan-out on the stream and register the new consumer with a starting timestamp 7 days in the past.

Answer

Use the Kinesis Client Library with a starting position of TRIM_HORIZON on the existing stream.

Question 9

A company stores raw sensor readings in Amazon S3 and must produce a cleaned, feature-engineered dataset for model training. The transformations are written in PySpark, the job must run on a schedule without managing servers, and the output must be immediately queryable by Amazon Athena. Which service should the data engineer use?

Accepted Answer

AWS Glue with a Spark ETL job that writes the transformed output to S3 in Parquet format.. AWS Glue provides managed, serverless Spark execution with job scheduling and a native path to the Data Catalog, so PySpark transformations can run on a schedule and land Parquet in S3 that Athena queries immediately. Lambda, EMR on EC2, and streaming Flink each violate at least one requirement around scale, server management, or batch semantics.

Answer

Amazon Kinesis Data Analytics with a Flink application writing results to S3.

Answer

Amazon EMR on Amazon EC2 with a long-running cluster and a nightly cron job.

Answer

AWS Lambda with a Python script that processes each S3 object on upload.

Question 10

A company runs an AWS Glue ETL job that reads JSON from Amazon S3, transforms records, and writes Parquet to another S3 prefix. The job must run every hour, and the team wants to minimize cost and manual scheduling effort. The transformation logic is already defined. What should the engineer configure?

Accepted Answer

Create a Glue trigger of type SCHEDULED with a cron expression that fires hourly and starts the existing job.. AWS Glue scheduled triggers are the native, serverless way to run a job on a recurring cron schedule. They require no external orchestration, keep cost low, and start the existing job definition automatically. Step Functions, EventBridge with Lambda, and bookmarks each address different concerns and either add overhead or fail to schedule the job at all.

Answer

Use AWS Step Functions with a Wait state to invoke the Glue job every hour.

Answer

Configure an Amazon EventBridge rule that runs on a schedule and starts an AWS Lambda function to submit the job.

Answer

Enable job bookmarks and set the job to run continuously.

Question 11

A machine learning platform team stores training data in Amazon S3 and uses AWS Lake Formation to manage access. Data scientists in one business unit must read only the columns relevant to their models from a shared table, while other columns containing personally identifiable information must remain hidden. The team also wants to enforce these restrictions consistently across Athena and Amazon Redshift Spectrum. What should the team do?

Accepted Answer

Configure Lake Formation column-level permissions and grant SELECT on specific columns to the data scientist role.. Lake Formation column-level permissions are the native mechanism to grant SELECT on a subset of columns while hiding PII, and they are enforced across integrated engines including Athena and Redshift Spectrum. IAM, presigned URLs, and table splitting cannot express column-level access within a shared table.

Answer

Enable S3 Block Public Access and require data scientists to use presigned URLs.

Answer

Create an IAM policy that denies access to the S3 prefixes containing the PII columns.

Answer

Use AWS Glue crawlers to create separate tables for each column and grant access per table.

Question 12

A data engineer must load a 5 GB uncompressed CSV file from Amazon S3 into Amazon Redshift using the COPY command. The cluster has four nodes and the file resides in a bucket in the same AWS Region. The load is taking far longer than expected and the engineer notices the data is being distributed unevenly across slices. What is the most likely cause?

Accepted Answer

The COPY command is reading a single file, so only one slice is used to parse and load the data.. COPY achieves parallelism by distributing input files across the cluster's slices. A single 5 GB file can be read by only one slice, leaving the rest of the cluster idle and making the load effectively serial. Splitting the source into many files, ideally a multiple of the total slice count, allows all slices to parse and insert concurrently and is the standard remedy for slow Redshift loads.

Answer

The CSV file uses a delimiter that does not match the table's column definitions.

Answer

The target table's distribution key does not match the sort key, preventing parallel writes.

Answer

The COPY command is missing the PARALLEL ON option, which defaults to OFF.

Question 13

A company runs an AWS Glue crawler over a data lake in Amazon S3. New partitions are added daily in Hive-style paths such as year=2024/month=05/day=12/, but the crawler is not detecting the newest partitions and queries return no data for the latest day. The data engineer wants the crawler to discover new partitions automatically going forward. Which action should the engineer take?

Accepted Answer

Edit the crawler's include path so it covers the parent prefix that contains all partitions, and verify the partition keys are encoded in the object key path.. Crawlers derive partitions from the S3 key structure under the configured include path. If the include path does not cover the prefix where daily folders are written, or if partition keys are not encoded in the path, the crawler cannot register those partitions in the Data Catalog, and Athena returns no rows for the newest day. Correcting the include path and confirming the Hive-style key layout restores automatic discovery.

Answer

Configure the crawler to run on a schedule and set the schema change policy to update the Data Catalog table.

Answer

Point the crawler at the S3 bucket root and enable the Add new columns only option in the crawler configuration.

Answer

Increase the crawler's maximum DPU capacity so it can scan the S3 prefix faster.

Question 14

A fraud detection team needs features computed over a rolling 10-minute window from a high-volume stream of card transactions. The data currently lands in Amazon S3 via Amazon Kinesis Data Firehose, and a nightly AWS Glue job computes aggregates, so features are up to 24 hours stale and model accuracy suffers. The team wants features available to the online model within seconds of each event while reusing the same aggregation logic for both training and inference. Which solution meets these requirements?

Accepted Answer

Use Amazon Kinesis Data Analytics for Apache Flink to compute the windowed aggregates and write them to an Amazon SageMaker Feature Store online store.. A managed Apache Flink application can maintain event-time state and emit ten-minute rolling aggregates continuously from the stream, giving seconds-level freshness. Routing those features into the SageMaker Feature Store online store makes them retrievable with low latency at inference, while the feature group's offline store supplies point-in-time correct training data from the same definition. That single aggregation definition satisfies both freshness and reuse.

Answer

Increase the Firehose buffer interval to 60 seconds and have the model query Amazon Athena against the raw S3 objects.

Answer

Enable Kinesis Data Streams enhanced fan-out and have the model consume the raw events directly.

Answer

Run the existing AWS Glue job every 10 minutes with a smaller window and have the model read the output Parquet from Amazon S3.

Question 15

An ML engineer needs to give a SageMaker training job access to training data in an Amazon S3 bucket in the same account. Security policy requires that the job assume a dedicated IAM role and that the role be restricted to only the specific S3 prefix used for training. Which configuration should the engineer use?

Accepted Answer

Create an IAM role with an S3 read policy scoped to the training prefix and a trust policy allowing sagemaker.amazonaws.com to assume it, then pass the role ARN as the execution role of the training job.. Training jobs run under an execution role, and that role must both trust sagemaker.amazonaws.com and carry an identity-based policy granting read access to the training prefix. Scoping the policy resource to the specific prefix satisfies least privilege, and supplying the role ARN when configuring the job connects the container's temporary credentials to those permissions. Broad managed policies or user-level permissions do not meet the stated restrictions.

Answer

Attach an S3 read policy to the user account running the training job and rely on the user's credentials being passed to the training container.

Answer

Create an S3 bucket policy that allows the SageMaker service principal to read the prefix, and use the default SageMaker execution role for the job.

Answer

Create an IAM role with the AmazonS3FullAccess managed policy and a trust policy for sagemaker.amazonaws.com, then use it as the execution role.