MLS-C01 Data Engineering Practice Test 29 — 15 Questions

Question 1

A machine learning team stores training data as approximately 4 million small JSON objects in an Amazon S3 bucket. A SageMaker training job that reads this prefix with File mode takes many hours just to start because the training instances must download every object before training begins. The team wants to reduce startup time without changing the training algorithm or the object contents. Which solution should a machine learning engineer recommend?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Convert the dataset to fewer, larger files such as sharded Parquet or RecordIO and configure the training job to use Pipe mode or Fast File mode.. Training performance on Amazon S3 data is limited by per-object overhead, not just raw bandwidth. Millions of tiny JSON files force the training container to issue millions of GET requests before training can start. Aggregating the records into a few large shards and streaming them with Pipe mode or Fast File mode removes both the request overhead and the full local download step, so the job starts far sooner.

Answer

Add an S3 Lifecycle rule that transitions the JSON objects to S3 Glacier Instant Retrieval, then rerun the training job.

Answer

Enable S3 Transfer Acceleration on the bucket and rerun the training job from the same Region.

Answer

Increase the number of training instances and enable network isolation on the training job.

Question 2

A company stores raw training data in an Amazon S3 bucket that uses SSE-KMS with a customer managed key. An ML engineer's SageMaker training job fails immediately with an access denied error when it tries to read the input channel, even though the engineer's IAM user has s3:GetObject permission on the bucket. The engineer confirms the bucket policy allows the role. What is the most likely cause?

Accepted Answer

The SageMaker execution role lacks kms:Decrypt permission on the customer managed key used to encrypt the objects.. Reading SSE-KMS encrypted objects requires two layers of authorization: S3 object access and permission to use the KMS key. A SageMaker training job operates under its execution role, so the engineer's own s3:GetObject permission does not apply. Granting kms:Decrypt on the customer managed key to the execution role, and ensuring the key policy allows it, resolves the immediate access denied error.

Answer

The training job must be configured with an S3 VPC endpoint because the bucket is in a different Availability Zone.

Answer

SSE-KMS encrypted objects cannot be read by SageMaker training jobs and must be decrypted to SSE-S3 before use.

Answer

The S3 bucket policy must explicitly list the SageMaker service principal as a principal and allow s3:GetObject.

Question 3

A data engineering team is building a pipeline that uses AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The team wants to ensure that the Glue job can read from S3 and write to Redshift securely. They have created an IAM role for the Glue job with permissions to access S3 and Redshift. However, when they run the job, it fails with an authorization error when attempting to write to Redshift. What is the most likely cause of this failure?

Accepted Answer

The IAM role lacks the redshift:GetClusterCredentials permission, and the Redshift database user does not have the necessary privileges to write to the target table.. When AWS Glue writes to Amazon Redshift, it uses the IAM role attached to the job to call redshift:GetClusterCredentials and obtain temporary database credentials. If that permission is missing, the job cannot authenticate to Redshift. Furthermore, the Redshift database user must have appropriate privileges (e.g., INSERT) on the target table. Network or S3 staging issues would produce different errors. Therefore, the combination of missing IAM permission and insufficient database privileges is the most likely cause.

Answer

The IAM role attached to the Glue job does not have permissions to assume the Redshift service role.

Answer

The Glue job's IAM role does not have s3:PutObject permission on the Redshift cluster's staging directory.

Answer

The Redshift cluster is in a different VPC than the Glue job, and the security groups do not allow traffic.

Question 4

A retail analytics team wants a fully managed, serverless way to run Spark ETL that reads from the AWS Glue Data Catalog and writes transformed Parquet back to Amazon S3 on a nightly schedule. They want to minimize operational overhead and pay only for the time the job runs. Which AWS service should they use?

Accepted Answer

AWS Glue jobs with a Spark ETL script, scheduled with a Glue trigger and using the Data Catalog for source and target tables.. AWS Glue provides a serverless Apache Spark environment that integrates directly with the AWS Glue Data Catalog for source and target schemas, and Glue triggers can run jobs on a nightly schedule. Because billing is based on DPU-hours consumed during job execution, the team avoids paying for idle capacity while keeping operational overhead low.

Answer

Amazon Athena with CREATE TABLE AS SELECT statements scheduled by Amazon EventBridge.

Answer

AWS Step Functions calling an AWS Lambda function that uses pandas for the transformation.

Answer

Amazon EMR running on Amazon EC2 with a persistent cluster and a nightly cron job.

Question 5

A machine learning engineer is using Amazon SageMaker to train a model on a large dataset stored in Amazon S3. The dataset is in CSV format and is 500 GB. The engineer wants to reduce training time by using SageMaker Pipe mode instead of File mode. However, the training script currently reads the entire CSV file into memory and performs preprocessing. What should the engineer do to adapt the training script for Pipe mode?

Accepted Answer

Modify the script to read the data as a streaming dataset using the SageMaker PipeModeDataset and apply preprocessing on the fly.. Pipe mode streams data from S3 to the training container, eliminating the need to download the entire dataset. The training script must use the PipeModeDataset to read the stream and perform preprocessing per batch. This adaptation allows efficient training on large CSV datasets without loading everything into memory.

Answer

Increase the instance count and use distributed training with File mode to handle the large dataset.

Answer

Convert the CSV files to TFRecord format and use the SageMaker Pipe mode with the TensorFlow dataset API.

Answer

Use SageMaker Augmented Manifest files to stream the CSV data and preprocess using the SageMaker TensorFlow estimator.

Question 6

A data engineer needs to catalog a large collection of JSON files in Amazon S3 so that analysts can query them with Amazon Athena. The files have evolving schemas, with new fields appearing over time. The engineer wants the catalog to automatically detect schema changes on subsequent crawls without manual edits. Which AWS Glue crawler configuration should the engineer choose?

Accepted Answer

Create a crawler with a JSON classifier and set the crawler to update the Data Catalog table's schema on each run.. A JSON classifier lets the crawler infer columns from JSON keys, and configuring the crawler to update the Data Catalog table on each run ensures newly appearing fields are added automatically. This combination supports evolving schemas without manual intervention, allowing Athena analysts to query new fields as soon as a crawl completes.

Answer

Create a crawler that targets the S3 bucket and configure it to use a CSV classifier with a custom delimiter.

Answer

Create a crawler that targets the S3 bucket and set the schema change policy to Log only so changes are recorded but not applied.

Answer

Create a crawler with an Apache Groovy classifier that hard-codes the expected column list and writes it to the Data Catalog.

Question 7

A data engineer needs to move 8 TB of on-premises clickstream logs into Amazon S3 once, and the site has a 100 Mbps internet connection with intermittent outages. The data must arrive intact within two weeks, and the engineer wants a managed service that validates the transfer and can resume after interruptions. Which approach should the engineer use?

Accepted Answer

Order an AWS Snowball Edge Storage Optimized device, copy the logs to it, ship it back, and let AWS import the data into Amazon S3.. For multi-terabyte one-time migrations over a slow or unreliable link, AWS Snowball Edge ships the data physically, avoiding the bandwidth and outage constraints of internet transfer. The service handles encryption, integrity validation, and import into Amazon S3, and shipping plus import comfortably fits within a two-week window.

Answer

Use the AWS CLI `aws s3 cp --recursive` with multipart upload and retry settings to push the files directly to Amazon S3.

Answer

Use AWS DataSync over the internet with a scheduled task and bandwidth throttling.

Answer

Configure an AWS Storage Gateway file gateway and replicate the logs through it to Amazon S3.

Question 8

A financial services company stores sensitive transaction logs in an Amazon S3 bucket. A data engineer must ensure that all objects written to the bucket are encrypted at rest with a customer-managed AWS KMS key. The engineer also needs to enforce that the key is used for every upload and that the encryption is applied automatically without the uploader specifying it. Which solution meets these requirements with the LEAST operational overhead?

Accepted Answer

Enable S3 default encryption with SSE-KMS using the customer-managed key, and modify the bucket policy to deny s3:PutObject requests that specify a different encryption method or no encryption, while allowing the default encryption to apply when no encryption headers are provided.. The scenario requires automatic encryption with a customer-managed KMS key and enforcement that the key is used, with minimal overhead. Enabling S3 default encryption with SSE-KMS applies the key automatically for uploads that do not specify encryption. A bucket policy that denies uploads attempting to use a different encryption method or no encryption ensures the key is used without requiring uploaders to change their requests. This combination meets the requirements with the least operational effort.

Answer

Enable S3 default encryption with SSE-KMS using the customer-managed key, and require that all uploads include the x-amz-server-side-encryption and x-amz-server-side-encryption-aws-kms-key-id headers by adding a bucket policy that denies requests missing these headers.

Answer

Use S3 server-side encryption with Amazon S3 managed keys (SSE-S3) and configure a bucket policy that requires the use of the customer-managed KMS key by checking the aws:RequestTag condition.

Answer

Configure the S3 bucket with default encryption using SSE-KMS with the customer-managed key, and add a bucket policy that denies s3:PutObject requests that do not include the x-amz-server-side-encryption header with value aws:kms.

Question 9

A retail company streams point-of-sale transactions into an Amazon Kinesis Data Stream with four shards. A consumer application using the Kinesis Client Library processes records and writes them to Amazon S3. During holiday peaks the stream's WriteProvisionedThroughputExceeded metric rises and consumers fall behind, yet the PutRecord API is called with the same partition key for every transaction. The team wants to reduce throttling and improve consumer parallelism with minimal application change. Which action should a data engineer take?

Accepted Answer

Increase the number of shards in the stream and modify the producer to use a high-cardinality partition key such as the transaction ID.. The stream is throttling because every record uses the same partition key, concentrating writes on one shard. Adding shards raises aggregate write capacity, and switching to a high-cardinality key distributes records so no single shard saturates. This also allows more KCL workers to lease shards in parallel, improving consumer throughput with limited code change.

Answer

Enable enhanced fan-out on the stream and keep the single partition key so each consumer gets dedicated throughput.

Answer

Raise the stream's retention period to 168 hours and increase the consumer's MaxRecords parameter per GetRecords call.

Answer

Switch the producer to the Kinesis Producer Library and rely on its aggregation to raise throughput per shard.

Question 10

A machine learning team stores training data as approximately 4 million small JSON objects in an Amazon S3 bucket. A SageMaker training job that reads this prefix with File mode takes many hours just to start because the training instances must download every object before training begins. The team wants to reduce startup time without changing the training algorithm or the object contents. Which solution should a machine learning engineer recommend?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Grant the SageMaker execution role only s3:GetObject and s3:ListBucket on the bucket, and enable S3 Versioning plus a bucket policy statement denying s3:DeleteObject for that role.. Limiting the SageMaker execution role to read and list actions enforces least privilege so training jobs can access data but cannot modify it. Adding an explicit deny for delete actions removes any accidental permission path, and enabling S3 Versioning preserves prior versions so an overwrite or deletion can be reversed by restoring an earlier version.

Answer

Enable S3 Object Lock in governance mode with a 30-day retention period on all objects and give the role full S3 permissions.

Answer

Attach an IAM policy that denies s3:DeleteObject only on the specific prefix used for training data, and rely on S3 Versioning for overwrite recovery.

Answer

Use an S3 bucket policy that requires MFA delete and give the SageMaker role permissions to use MFA when deleting objects.

Question 11

An ML engineer is preparing a SageMaker training job that reads a large dataset from Amazon S3. The dataset consists of many Parquet files under a single S3 prefix, and the job will run on a multi-instance cluster. The engineer wants to reduce training startup time and maximize read throughput across instances. (Choose two.)

Accepted Answer

Set the input channel's S3DataDistributionType to ShardedByS3Key so each instance reads a distinct subset of the objects.. ShardedByS3Key spreads the Parquet objects across the instances so each reads a distinct subset, and Pipe mode streams data directly from S3 instead of staging everything locally first. Together they reduce redundant transfer and startup delay while raising aggregate read throughput on a multi-instance training cluster.

Answer

Set the input channel's S3DataDistributionType to FullyReplicated so every instance has a complete copy of the dataset.

Answer

Increase the training job's MaxRuntimeInSeconds and MaxWaitTimeInSeconds parameters to allow more time for data download.

Answer

Enable Amazon S3 Transfer Acceleration on the bucket to speed up downloads to the training instances.

Question 12

A machine learning engineer needs to train a model on a 2 TB dataset stored in Amazon S3 as CSV files. The training job will run on an Amazon SageMaker training instance. The engineer wants to minimize training time and avoid downloading the entire dataset to the instance's Amazon EBS volume. Which data input mode should be used?

Accepted Answer

FastFile mode. FastFile mode allows SageMaker training jobs to stream data from S3 as if it were a local file system, avoiding the need to download the full dataset to EBS. This reduces startup time and disk requirements, which is ideal for large datasets. Pipe mode also streams but has stricter algorithm compatibility requirements. FastFile offers a balance of performance and compatibility.

Answer

Augmented manifest file mode

Answer

Pipe mode

Answer

File mode

Question 13

A data engineering team stores daily CSV extracts in an Amazon S3 bucket under prefixes such as s3://sales-data/2024/01/15/. A nightly AWS Glue job must process only the previous day's files from the data lake and write curated output to another bucket. The team wants the job to avoid scanning the entire bucket and to keep the catalog current with minimal manual effort. Which combination of steps should the engineer implement?

Accepted Answer

Run an AWS Glue crawler to register the bucket, create the job with a from_catalog or from_options source restricted to the previous day's partition, and enable job bookmarks.. Cataloging the bucket exposes partition metadata that Glue and Athena can prune on, so a job reading the previous day's partition touches only those objects. Job bookmarks then make each nightly run incremental by tracking processed files. Together these deliver minimal scanned data and low-maintenance catalog updates.

Answer

Create the job with a from_options source pointing at s3://sales-data/ and add a filter transform that drops rows whose date column is not yesterday.

Answer

Enable S3 Transfer Acceleration on the bucket and schedule the Glue job with a smaller number of DPUs to reduce cost.

Answer

Use Amazon S3 Select from the Glue job to run a SQL expression against each object and return only matching rows.

Question 14

A company runs an AWS Glue ETL job that processes data incrementally. The job uses job bookmarks to track previously processed S3 objects. After a schema change in the source data, the team notices that newly arriving objects are not being processed. They confirm the job bookmark is enabled and that the new objects have different ETags and modification times. Which action should the data engineer take?

Accepted Answer

Reset the job bookmark and rerun the job so it reprocesses the source from the beginning with the updated schema.. Glue job bookmarks store state about which objects and partitions were previously processed. A source schema change can leave the bookmark unable to recognize new objects as unprocessed. Resetting the bookmark clears that state so the next run reprocesses the source with the updated schema, restoring processing of the newly arriving objects.

Answer

Convert the job to use AWS Glue DataBrew instead, because DataBrew automatically handles schema changes with bookmarks.

Answer

Disable job bookmarks temporarily and enable them again after the next successful run to refresh the bookmark state.

Answer

Increase the Glue job's number of workers and enable auto scaling to give the bookmark logic more resources to detect new objects.

Question 15

A data engineering team is preparing a training dataset stored as 12,000 small gzip-compressed CSV objects (about 40 KB each) in an Amazon S3 prefix. A SageMaker training job reading these objects directly spends most of its time on per-object GET requests rather than on model computation. The team must reduce I/O overhead without changing the data content. Which approach should the team take?

Accepted Answer

Convert the objects into a single RecordIO-Web (application/x-recordio-protobuf) file and point the training job's channel at that file.. Packing thousands of tiny objects into a sharded container format such as RecordIO-Web lets SageMaker stream large sequential blocks, eliminating the per-object request overhead that dominates runtime. Storage-class changes, transfer acceleration, and more workers all leave the fundamental small-file access pattern intact, so they do not address the measured bottleneck.

Answer

Enable S3 Transfer Acceleration on the bucket and keep the 12,000 objects as they are.

Answer

Increase the training instance count so that more workers read the small objects in parallel.

Answer

Change the objects' storage class from S3 Standard to S3 Intelligent-Tiering before training.