MLS-C01 Machine Learning Implementation and Operations Practice Test 19 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker to train a model on a large dataset (10 TB) stored in S3 in Parquet format. The training job uses an ml.p3.16xlarge instance with multiple GPUs. The data scientist notices that the GPU utilization is low (around 30%) and the training is slow. The dataset consists of hundreds of thousands of small Parquet files. The data scientist suspects that the I/O is bottlenecked. What should the data scientist do to improve GPU utilization and training speed?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Consolidate the small Parquet files into larger files (e.g., 1 GB each). Consolidating small Parquet files into larger files (e.g., 1 GB each) reduces the overhead of reading many small files from S3, improving I/O throughput and keeping GPUs busy. Option A (increase batch size) may help GPU utilization but does not address the I/O bottleneck. Option C (use a smaller instance) would not improve speed and may worsen the situation. Option D (Pipe input mode) can help with streaming but does not solve the small file issue; the data still comes from many small files.

Answer

Increase the batch size

Answer

Use a smaller instance type to reduce cost

Answer

Use Pipe input mode to stream data directly

Question 2

A data scientist is using SageMaker to train a model. The training job is failing with a 'ResourceLimitExceeded' error. Which action should be taken to resolve this issue?

Accepted Answer

Request a service limit increase for the instance type.. The 'ResourceLimitExceeded' error in SageMaker indicates that the AWS account has reached the maximum number of allowed resources (e.g., instances, vCPUs, or storage) for a given instance type in the current region. Requesting a service limit increase via the AWS Service Quotas console or API directly resolves this by raising the cap for that specific instance type, allowing the training job to proceed.

Answer

Use a different AWS Region.

Answer

Reduce the training dataset size.

Answer

Switch to a different instance type with lower resource requirements.

Question 3

A machine learning team is using AWS Glue to prepare data for training. They notice that the ETL job takes a long time to process large datasets. Which change is most likely to improve performance?

Accepted Answer

Increase the number of DPUs for the Glue job.. Increasing the number of DPUs (Data Processing Units) for the AWS Glue job allocates more distributed computing resources, which allows the job to process data in parallel across more executors. This directly reduces the runtime for large datasets by improving the parallelism of Spark transformations and actions.

Answer

Decrease the number of workers in the Glue job.

Answer

Disable Spark shuffle operations.

Answer

Reduce the dataset size by sampling.

Question 4

A data scientist is using SageMaker Debugger to monitor a training job. The training loss is not decreasing as expected. Which Debugger feature can help identify the issue?

Accepted Answer

Built-in rules to detect training anomalies. SageMaker Debugger's built-in rules are designed to automatically monitor training jobs for common issues such as vanishing gradients, overfitting, and loss not decreasing. When the training loss plateaus or fails to decrease, a rule like 'LossNotDecreasing' can trigger a CloudWatch alarm or stop the training job, providing immediate insight into the problem without manual inspection of tensors.

Answer

Automatic hyperparameter tuning

Answer

Saving tensors every step

Answer

Deploying a model endpoint for real-time monitoring

Question 5

A company is using Amazon Rekognition to detect objects in images stored in S3. They want to reduce costs by processing images only when they are uploaded. Which AWS service should be used to trigger Rekognition automatically?

Accepted Answer

AWS Lambda. AWS Lambda is the correct service because it can be triggered directly by S3 events (e.g., s3:ObjectCreated:Put) to invoke Amazon Rekognition's DetectLabels API on the newly uploaded image. This serverless architecture ensures processing occurs only on upload, eliminating idle costs and manual polling.

Answer

Amazon CloudWatch Events

Answer

Amazon Simple Notification Service (SNS)

Answer

AWS Step Functions

Question 6

A machine learning engineer needs to deploy a model that requires custom inference code with dependencies. Which SageMaker deployment option should be used?

Accepted Answer

Create a custom Docker container and deploy to SageMaker endpoint.. When a model requires custom inference code with dependencies, the only way to fully control the runtime environment, libraries, and inference logic is to package everything into a custom Docker container. SageMaker endpoints can then deploy this container, allowing the engineer to specify the exact inference script and dependencies (e.g., via a Dockerfile and a requirements.txt). This approach supports any framework or custom logic that built-in algorithms cannot provide.

Answer

Use a SageMaker notebook instance as an endpoint.

Answer

Use a built-in SageMaker algorithm.

Answer

Use a SageMaker batch transform job.

Question 7

A data scientist is using SageMaker to build a model for fraud detection. The dataset is highly imbalanced. Which THREE techniques should be applied to address class imbalance?

Accepted Answer

Apply SMOTE to generate synthetic samples of the minority class.. Option C is correct because SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic examples of the minority (fraud) class by interpolating between existing minority instances, which increases minority representation and helps the model learn fraud patterns instead of ignoring them. Option D is correct because assigning class weights in the loss function penalizes misclassification of the minority class more heavily, so the model's optimization is not dominated by the majority class. Option E is correct because undersampling the majority class reduces the number of non-fraud examples, balancing the class distribution and preventing the majority class from overwhelming the minority during training. Option A is not appropriate because training only on the majority class discards all fraud examples, so the model cannot learn to detect fraud. Option B is not appropriate because accuracy is misleading on imbalanced data—a trivial majority-class predictor can score high accuracy while failing to detect fraud—so metrics like precision, recall, F1, or AUC-PR should be used instead.

Answer

Train the model only on the majority class.

Answer

Use accuracy as the evaluation metric.

Question 8

A research lab is using SageMaker to train deep learning models on a custom dataset stored in S3. Each training job uses a single ml.p3.2xlarge instance. Recently, training jobs have been failing intermittently with 'NetworkError: Connection reset by peer' during the data download phase. The data scientist notices that the dataset is 50GB and the network throughput is low. The training script uses the default S3 download method (boto3) to copy data from S3 to the local instance storage. Which solution should the data scientist implement to resolve the issue?

Accepted Answer

Use SageMaker Pipe mode to stream data directly from S3.. SageMaker Pipe mode streams training data directly from S3 to the training container over a high-throughput channel without downloading the full dataset to local instance storage first, eliminating the large upfront boto3 download that is failing with connection resets. This is the AWS-recommended approach for large datasets (tens of GB or more) where download time and network instability are bottlenecks. Pipe mode also reduces disk space requirements and starts training faster.

Answer

Mount an EBS volume to the instance and copy data there before training.

Answer

Add retry logic in the training script to handle network errors.

Answer

Use a larger instance type like p3.8xlarge for better network bandwidth.

Question 9

A media company uses SageMaker to train a recommendation model. The training data is stored in an S3 bucket with versioning enabled. The data pipeline updates the training data daily by overwriting objects with new data. Recently, the model's performance degraded, and the team suspects that the training data was corrupted on a specific day. They want to train the model using the data from a previous version. How can the team retrieve the previous version of the training data?

Accepted Answer

Use the S3 GET Object Version API to download the specific version of each object.. The S3 GET Object Version API allows retrieval of a specific version of an object by specifying the version ID. Since the bucket has versioning enabled, each overwrite creates a new version, and the previous version can be downloaded using its version ID. This is the correct method to retrieve the previous version of the training data.

Answer

Restore the bucket from S3 Glacier, which contains the previous version.

Answer

Use S3 Select to query the previous version of the data.

Answer

Enable S3 replication to a different bucket and use the replicated data.

Question 10

A machine learning team is using Amazon SageMaker to train a PyTorch model on a dataset that is 500 GB in size. The training job runs on a single ml.p3.2xlarge instance, but the training takes over 48 hours, which exceeds the maximum allowed time. The team wants to reduce training time to under 24 hours. They are open to using multiple instances and have budget for up to 4 instances. The dataset is stored in Amazon S3 and can be split into shards by a key. The model architecture must remain unchanged. What should the team do?

Accepted Answer

Use SageMaker distributed data parallelism with 4 ml.p3.2xlarge instances.. SageMaker distributed data parallelism shards the dataset across multiple instances and uses an all-reduce communication pattern to synchronize gradients, allowing the same model architecture to train on 4 instances in parallel. With the dataset already shardable by key and budget for 4 instances, this is the correct approach to cut training time while keeping the model unchanged. It scales throughput roughly linearly with the number of instances for data-parallel workloads.

Answer

Use SageMaker Processing to split the data and train separate models.

Answer

Change the instance type to ml.p3.16xlarge.

Answer

Switch to Pipe input mode to stream data faster.

Question 11

A data scientist is using Amazon SageMaker to train a model on a large dataset (10 TB) stored in S3 in Parquet format. The training job uses an ml.p3.16xlarge instance with multiple GPUs. The data scientist notices that the GPU utilization is low (around 30%) and the training is slow. The dataset consists of hundreds of thousands of small Parquet files. The data scientist suspects that the I/O is bottlenecked. What should the data scientist do to improve GPU utilization and training speed?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Add kms:Decrypt permission to the SageMaker execution role.. When S3 objects are encrypted with SSE-KMS, the SageMaker execution role must have the kms:Decrypt permission to decrypt the data during training. Even though the role has s3:GetObject access, the KMS key policy or the role's IAM policy must explicitly allow decryption of the KMS key used for server-side encryption. Without this, SageMaker cannot read the encrypted objects, resulting in an Access Denied error.

Answer

Add kms:Encrypt permission to the SageMaker execution role.

Answer

Add a bucket policy that grants s3:GetObject to the SageMaker role.

Answer

Configure a VPC endpoint for S3 and attach a policy.

Question 12

A data scientist is using Amazon SageMaker to train a model on a large dataset (10 TB) stored in S3 in Parquet format. The training job uses an ml.p3.16xlarge instance with multiple GPUs. The data scientist notices that the GPU utilization is low (around 30%) and the training is slow. The dataset consists of hundreds of thousands of small Parquet files. The data scientist suspects that the I/O is bottlenecked. What should the data scientist do to improve GPU utilization and training speed?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure the tuning job with early stopping enabled.. SageMaker's automatic model tuning supports early stopping. When enabled, the tuning job stops automatically if no significant improvement is observed. Option B is incorrect because CloudWatch alarms can monitor metrics but cannot directly stop a tuning job; they can trigger actions like notifications but not stop the tuning. Option C is incorrect because SageMaker Experiments is for tracking experiments, not for stopping tuning jobs. Option D is incorrect because SageMaker Debugger stops individual training jobs, not the hyperparameter tuning job itself; early stopping is a built-in feature of the tuning job.

Answer

Set up a CloudWatch alarm to stop the tuning job if a metric does not improve.

Answer

Use SageMaker Experiments to monitor and manually stop the tuning job.

Answer

Use SageMaker Debugger to stop training jobs that are not improving.

Question 13

A data scientist is training a model using Amazon SageMaker with a custom Docker container. The training job fails with an error: 'Resource exhausted: Out of memory'. The training data is stored in S3. What should the data scientist do to resolve this issue?

Accepted Answer

Increase the instance memory by selecting a larger instance type.. The 'Resource exhausted: Out of memory' error indicates that the training instance's RAM is insufficient for the workload. Selecting a larger instance type with more memory directly addresses the OOM condition by providing additional physical RAM for model parameters, data batches, and intermediate computations. In SageMaker, instance types like ml.p3.2xlarge (61 GB RAM) vs. ml.p3.8xlarge (244 GB RAM) offer different memory capacities, and upgrading resolves memory exhaustion without altering the training logic.

Answer

Increase the EBS volume size attached to the training instance.

Answer

Use Pipe mode for data loading instead of File mode.

Answer

Reduce the batch size in the training script.

Question 14

A machine learning engineer needs to deploy a model that performs real-time fraud detection. The model must be highly available and scalable. Which AWS service should be used to host the model?

Accepted Answer

Amazon SageMaker real-time endpoint. Amazon SageMaker real-time endpoints are designed for low-latency, synchronous inference, making them ideal for real-time fraud detection. They automatically scale across multiple instances and Availability Zones, providing high availability and elasticity to handle variable traffic loads without manual intervention.

Answer

AWS Lambda

Answer

Amazon ECS with a custom container

Answer

Amazon SageMaker batch transform

Question 15

A data scientist is using Amazon SageMaker Debugger to monitor training jobs. The training loss is decreasing but then suddenly spikes. What is the most likely cause and how should it be addressed?

Accepted Answer

Gradient explosion; apply gradient clipping.. A sudden spike in training loss after a period of decreasing loss is a classic symptom of gradient explosion, where gradients become excessively large during backpropagation, causing the model parameters to update erratically. Amazon SageMaker Debugger can monitor tensors and gradients in real time, and applying gradient clipping (e.g., via `max_grad_norm` in PyTorch or `clip_by_global_norm` in TensorFlow) directly addresses this by capping the gradient norm to prevent destabilizing updates.

Answer

Overfitting; apply regularization.

Answer

Learning rate too low; increase learning rate.

Answer

Vanishing gradients; use ReLU activation.