MLS-C01 Practice Test 26 — 15 Questions

Question 1

A data scientist is training a deep learning model for image classification on Amazon SageMaker. The dataset consists of 10,000 images of size 224x224 pixels. The training job uses a single ml.p3.2xlarge instance. The data scientist notices that the GPU utilization is very low (~20%) and the training is slow. Which change would most likely improve GPU utilization?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the number of data loader workers to load data in parallel. Low GPU utilization (~20%) during training typically indicates the GPU is starved for data — the CPU-bound data loading pipeline cannot feed batches fast enough. Increasing the number of data loader workers (D) parallelizes image decoding, augmentation, and batching across CPU cores, keeping the GPU busy. This is the classic fix for I/O-bound training on a single GPU instance.

Answer

Use gradient accumulation

Answer

Use a larger instance type with more GPUs

Answer

Increase the batch size

Question 2

A data scientist is building a binary classifier and obtains the following confusion matrix on the test set: TP=80, FP=20, TN=70, FN=30. What is the precision?

Accepted Answer

0.8. Precision is defined as TP / (TP + FP), which measures the proportion of positive predictions that are actually correct. Plugging in the values: 80 / (80 + 20) = 80 / 100 = 0.8. This matches option B exactly, confirming it as the correct answer.

Answer

0.727

Answer

0.75

Answer

0.762

Question 3

A data scientist is training a binary classification model on a dataset with 100,000 positive samples and 1,000 negative samples. The model achieves 99% accuracy on the test set but a very low F1 score. What is the most likely cause?

Accepted Answer

The model predicts the majority class most of the time due to class imbalance. With 100,000 positive and 1,000 negative samples, the dataset is highly imbalanced (100:1 ratio). A model that simply predicts the majority class (positive) for all instances achieves 99% accuracy but fails to identify the minority class, resulting in a very low F1 score. This is the classic accuracy paradox in imbalanced classification.

Answer

The test set contains only positive samples

Answer

The model is overfitting due to too many features

Answer

The model is underfitting due to insufficient training

Question 4

A company is using Amazon SageMaker to deploy a model for real-time inference. The model is a deep neural network that requires GPU for low latency. The endpoint currently uses a single ml.p3.2xlarge instance. Traffic is expected to increase by 5x. Which TWO actions should the company take to handle the increased traffic?

Accepted Answer

Use a larger instance type with more GPUs. Option A is correct because moving to a larger GPU instance type (e.g., ml.p3.8xlarge or ml.p3.16xlarge, which have 4 or 8 NVIDIA V100 GPUs respectively) increases per-instance compute and memory capacity, allowing each endpoint instance to serve more concurrent inference requests with the low latency the deep neural network requires. Option C is correct because SageMaker supports automatic scaling of real-time endpoints via Application Auto Scaling, which adjusts the number of instances based on metrics such as InvocationsPerInstance or ModelLatency, so the endpoint can elastically add instances to absorb the 5x traffic increase. Option B is not appropriate because switching to a CPU-based instance would remove the GPU acceleration the model needs for low latency. Option D is not appropriate because multi-model endpoints are designed to host many models behind a single endpoint to save hosting costs, not to scale throughput for one latency-sensitive GPU model. Option E is not appropriate because decreasing the batch size reduces per-request work but does not add capacity, and it can actually lower GPU utilization and throughput.

Answer

Switch to a CPU-based instance

Answer

Use a multi-model endpoint

Answer

Decrease the batch size

Question 5

A machine learning team is deploying a real-time inference endpoint on Amazon SageMaker for a model that requires low latency (<100 ms). The model is a PyTorch model with custom pre- and post-processing logic. The team uses a SageMaker Model with a custom inference container. After deployment, they observe that the endpoint takes over 500 ms for the first request, but subsequent requests are fast (~50 ms). What is the MOST likely cause?

Accepted Answer

The container has a cold start delay because the model needs to be loaded into memory from Amazon S3 on the first request.. The most likely cause is a cold start delay: the custom inference container must download the model artifacts from Amazon S3 and load them into memory on the first invocation. Subsequent requests reuse the loaded model, resulting in fast inference. This is a common pattern in SageMaker when the model is not pre-loaded or when the container lacks a warm-up mechanism.

Answer

The instance type is too small to handle the model size.

Answer

The model is too large and exceeds the instance memory.

Answer

The endpoint is not configured with auto-scaling.

Question 6

A company is using Amazon SageMaker to train a large language model with billions of parameters. The training job uses multiple GPU instances in a distributed fashion. The training is converging but the loss is not decreasing as expected. The data scientist suspects that the learning rate is too high. Which technique should the data scientist use to automatically adjust the learning rate during training?

Accepted Answer

Implement learning rate scheduling with a cosine annealing schedule. Cosine annealing is a learning rate schedule that starts with a relatively high learning rate and gradually decreases it following a cosine curve, which helps the model converge when the initial rate is too high. It automatically adjusts the learning rate during training without manual intervention, matching the requirement. This is a standard technique for large-model training where a fixed high rate causes the loss to plateau or oscillate.

Answer

Use a fixed learning rate and train for more epochs

Answer

Increase the batch size to reduce variance

Answer

Use gradient clipping

Question 7

A team deployed a SageMaker endpoint with the configuration shown in the exhibit. During a traffic spike, the endpoint becomes unresponsive. Which change to the endpoint configuration would best improve availability?

Accepted Answer

Configure auto-scaling for the endpoint. The endpoint is unresponsive during a traffic spike because the fixed instance count cannot handle the increased load. SageMaker endpoints support automatic scaling, which dynamically adjusts the number of instances based on metrics like InvocationsPerInstance or CPUUtilization. Configuring auto-scaling allows the endpoint to add instances during spikes, thereby maintaining availability and performance. This is the best change because it directly addresses the root cause—insufficient capacity under variable load—without over-provisioning.

Answer

Reduce the initial instance count to 0 and use on-demand invocation

Answer

Add a second production variant with the same model

Answer

Change the instance type to ml.m5.xlarge

Question 8

A data scientist is analyzing a dataset and finds that the target variable has a bimodal distribution. Which preprocessing step is most appropriate before modeling?

Accepted Answer

Consider clustering to separate the two modes and model them separately.. A bimodal target distribution suggests the data may come from two distinct subpopulations or generating processes. Rather than forcing a single model to fit both modes, the most appropriate preprocessing step is to investigate whether clustering can separate the two modes so each can be modeled separately (e.g., a mixture-of-experts or two-model approach). This preserves the underlying structure instead of distorting it with transformations that assume unimodality.

Answer

Standardize the target variable to have mean 0 and variance 1.

Answer

Remove outliers from the target variable.

Answer

Apply a log transformation to the target variable.

Question 9

A company wants to deploy a machine learning model that requires very low latency predictions (under 10ms). The model is a small ensemble of decision trees. Which SageMaker deployment option is most suitable?

Accepted Answer

SageMaker endpoint with a single instance. C is correct because a SageMaker endpoint with a single instance provides a persistent, real-time inference API that can achieve sub-10ms latency for a small ensemble of decision trees. The endpoint keeps the model loaded in memory and uses synchronous HTTP requests, minimizing cold start and network overhead, which is essential for low-latency predictions.

Answer

SageMaker Notebook instance

Answer

AWS Lambda function with the model packaged

Answer

SageMaker Batch Transform

Question 10

A data scientist is training a binary classification model on a highly imbalanced dataset (0.1% positive class). To improve recall, the team decides to use SageMaker's built-in XGBoost algorithm. Which parameter adjustment is most likely to increase recall without significantly sacrificing precision?

Accepted Answer

Set scale_pos_weight to the ratio of negative to positive samples. Setting scale_pos_weight to the ratio of negative to positive samples (approximately 999:1) tells XGBoost to assign a higher penalty to misclassifications of the minority positive class. This directly increases the gradient contribution from positive samples during training, which shifts the decision boundary to improve recall while maintaining a balance that avoids excessive false positives, thus preserving precision.

Answer

Increase max_depth from 5 to 10

Answer

Reduce num_round from 100 to 50

Answer

Increase subsample from 0.8 to 1.0

Question 11

A company runs a nightly AWS Glue ETL job that processes data from an Amazon Redshift cluster and writes to Amazon S3. The job fails intermittently with 'ERROR: cannot execute INSERT in a read-only transaction'. What is the most likely cause?

Accepted Answer

The Redshift cluster is in a read-only state due to a failover or maintenance. The error 'cannot execute INSERT in a read-only transaction' is a Redshift-side error, not a client-side permission or driver issue. When a Redshift cluster is in read-only mode — which happens during a failover to a new leader node, during certain maintenance windows, or when the cluster is being restored — all write transactions are rejected with exactly this message. The Glue job's INSERT statements therefore fail until the cluster returns to a writable state.

Answer

The IAM role used by Glue does not have permissions to insert into Redshift

Answer

The JDBC driver version is incompatible with the Redshift cluster

Answer

The Glue job's connection pool is exhausted

Question 12

A data scientist needs to store and version machine learning models, along with metadata such as hyperparameters and metrics. Which AWS service is designed for this purpose?

Accepted Answer

Amazon SageMaker Model Registry. Amazon SageMaker Model Registry is purpose-built for cataloging, versioning, and managing machine learning models. It stores model artifacts, metadata such as hyperparameters and metrics, and supports approval workflows for deployment. This directly matches the requirement to store and version models with associated metadata.

Answer

Amazon S3 with versioning enabled

Answer

Amazon DynamoDB

Answer

Amazon Elastic Container Registry (ECR)

Question 13

A data scientist is using Amazon SageMaker to perform hyperparameter tuning for a neural network. The tuning job uses the 'Random' search strategy. After 10 training jobs, the best objective metric has plateaued. The scientist wants to improve the results without increasing the total number of training jobs. Which approach should they take?

Accepted Answer

Switch the hyperparameter tuning strategy to 'Bayesian'. Bayesian search uses a probabilistic model of the objective function to choose the next hyperparameter configuration, focusing trials in promising regions based on prior results. This is more sample-efficient than random search, so it can find better hyperparameters within the same number of training jobs.

Answer

Use a different objective metric that is easier to optimize

Answer

Normalize the input features to have zero mean and unit variance

Answer

Increase the maximum number of training jobs

Question 14

Which TWO of the following are valid reasons to use a sample of the data during exploratory data analysis instead of the full dataset? (Select TWO.)

Accepted Answer

Reduce memory usage and computation time. Option D is correct because working with a sample reduces the volume of data loaded into memory and the number of computations required, which lowers memory usage and speeds up processing during exploratory data analysis. Option E is correct because sampling makes it feasible to render and interact with visualizations in real time when the full dataset is too large for interactive plotting tools to handle smoothly. Options A, B, and C are not valid reasons: sampling does not remove bias from the original dataset (it can even introduce sampling bias), it does not guarantee that rare events are captured (rare events may be missed unless specialized sampling is used), and it does not inherently improve model accuracy by reducing noise, since sampling discards data rather than cleaning it.

Answer

Remove bias from the original dataset

Answer

Ensure rare events are captured in the analysis

Answer

Improve model accuracy by reducing noise

Question 15

A team is using SageMaker to train a model. They want to track hyperparameters, metrics, and model artifacts. Which SageMaker feature should they use?

Accepted Answer

SageMaker Experiments. SageMaker Experiments is the correct choice because it is purpose-built for tracking hyperparameters, metrics, and model artifacts across training runs. It automatically captures input parameters, output metrics, and artifact locations (e.g., S3 paths) for each trial, enabling comparison and lineage tracking without manual logging.

Answer

SageMaker Pipelines

Answer

SageMaker Debugger

Answer

SageMaker Model Registry