MLS-C01 Modeling Practice Test 36 — 15 Questions

Question 1

A company is deploying a real-time inference endpoint using SageMaker. The model is a large deep learning model (5 GB) with strict latency requirements (< 100 ms per request). The team expects bursty traffic with up to 1000 requests per second. Which configuration best meets the latency and throughput requirements?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Deploy an ml.p3.2xlarge instance with automatic scaling based on a custom metric like 'InvocationsPerInstance'. Deploying on an ml.p3.2xlarge instance with automatic scaling based on 'InvocationsPerInstance' allows the endpoint to handle bursty traffic up to 1000 requests per second while maintaining sub-100 ms latency. The GPU-accelerated p3 instance provides the necessary compute for a 5 GB deep learning model, and custom scaling on invocations per instance ensures that additional instances are provisioned quickly during traffic spikes without over-provisioning.

Answer

Use a multi-model endpoint with ml.c5.4xlarge instances

Answer

Use SageMaker Serverless Inference with a memory size of 6 GB

Answer

Deploy a single ml.p3.16xlarge instance with a production variant

Question 2

A data scientist is using SageMaker to train a linear learner algorithm. After training, the evaluation shows that the model has high bias. Which action is most likely to reduce bias?

Accepted Answer

Add feature crosses for categorical variables. High bias indicates that the model is underfitting the data, meaning it is too simple to capture the underlying patterns. Adding feature crosses for categorical variables creates interaction features that allow the linear learner to model non-linear relationships, increasing model complexity and reducing bias. This is a standard technique in linear models to address underfitting without switching to a non-linear algorithm.

Answer

Increase the L2 regularization strength

Answer

Reduce the amount of training data

Answer

Remove some features that have low variance

Question 3

A company uses SageMaker built-in BlazingText algorithm for text classification. The model performance is poor on the validation set. The data consists of short documents (average 50 words). Which hyperparameter tuning strategy is most likely to improve performance?

Accepted Answer

Decrease window size from 5 to 2. BlazingText's default window size of 5 may be too large for short documents (average 50 words), causing the model to learn overly broad context that dilutes local semantic patterns. Decreasing the window size to 2 forces the model to focus on tighter word co-occurrences, which is more effective for short text classification where local n-gram signals are critical.

Answer

Increase bucket size from 0 to 1000000

Answer

Increase vector dimension from 100 to 300

Answer

Increase minCount from 1 to 5

Question 4

During training of a SageMaker built-in object detection algorithm, the loss is not decreasing after several epochs. Which troubleshooting step should be taken first?

Accepted Answer

Check whether the learning rate is appropriate. When the loss is not decreasing during training of a SageMaker built-in object detection algorithm, the most common cause is an inappropriate learning rate. A learning rate that is too high can cause the loss to oscillate or diverge, while one that is too low can cause the loss to plateau. Checking and adjusting the learning rate is the first troubleshooting step because it directly controls the step size of gradient updates and is a fundamental hyperparameter in optimization.

Answer

Increase the mini-batch size

Answer

Add more classes to the dataset

Answer

Increase the number of epochs

Question 5

A machine learning team is using SageMaker to train a custom TensorFlow model on a dataset that fits in memory. The training job is taking too long. The team wants to reduce training time without changing the model architecture. Which approach is most effective?

Accepted Answer

Switch the input mode from File to Pipe. Switching the input mode from File to Pipe is the most effective approach because it streams data directly from Amazon S3 to the training container, eliminating the need to download the entire dataset to the local storage before training begins. This reduces the I/O bottleneck and significantly cuts down the time spent on data loading, especially for datasets that fit in memory, as the model can start training almost immediately while data is being streamed.

Answer

Use SageMaker managed spot training

Answer

Use Amazon EFS as the input data source instead of S3

Answer

Use a larger instance type with more vCPUs

Question 6

A data scientist is using SageMaker to train a model that requires access to a private S3 bucket in another account. The scientist has set up the correct IAM roles and bucket policies. However, the training job fails with an access denied error. What is the most likely cause?

Accepted Answer

The VPC used for training does not have a route to S3 (e.g., missing VPC endpoint or NAT). When a SageMaker training job runs inside a VPC (which is common for cross-account access), the VPC must have a route to S3, either through a VPC endpoint (Gateway or Interface) or a NAT gateway. Without such a route, the training instances cannot reach the private S3 bucket in the other account, even if IAM roles and bucket policies are correctly configured, resulting in an 'access denied' error because the network path is blocked.

Answer

The SageMaker execution role does not have s3:GetObject permission

Answer

The S3 objects are encrypted with SSE-KMS and the KMS key is not accessible

Answer

The training instance type does not support S3 access

Question 7

A company is using SageMaker Autopilot to automatically build a binary classification model. After the AutoML job completes, the data scientist wants to understand which features are most important for the best candidate model. How can the scientist get feature importance?

Accepted Answer

Open the SageMaker Autopilot job details and view the 'Explainability' tab. SageMaker Autopilot automatically generates a 'Explainability' tab within the job details for the best candidate model. This tab uses SHAP (SHapley Additive exPlanations) values to provide feature importance, showing which features most influence the model's predictions. The data scientist can directly access this information without any additional configuration or re-running the model.

Answer

Re-run the best model using SageMaker built-in XGBoost with the 'feature_importance' hyperparameter

Answer

Check the CloudWatch Logs for the training job

Answer

Use SageMaker Ground Truth to label a new dataset

Question 8

A team is using SageMaker to train a custom PyTorch model on a large dataset (10 TB) stored in S3. The training job is repeatedly failing due to 'OutOfMemory' errors on the GPU. The team is using a single ml.p3.8xlarge instance. Which change is most likely to resolve the issue?

Accepted Answer

Reduce the batch size in the training script. The 'OutOfMemory' error on the GPU indicates that the model and its associated data exceed the available GPU memory. Reducing the batch size directly decreases the memory footprint per training step, allowing the model to fit within the GPU's memory limits. This is the most direct and effective fix for GPU OOM errors, as it reduces the amount of data processed simultaneously without changing the instance type or input mode.

Answer

Change the instance type to ml.p3.16xlarge (more GPUs)

Answer

Use managed spot training to reduce cost

Answer

Switch the input mode from Pipe to File

Question 9

Which TWO options are best practices for training machine learning models using SageMaker? (Choose TWO.)

Accepted Answer

Use incremental training when you have new data that is similar to the original training data. SageMaker's incremental training allows you to continue training an existing model with new data that shares the same schema and feature space, without retraining from scratch. This is a best practice when you have a steady stream of similar data, as it saves time and compute resources while preserving previously learned patterns.

Answer

Train the final model on the combined training and test sets to maximize data usage

Answer

Always use the largest possible instance type to minimize training time

Answer

Always enable checkpointing to save the model after every epoch

Question 10

Which THREE factors should be considered when choosing between SageMaker built-in algorithms and custom algorithms? (Choose THREE.)

Accepted Answer

Custom algorithms allow you to implement any architecture, including proprietary ones. Custom algorithms in SageMaker allow you to implement any architecture, including proprietary or novel models that are not available as built-in algorithms. This flexibility is essential when you need to use a custom neural network, a unique loss function, or a model from a research paper that SageMaker does not natively support.

Answer

Built-in algorithms can only be used with CSV and JSON formats

Answer

Custom algorithms require you to bring your own Docker container, but SageMaker built-in algorithms do not support frameworks like PyTorch

Question 11

An IAM policy attached to a SageMaker execution role is shown. A training job executed with this role fails with an error that the role cannot access the S3 bucket. The training job uses input data from s3://my-bucket/train/data.csv and output to s3://my-bucket/output/. What is the most likely cause?

Accepted Answer

The training job does not have s3:PutObject permission for the output location. The error message indicates the role cannot access the S3 bucket, which typically occurs when the role lacks write permissions to the output location. The training job needs s3:PutObject permission to write the output artifacts (model, logs, etc.) to s3://my-bucket/output/. Without this permission, SageMaker fails to save the training results, resulting in an access error.

Answer

The training job does not have s3:GetObject permission for the input data

Answer

The training data is encrypted with SSE-KMS and the role lacks KMS permissions

Answer

The S3 bucket is in a different region than the training job

Question 12

A data scientist is troubleshooting a failed SageMaker training job that uses a custom Docker image. The failure reason shows 'unrecognized arguments: --sagemaker_program'. What is the most likely cause?

Accepted Answer

The custom Docker image does not use the SageMaker training toolkit and thus does not accept SageMaker hyperparameters. The error 'unrecognized arguments: --sagemaker_program' indicates that the custom Docker image does not include the SageMaker Training Toolkit. The SageMaker Training Toolkit is a Python library that provides a default entry point to parse and handle SageMaker-specific hyperparameters (like --sagemaker_program, --sagemaker_submit_directory, etc.). Without this toolkit, the container's entry point does not recognize these arguments, causing the training job to fail.

Answer

The Docker image is tagged incorrectly and cannot be pulled

Answer

The training job is in a different region than the ECR repository

Answer

The input mode is File mode, but the container expects Pipe mode

Question 13

A data scientist is reviewing the training logs from a SageMaker training job. The logs show training and validation loss per epoch. Based on the exhibited logs, which statement is correct?

Accepted Answer

The model is overfitting because training loss decreases while validation loss does not. The training logs show a classic overfitting pattern: training loss consistently decreases across epochs, indicating the model is memorizing the training data, while validation loss does not decrease (or may even increase), indicating poor generalization to unseen data. In SageMaker, monitoring both losses during training is critical to detect overfitting early, often prompting regularization or early stopping.

Answer

The model is not learning because the loss is not decreasing

Answer

The model is underfitting because both losses are high

Answer

The model is performing well because validation loss is stable

Question 14

A data scientist is training a deep learning model on Amazon SageMaker for image classification. The training is taking a long time and the GPU utilization is consistently below 30%. What should the data scientist do to improve GPU utilization and reduce training time?

Accepted Answer

Increase the batch size.. Low GPU utilization (below 30%) indicates that the GPU is spending most of its time waiting for data to process, often due to small batch sizes that underutilize the GPU's parallel compute capacity. Increasing the batch size allows the GPU to process more samples per forward/backward pass, improving arithmetic intensity and hardware utilization, which directly reduces total training time on SageMaker.

Answer

Use early stopping to stop training earlier.

Answer

Switch to a CPU-only instance.

Answer

Reduce the number of layers in the model.

Question 15

A machine learning engineer is building a binary classification model to predict customer churn. The dataset is highly imbalanced (5% churn). The engineer wants to use Amazon SageMaker's built-in XGBoost algorithm. Which combination of hyperparameters is most appropriate for this scenario?

Accepted Answer

scale_pos_weight=19, subsample=0.8. In a highly imbalanced dataset with only 5% churn, the ratio of negative to positive classes is 95:5, or 19:1. The `scale_pos_weight` hyperparameter in XGBoost should be set to this ratio (19) to penalize misclassifications of the minority class more heavily. A `subsample` of 0.8 introduces stochasticity and helps prevent overfitting, which is especially important when the minority class is small.

Answer

scale_pos_weight=0.05, subsample=0.8

Answer

scale_pos_weight=19, subsample=1.0

Answer

scale_pos_weight=1, subsample=1.0