MLS-C01 Modeling Practice Test 36 — 15 Questions

Question 1

A data scientist is using SageMaker to train a model that requires access to a private S3 bucket in another account. The scientist has set up the correct IAM roles and bucket policies. However, the training job fails with an access denied error. What is the most likely cause?

Accepted Answer

The VPC used for training does not have a route to S3 (e.g., missing VPC endpoint or NAT). When a SageMaker training job runs inside a VPC (which is common for cross-account access), the VPC must have a route to S3, either through a VPC endpoint (Gateway or Interface) or a NAT gateway. Without such a route, the training instances cannot reach the private S3 bucket in the other account, even if IAM roles and bucket policies are correctly configured, resulting in an 'access denied' error because the network path is blocked.

Answer

The SageMaker execution role does not have s3:GetObject permission

Answer

The S3 objects are encrypted with SSE-KMS and the KMS key is not accessible

Answer

The training instance type does not support S3 access

Question 2

A company is using SageMaker Autopilot to automatically build a binary classification model. After the AutoML job completes, the data scientist wants to understand which features are most important for the best candidate model. How can the scientist get feature importance?

Accepted Answer

Open the SageMaker Autopilot job details and view the 'Explainability' tab. SageMaker Autopilot automatically generates a 'Explainability' tab within the job details for the best candidate model. This tab uses SHAP (SHapley Additive exPlanations) values to provide feature importance, showing which features most influence the model's predictions. The data scientist can directly access this information without any additional configuration or re-running the model.

Answer

Re-run the best model using SageMaker built-in XGBoost with the 'feature_importance' hyperparameter

Answer

Check the CloudWatch Logs for the training job

Answer

Use SageMaker Ground Truth to label a new dataset

Question 3

A team is using SageMaker to train a custom PyTorch model on a large dataset (10 TB) stored in S3. The training job is repeatedly failing due to 'OutOfMemory' errors on the GPU. The team is using a single ml.p3.8xlarge instance. Which change is most likely to resolve the issue?

Accepted Answer

Reduce the batch size in the training script. The 'OutOfMemory' error on the GPU indicates that the model and its associated data exceed the available GPU memory. Reducing the batch size directly decreases the memory footprint per training step, allowing the model to fit within the GPU's memory limits. This is the most direct and effective fix for GPU OOM errors, as it reduces the amount of data processed simultaneously without changing the instance type or input mode.

Answer

Change the instance type to ml.p3.16xlarge (more GPUs)

Answer

Use managed spot training to reduce cost

Answer

Switch the input mode from Pipe to File

Question 4

Which TWO options are best practices for training machine learning models using SageMaker? (Choose TWO.)

Accepted Answer

Use incremental training when you have new data that is similar to the original training data. Option B is correct because SageMaker's incremental training allows you to continue training an existing model with new data that shares the same schema and feature space, without retraining from scratch. This is a best practice when you have a steady stream of similar data, as it saves time and compute resources while preserving previously learned patterns.

Answer

Train the final model on the combined training and test sets to maximize data usage

Answer

Always use the largest possible instance type to minimize training time

Answer

Always enable checkpointing to save the model after every epoch

Question 5

Which THREE factors should be considered when choosing between SageMaker built-in algorithms and custom algorithms? (Choose THREE.)

Accepted Answer

Custom algorithms allow you to implement any architecture, including proprietary ones. Option A is correct because custom algorithms in SageMaker allow you to implement any architecture, including proprietary or novel models that are not available as built-in algorithms. This flexibility is essential when you need to use a custom neural network, a unique loss function, or a model from a research paper that SageMaker does not natively support.

Answer

Built-in algorithms can only be used with CSV and JSON formats

Answer

Custom algorithms require you to bring your own Docker container, but SageMaker built-in algorithms do not support frameworks like PyTorch

Question 6

Which TWO actions can help reduce overfitting when training a model on SageMaker? (Choose TWO.)

Accepted Answer

Use early stopping based on validation error. Early stopping monitors the validation error during training and halts the process when the error stops improving, preventing the model from learning noise in the training data. This directly reduces overfitting by ensuring the model does not continue to fit to spurious patterns after generalization has peaked.

Answer

Reduce the amount of training data

Answer

Increase the maximum depth of the trees in XGBoost

Answer

Increase the number of training epochs

Question 7

An IAM policy attached to a SageMaker execution role is shown. A training job executed with this role fails with an error that the role cannot access the S3 bucket. The training job uses input data from s3://my-bucket/train/data.csv and output to s3://my-bucket/output/. What is the most likely cause?

Accepted Answer

The training job does not have s3:PutObject permission for the output location. Option C is correct because the error message indicates the role cannot access the S3 bucket, which typically occurs when the role lacks write permissions to the output location. The training job needs s3:PutObject permission to write the output artifacts (model, logs, etc.) to s3://my-bucket/output/. Without this permission, SageMaker fails to save the training results, resulting in an access error.

Answer

The training job does not have s3:GetObject permission for the input data

Answer

The training data is encrypted with SSE-KMS and the role lacks KMS permissions

Answer

The S3 bucket is in a different region than the training job

Question 8

A data scientist is troubleshooting a failed SageMaker training job that uses a custom Docker image. The failure reason shows 'unrecognized arguments: --sagemaker_program'. What is the most likely cause?

Accepted Answer

The custom Docker image does not use the SageMaker training toolkit and thus does not accept SageMaker hyperparameters. The error 'unrecognized arguments: --sagemaker_program' indicates that the custom Docker image does not include the SageMaker Training Toolkit. The SageMaker Training Toolkit is a Python library that provides a default entry point to parse and handle SageMaker-specific hyperparameters (like --sagemaker_program, --sagemaker_submit_directory, etc.). Without this toolkit, the container's entry point does not recognize these arguments, causing the training job to fail.

Answer

The Docker image is tagged incorrectly and cannot be pulled

Answer

The training job is in a different region than the ECR repository

Answer

The input mode is File mode, but the container expects Pipe mode

Question 9

A data scientist is reviewing the training logs from a SageMaker training job. The logs show training and validation loss per epoch. Based on the exhibited logs, which statement is correct?

Accepted Answer

The model is overfitting because training loss decreases while validation loss does not. Option D is correct because the training logs show a classic overfitting pattern: training loss consistently decreases across epochs, indicating the model is memorizing the training data, while validation loss does not decrease (or may even increase), indicating poor generalization to unseen data. In SageMaker, monitoring both losses during training is critical to detect overfitting early, often prompting regularization or early stopping.

Answer

The model is not learning because the loss is not decreasing

Answer

The model is underfitting because both losses are high

Answer

The model is performing well because validation loss is stable

Question 10

A data scientist is training a deep learning model on Amazon SageMaker for image classification. The training is taking a long time and the GPU utilization is consistently below 30%. What should the data scientist do to improve GPU utilization and reduce training time?

Accepted Answer

Increase the batch size.. Low GPU utilization (below 30%) indicates that the GPU is spending most of its time waiting for data to process, often due to small batch sizes that underutilize the GPU's parallel compute capacity. Increasing the batch size allows the GPU to process more samples per forward/backward pass, improving arithmetic intensity and hardware utilization, which directly reduces total training time on SageMaker.

Answer

Use early stopping to stop training earlier.

Answer

Switch to a CPU-only instance.

Answer

Reduce the number of layers in the model.

Question 11

A machine learning engineer is building a binary classification model to predict customer churn. The dataset is highly imbalanced (5% churn). The engineer wants to use Amazon SageMaker's built-in XGBoost algorithm. Which combination of hyperparameters is most appropriate for this scenario?

Accepted Answer

scale_pos_weight=19, subsample=0.8. In a highly imbalanced dataset with only 5% churn, the ratio of negative to positive classes is 95:5, or 19:1. The `scale_pos_weight` hyperparameter in XGBoost should be set to this ratio (19) to penalize misclassifications of the minority class more heavily. A `subsample` of 0.8 introduces stochasticity and helps prevent overfitting, which is especially important when the minority class is small.

Answer

scale_pos_weight=0.05, subsample=0.8

Answer

scale_pos_weight=19, subsample=1.0

Answer

scale_pos_weight=1, subsample=1.0

Question 12

A company wants to deploy a machine learning model that provides real-time inference with low latency. The model is a small ensemble of three tree-based models. Which Amazon SageMaker approach is most appropriate?

Accepted Answer

Use a SageMaker real-time endpoint with a single inference container.. A SageMaker real-time endpoint with a single inference container is the most appropriate approach because it provides persistent, low-latency inference by keeping the model loaded in memory and handling requests synchronously. For a small ensemble of three tree-based models, a single container can host all models (e.g., using a custom inference script or a multi-model endpoint) and deliver sub-second response times, meeting the real-time requirement.

Answer

Use a SageMaker batch transform job.

Answer

Use AWS Lambda with the model packaged in a layer.

Answer

Use a SageMaker Serverless Inference endpoint.

Question 13

A data scientist is using Amazon SageMaker to train a custom TensorFlow model. The training job is failing with the error: 'OutOfRangeError: End of sequence'. The input data is stored in TFRecord format in S3. What is the most likely cause?

Accepted Answer

The number of training steps or epochs specified exceeds the dataset size.. The 'OutOfRangeError: End of sequence' error in TensorFlow occurs when the training loop attempts to read more data than is available in the dataset. This typically happens when the number of training steps or epochs specified exceeds the total number of records in the TFRecord files, causing the iterator to reach the end of the dataset prematurely.

Answer

The TFRecord files are corrupted.

Answer

The instance type does not have enough memory.

Answer

The shuffle buffer size is too large.

Question 14

A company is using Amazon SageMaker to train a model on a dataset with many categorical features. They want to use SageMaker's built-in Linear Learner algorithm. What preprocessing step is required for the categorical features?

Accepted Answer

Apply one-hot encoding to convert them to numerical vectors.. The SageMaker Linear Learner algorithm requires numerical input features. Categorical features must be converted to numerical vectors, typically via one-hot encoding, because the algorithm performs linear regression or classification on numerical data. Without this preprocessing, the algorithm cannot interpret categorical values directly.

Answer

Use label encoding to assign integers to categories.

Answer

Normalize the categorical features using min-max scaling.

Answer

Remove categorical features with high cardinality.

Question 15

A data scientist is tuning a neural network on a small dataset and observes that the training loss decreases but validation loss increases after a few epochs. Which technique should be applied to mitigate this issue?

Accepted Answer

Add dropout layers to the model.. Overfitting occurs when validation loss increases. Dropout is a regularization technique that reduces overfitting. Learning rate decay may help convergence but not specifically for overfitting. Batch normalization helps training stability. Data augmentation is useful but not always applicable.

Answer

Increase the learning rate.

Answer

Remove regularization terms from the loss function.

Answer

Increase the number of epochs.