MLS-C01 Modeling Practice Test 24 — 15 Questions

Question 1

A company uses SageMaker to train a model. The training job fails with 'ResourceLimitExceeded' error. Which TWO actions should the company take to resolve this?

Accepted Answer

Use a different instance type that is not at its limit.. Option B is correct because the 'ResourceLimitExceeded' error indicates that the requested instance type has reached its concurrent usage limit in the current AWS region. Switching to a different instance type that is not at its limit allows the training job to proceed without exceeding the service quota. Option E is correct because requesting a service limit increase for SageMaker training job resources directly raises the cap on the number of concurrent instances or total instance count, resolving the underlying quota issue.

Answer

Launch the training job in a different AWS region.

Answer

Use SageMaker Managed Spot Training to reduce cost.

Answer

Compress the training data to reduce storage requirements.

Question 2

A data scientist is building a recommender system using Amazon SageMaker. The dataset contains user-item interactions with implicit feedback (clicks). Which THREE evaluation metrics are appropriate for this use case?

Accepted Answer

Precision@k. Precision@k is appropriate for implicit feedback (clicks) because it measures the proportion of relevant items among the top-k recommendations, focusing on the accuracy of the ranked list. In recommender systems with implicit feedback, where only positive interactions are observed, ranking metrics like Precision@k are standard as they evaluate the quality of the top recommendations without requiring explicit ratings.

Answer

Root Mean Squared Error (RMSE)

Answer

Area Under the ROC Curve (AUC-ROC)

Question 3

A data scientist is trying to create a SageMaker endpoint using an IAM role with the attached policy. The operation fails with 'AccessDenied'. What is the MOST likely cause?

Accepted Answer

The policy does not allow sagemaker:InvokeEndpoint.. The error 'AccessDenied' when creating a SageMaker endpoint indicates that the IAM role lacks the necessary permissions for the specific API call being made. Since the operation is to create an endpoint, the required action is sagemaker:CreateEndpoint, not sagemaker:InvokeEndpoint. Option A is incorrect because InvokeEndpoint is used to invoke a deployed endpoint for inference, not to create it. The most likely cause is that the policy does not include sagemaker:CreateEndpoint.

Answer

The policy does not allow access to the S3 bucket for model artifacts.

Answer

The policy does not allow sagemaker:CreateEndpoint.

Answer

The policy does not allow sagemaker:CreateModel.

Question 4

A training job fails with the error shown. The training script expects a file named 'train.csv' in the 'training' channel. What is the most likely cause?

Accepted Answer

The 'train.csv' file is located inside a subfolder within 's3://my-bucket/data/', and the script expects it directly in the channel path.. The error indicates that the training script cannot find 'train.csv' in the expected location. When SageMaker copies data from an S3 channel path (e.g., 's3://my-bucket/data/') to the training instance, it places the contents of that S3 prefix directly into the channel directory (e.g., '/opt/ml/input/data/training/'). If the CSV file is inside a subfolder (e.g., 's3://my-bucket/data/subfolder/train.csv'), the script will not find it at the top level of the channel path, causing a 'file not found' error.

Answer

The S3 bucket policy denies access to the 'train.csv' file.

Answer

The channel name in the input data configuration does not match the script's expected channel name.

Answer

The training script has a bug that prevents it from reading the file.

Question 5

A CloudFormation stack creation failed. The SageMaker endpoint resource shows CREATE_FAILED. What is the most likely issue?

Accepted Answer

The S3 bucket 'my-bucket' does not contain the object 'model.tar.gz'.. The correct answer is B because a CREATE_FAILED status on a SageMaker endpoint resource during CloudFormation stack creation most commonly indicates that the model artifact specified in the Model definition cannot be located. SageMaker requires the S3 bucket and object path (e.g., 's3://my-bucket/model.tar.gz') to exist and be accessible at the time of model creation. If the object is missing, the model resource fails, cascading to the endpoint creation failure.

Answer

The IAM role used by CloudFormation lacks permissions to create endpoints.

Answer

The SageMaker endpoint configuration is invalid.

Answer

The instance type specified for the endpoint is not available in the region.

Question 6

A company is building a binary classifier to detect fraudulent transactions. The dataset is highly imbalanced with only 0.1% positive cases. The data scientist uses logistic regression and obtains 99.9% accuracy on the test set. Which metric should the data scientist use to evaluate the model's performance?

Accepted Answer

Precision-recall curve. With only 0.1% positive cases, accuracy is misleading because a model that always predicts 'not fraudulent' achieves 99.9% accuracy. The precision-recall curve focuses on the positive class and is robust to extreme class imbalance, showing the trade-off between precision and recall across thresholds. This makes it the best choice for evaluating a binary classifier on highly imbalanced fraud detection data.

Answer

ROC AUC

Answer

Precision

Answer

F1 score

Question 7

A data scientist trains a gradient boosting model on a large dataset using SageMaker. The training completes successfully, but when deploying the model to a real-time endpoint, inference latency is too high. Which change is MOST likely to reduce latency without significant accuracy loss?

Accepted Answer

Prune the trees by removing nodes with low importance. Pruning trees by removing nodes with low importance reduces the model's complexity, which directly decreases inference latency because fewer decision paths need to be evaluated. In gradient boosting, this can be done with minimal accuracy loss if the removed nodes correspond to splits that contribute little to the overall prediction, as measured by feature importance or gain.

Answer

Use a larger instance type for the endpoint

Answer

Increase the number of trees in the ensemble

Answer

Use SageMaker Batch Transform instead of real-time

Question 8

A data scientist is training a linear regression model and notices that the model performs well on training data but poorly on validation data. Which technique should be applied to reduce overfitting?

Accepted Answer

Apply L2 regularization (Ridge). L2 regularization (Ridge) adds a penalty term proportional to the square of the magnitude of the coefficients to the loss function. This shrinks the weights toward zero, reducing the model's sensitivity to individual features and preventing it from fitting noise in the training data, which directly addresses overfitting.

Answer

Increase the number of epochs

Answer

Add more features

Answer

Remove training examples

Question 9

A data scientist is using SageMaker to train a deep learning model for image classification. The training job is taking too long. Which approach can reduce training time?

Accepted Answer

Use SageMaker's distributed data parallelism. SageMaker's distributed data parallelism splits the training data across multiple GPUs or instances, allowing each worker to process a different subset of the data simultaneously. This reduces the wall-clock time per epoch by parallelizing the computation, which directly addresses the 'taking too long' issue for deep learning image classification models.

Answer

Use SageMaker Neo to compile the model

Answer

Increase the number of epochs

Answer

Use a smaller image size

Question 10

A data scientist is building a model to predict customer churn. The dataset contains categorical features with high cardinality (e.g., ZIP code, customer ID). Which encoding method is MOST suitable?

Accepted Answer

Target encoding. Target encoding is most suitable for high-cardinality categorical features because it replaces each category with the mean of the target variable for that category, effectively capturing the predictive signal while keeping the feature space dense. This avoids the curse of dimensionality from one-hot encoding and the arbitrary ordinality of label encoding, which can mislead tree-based models.

Answer

One-hot encoding

Answer

Label encoding

Answer

Hashing encoding

Question 11

A data scientist is training a model and wants to monitor training progress. Which AWS service can be used to track metrics like loss and accuracy in real time?

Accepted Answer

Amazon CloudWatch. Amazon CloudWatch is the correct service because it provides real-time monitoring of metrics such as loss and accuracy during model training. When using SageMaker, training jobs automatically emit metrics to CloudWatch via the CloudWatch agent, allowing you to view logs and set alarms on metric thresholds in near real-time.

Answer

Amazon SageMaker Ground Truth

Answer

Amazon SageMaker Automatic Model Tuning

Answer

AWS Glue

Question 12

A company is using SageMaker to train a model, but the training job fails with an out-of-memory error. Which action should the data scientist take to resolve this issue?

Accepted Answer

Decrease the batch size. Decreasing the batch size reduces the memory footprint per training step, directly addressing the out-of-memory (OOM) error. In SageMaker, the training instance's GPU or CPU memory is shared between model parameters, activations, and the batch data; a smaller batch size lowers the peak memory usage, allowing the training job to complete without exceeding the instance's memory limit.

Answer

Use a larger instance type for training

Answer

Increase the learning rate

Answer

Increase the number of layers

Question 13

A data scientist is building a recommender system using collaborative filtering. The dataset is sparse (99% missing values). Which algorithm is best suited?

Accepted Answer

Matrix Factorization (e.g., SVD). Matrix factorization (e.g., SVD) is best suited for sparse collaborative filtering because it learns latent factors that capture underlying user-item interactions, effectively handling the 99% missing values by generalizing patterns rather than relying on explicit pairwise similarities. Unlike memory-based methods, it decomposes the sparse user-item matrix into lower-dimensional representations, enabling accurate predictions even when most entries are unobserved.

Answer

Random Forest

Answer

K-Nearest Neighbors

Answer

Hidden Markov Model

Question 14

A data scientist wants to automate the selection of optimal hyperparameters for a model. Which SageMaker feature should be used?

Accepted Answer

SageMaker Automatic Model Tuning. SageMaker Automatic Model Tuning (AMT) is the correct feature because it automates hyperparameter optimization by running multiple training jobs with different hyperparameter combinations, using algorithms like Bayesian optimization or random search to find the best set. This directly addresses the requirement to automate selection of optimal hyperparameters.

Answer

SageMaker Debugger

Answer

SageMaker Model Monitor

Answer

SageMaker Experiments

Question 15

Which TWO of the following are valid techniques to handle missing data in a dataset?

Accepted Answer

Mean imputation. Mean imputation (Option C) is a valid technique for handling missing data because it replaces missing values with the mean of the observed values for that feature, preserving the overall mean of the dataset. This approach is simple and effective for numerical data that is missing completely at random (MCAR), as it does not introduce bias in the mean estimate.

Answer

Normalizing the data

Answer

Adding a constant value of 0

Answer

Synthetic Minority Over-sampling (SMOTE)