MLS-C01 Modeling Practice Test 27 — 15 Questions

Question 1

Refer to the exhibit. A data scientist is trying to run a SageMaker training job using a script that reads data from the S3 bucket 'my-bucket' and writes the model artifact to the same bucket. The training job fails with an access denied error. What is the likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The IAM role does not have permission to write to the S3 bucket for the model artifact. The training job fails with an access denied error because the IAM role used by SageMaker lacks the s3:PutObject permission (or equivalent write access) for the S3 bucket 'my-bucket'. While the script reads data from the bucket, writing the model artifact requires explicit write permissions on the same bucket. Without this, SageMaker cannot upload the model artifact, causing the job to fail.

Answer

The IAM role does not have sagemaker:CreateModel permission

Answer

The IAM role does not have s3:ListBucket permission

Answer

The IAM role does not have ec2:DescribeInstances permission

Question 2

Refer to the exhibit. A data scientist checks the status of a SageMaker endpoint and sees the output above. The endpoint is receiving traffic, but the data scientist notices that the number of instances has not increased to the desired count. What is the most likely reason?

Accepted Answer

Automatic scaling is not configured for the endpoint. The endpoint is receiving traffic but not scaling out, which indicates that automatic scaling (Application Auto Scaling) has not been configured for the SageMaker endpoint. Without a scaling policy, the endpoint will only use the initial instance count, regardless of traffic load. The status shown does not indicate any update or quota issue, so the lack of scaling is the most likely cause.

Answer

The endpoint is performing a rolling update

Answer

The endpoint is currently being updated

Answer

The account has reached its instance limit

Question 3

A data scientist is building a binary classification model to predict loan default. The dataset is highly imbalanced (5% default, 95% non-default). Which TWO techniques should the data scientist use to address the class imbalance?

Accepted Answer

Oversample the minority class using SMOTE. Oversampling the minority class using SMOTE (Synthetic Minority Oversampling Technique) is correct because it generates synthetic samples for the minority class by interpolating between existing minority instances, rather than simply duplicating them. This helps balance the dataset without introducing exact copies, which can reduce overfitting and improve the model's ability to generalize to the minority class.

Answer

Undersample the majority class

Answer

Use RMSE as the evaluation metric

Answer

Use accuracy as the evaluation metric

Question 4

A data scientist is training a deep neural network on Amazon SageMaker. The training is taking a long time and the data scientist wants to speed it up. Which THREE actions can help reduce training time?

Accepted Answer

Use GPU instances instead of CPU instances. GPU instances (e.g., P3, P4d) are optimized for the massively parallel matrix operations required by deep neural networks, providing orders-of-magnitude faster computation than CPU instances for training tasks. By offloading tensor operations to GPU cores, the training time is significantly reduced, especially for large models and datasets.

Answer

Increase the batch size

Answer

Use a smaller instance type

Question 5

A data scientist is evaluating a linear regression model. Which TWO metrics are appropriate for evaluating the model's performance?

Accepted Answer

R-squared. R-squared is a standard metric for linear regression that measures the proportion of variance in the dependent variable explained by the independent variables. It ranges from 0 to 1, with higher values indicating better fit, making it directly appropriate for evaluating regression model performance.

Answer

Precision

Answer

Area Under the ROC Curve (AUC-ROC)

Answer

F1 score

Question 6

A company is building a fraud detection model that must achieve low false positive rates. The dataset is highly imbalanced (0.1% positive class). Which metric is most appropriate for model evaluation?

Accepted Answer

Area under the Precision-Recall curve. In highly imbalanced datasets (0.1% positive class), the Precision-Recall curve focuses on the performance of the positive class, which is the minority class of interest. Area under the Precision-Recall curve (AUPRC) is insensitive to the large number of true negatives, making it a robust metric for evaluating models where false positives must be minimized. Unlike ROC-AUC, which can be overly optimistic in severe imbalance, AUPRC directly reflects the trade-off between precision and recall for the rare positive class.

Answer

RMSE

Answer

Accuracy

Answer

R-squared

Question 7

A data scientist is training a binary classification model using SageMaker XGBoost and notices that training loss decreases but validation loss increases after a few epochs. Which action should the data scientist take to address this issue?

Accepted Answer

Set early stopping based on validation loss. The increasing validation loss while training loss decreases is a classic sign of overfitting. Setting early stopping based on validation loss halts training when the validation loss stops improving, preventing the model from memorizing noise in the training data. SageMaker XGBoost's `early_stopping_rounds` parameter monitors the evaluation metric on the validation set and stops training if no improvement is seen for a specified number of rounds.

Answer

Increase the number of rounds

Answer

Increase the learning rate

Answer

Increase the maximum tree depth

Question 8

A company wants to deploy a machine learning model that requires very low latency predictions (under 10ms). The model is a small ensemble of decision trees. Which SageMaker deployment option is most suitable?

Accepted Answer

SageMaker endpoint with a single instance. C is correct because a SageMaker endpoint with a single instance provides a persistent, real-time inference API that can achieve sub-10ms latency for a small ensemble of decision trees. The endpoint keeps the model loaded in memory and uses synchronous HTTP requests, minimizing cold start and network overhead, which is essential for low-latency predictions.

Answer

SageMaker Notebook instance

Answer

AWS Lambda function with the model packaged

Answer

SageMaker Batch Transform

Question 9

A data scientist is using SageMaker to train a deep learning model with a large dataset stored in S3. The training is taking a long time. Which action would most likely reduce training time without sacrificing accuracy?

Accepted Answer

Use SageMaker Pipe Input mode. SageMaker Pipe Input mode streams training data directly from S3 into the algorithm without first downloading it to the local EBS volume. This eliminates the I/O bottleneck caused by large dataset downloads, significantly reducing training time while preserving accuracy because the model sees the same data.

Answer

Increase the batch size

Answer

Use a smaller instance type

Answer

Reduce the number of epochs

Question 10

A data scientist is tuning a linear regression model and observes that the model has high bias and low variance. Which action is most likely to improve model performance?

Accepted Answer

Add more features. High bias and low variance indicate underfitting, meaning the model is too simple to capture the underlying patterns in the data. Adding more features increases model complexity, allowing it to learn more relevant relationships and reduce bias. This directly addresses the core issue of underfitting in linear regression.

Answer

Reduce the number of features

Answer

Increase regularization

Answer

Reduce the amount of training data

Question 11

A company is building a recommendation system for an e-commerce platform. The data includes user IDs and item IDs. Which SageMaker built-in algorithm is most appropriate?

Accepted Answer

Factorization Machines. Factorization Machines (FM) are specifically designed for recommendation tasks with sparse, high-dimensional categorical data like user IDs and item IDs. They model pairwise interactions between features (e.g., user-item interactions) using factorized parameters, making them highly effective for collaborative filtering and implicit feedback scenarios in e-commerce.

Answer

BlazingText

Answer

XGBoost

Answer

Image Classification

Question 12

A data scientist is using principal component analysis (PCA) for dimensionality reduction before training a classifier. The classifier's performance on the test set is poor. What is the most likely cause?

Accepted Answer

Too few principal components were retained, losing important information. C is correct because PCA is an unsupervised dimensionality reduction technique that projects data onto principal components capturing the maximum variance. If too few components are retained, the reduced representation may discard features that are critical for the classifier to distinguish between classes, leading to poor test performance due to underfitting.

Answer

The classifier is overfitting

Answer

The data was not scaled before applying PCA

Answer

Too many principal components were retained, including noise

Question 13

A company is using a custom Docker container in SageMaker for training. The training job fails with 'ResourceLimitExceeded' error. Which action should the data scientist take?

Accepted Answer

Request a limit increase for the instance type. The 'ResourceLimitExceeded' error in SageMaker indicates that the AWS account has reached a service quota for the specified instance type (e.g., ml.p3.2xlarge). This is a quota limit, not a performance or resource exhaustion issue within the training job itself. The correct action is to request a limit increase via the AWS Service Quotas console or AWS Support, which raises the maximum number of concurrent instances or total vCPUs allowed for that instance family.

Answer

Use a smaller instance type

Answer

Reduce the number of epochs

Answer

Use a pre-built SageMaker container instead

Question 14

A data scientist needs to evaluate a binary classification model's performance. Which metric is most appropriate when the cost of false positives is very high?

Accepted Answer

Precision. Precision is the most appropriate metric when the cost of false positives is very high because it measures the proportion of positive identifications that were actually correct. In binary classification, precision = TP / (TP + FP), so a high precision means very few false positives occur, directly minimizing the costly error type.

Answer

Accuracy

Answer

F1 score

Answer

Recall

Question 15

A data scientist is training a model using SageMaker and wants to automatically stop training when the model stops improving. Which TWO options can be used?

Accepted Answer

Built-in early stopping in XGBoost. Built-in early stopping in XGBoost (Option B) is correct because XGBoost natively supports an `early_stopping_rounds` parameter that halts training when the validation metric stops improving for a specified number of rounds. SageMaker Debugger (Option C) is correct because it can monitor training metrics in real time and trigger a stop action via a built-in or custom rule (e.g., `VanishingGradient` or `LossNotDecreasing`) when the model stops improving, integrating with SageMaker's `StopTraining` API.

Answer

AWS Step Functions

Answer

CloudWatch Alarms

Answer

SageMaker Model Monitor