MLS-C01 Modeling Practice Test 24 — 15 Questions

Question 1

A team is training a deep learning model on SageMaker using a custom Docker container. The training job fails with 'OutOfMemoryError'. The instance type is ml.p3.2xlarge with 61 GB memory. Which change should increase available memory?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Set the 'shm-size' parameter in the SageMaker training container to a larger value.. The 'OutOfMemoryError' in a SageMaker training container often stems from insufficient shared memory (/dev/shm) for data-loading workers, especially with PyTorch or TensorFlow dataloaders that use multiprocessing. Increasing the 'shm-size' parameter allocates more shared memory to the container, resolving the error without altering the model or instance type.

Answer

Reduce the batch size to use less memory.

Answer

Use SageMaker distributed data parallelism to distribute the model across multiple instances.

Answer

Mount an Amazon FSx for Lustre file system to offload data.

Question 2

A company wants to deploy a machine learning model that requires GPU acceleration for inference. The model is small and can fit on a single GPU. Which SageMaker endpoint configuration is MOST cost-effective?

Accepted Answer

Use a ml.p3.2xlarge instance with 1 GPU and enable automatic scaling.. The most cost-effective because it uses a single-GPU ml.p3.2xlarge instance, which matches the requirement that the model fits on one GPU, and enables automatic scaling to handle variable traffic without over-provisioning. This avoids paying for unused GPU capacity while still providing the necessary GPU acceleration for inference.

Answer

Use a ml.p3.16xlarge instance with 8 GPUs.

Answer

Use a SageMaker Serverless Inference endpoint.

Answer

Use a Multi-Model Endpoint on a ml.g4dn.xlarge instance.

Question 3

A data scientist is training a linear regression model and notices high bias in the training set. What action is most likely to reduce bias?

Accepted Answer

Add more relevant features to the model.. High bias indicates that the model is underfitting the training data, meaning it is too simple to capture the underlying patterns. Adding more relevant features increases the model's capacity to learn complex relationships, directly reducing bias. This is a standard approach in linear regression to address underfitting.

Answer

Apply L1 regularization.

Answer

Increase the learning rate.

Answer

Increase the amount of training data.

Question 4

A machine learning engineer is tuning a gradient boosting model using SageMaker Hyperparameter Tuning. The objective is to minimize MAE. The tuning job uses 20 training jobs. After 10 jobs, the best objective value is 5.2. Which action should the engineer take to potentially improve the result?

Accepted Answer

Increase the total number of training jobs to 50.. Increasing the total number of training jobs from 20 to 50 gives the Bayesian optimization algorithm more opportunities to explore the hyperparameter space and exploit promising regions. With only 10 jobs completed, the tuning job may not have converged to the global minimum of MAE, and additional jobs can refine the search, especially since Bayesian search builds a probabilistic model that improves with more observations.

Answer

Set early stopping to avoid overfitting.

Answer

Change the objective metric to RMSE.

Answer

Switch the tuning strategy from Bayesian to Random search.

Question 5

A company has a time series forecasting problem with daily sales data. The data shows both trend and seasonality. Which Amazon SageMaker built-in algorithm is most suitable?

Accepted Answer

DeepAR. DeepAR is a supervised learning algorithm for time series forecasting that explicitly models both trend and seasonality using recurrent neural networks (RNNs). It is designed to handle multiple related time series, incorporate additional features like holidays or promotions, and produce probabilistic forecasts, making it the most suitable choice for daily sales data with trend and seasonal patterns.

Answer

K-Means

Answer

Linear Learner

Answer

XGBoost

Question 6

A data scientist is using SageMaker to train a model. The training data is stored in an S3 bucket in a different AWS account. What is required to allow SageMaker to access the data?

Accepted Answer

Configure the SageMaker execution role with a policy that grants cross-account access to the S3 bucket.. SageMaker uses an IAM execution role to access resources. To allow cross-account access to an S3 bucket, the SageMaker execution role must have an IAM policy that grants s3:GetObject and s3:ListBucket permissions for the bucket, and the S3 bucket policy must also grant cross-account access to that role. This is the standard AWS mechanism for cross-account resource access.

Answer

Set up VPC peering between the two accounts.

Answer

Create a SageMaker notebook instance in the same account as the S3 bucket.

Answer

Launch the training job from a SageMaker notebook in the account containing the S3 bucket.

Question 7

A company is deploying a model for real-time inference with SageMaker. The endpoint receives spiky traffic, with occasional bursts of 10x normal load. Which scaling policy is MOST cost-effective while maintaining availability?

Accepted Answer

Use a combination of scheduled scaling for predictable peaks and simple scaling for additional bursts.. It combines scheduled scaling for predictable traffic patterns (e.g., known peak hours) with simple scaling to handle unexpected bursts, ensuring availability during 10x load spikes without over-provisioning. This hybrid approach is more cost-effective than always-on large instances, as it dynamically adjusts capacity only when needed, aligning with SageMaker's automatic scaling capabilities.

Answer

Provision a large instance type that can handle the peak load at all times.

Answer

Manually scale the endpoint based on historical traffic patterns.

Answer

Use a target tracking scaling policy based on average latency.

Question 8

A data scientist is training a neural network for image classification. The training loss decreases but validation loss increases after a few epochs. Which TWO actions should be taken to address this?

Accepted Answer

Implement early stopping based on validation loss.. Early stopping monitors validation loss and halts training when it stops improving, preventing overfitting. This directly addresses the symptom of decreasing training loss with increasing validation loss, which is a classic sign of overfitting.

Answer

Increase the learning rate.

Answer

Increase the number of epochs.

Answer

Add more convolutional layers.

Question 9

A company uses SageMaker to train a model. The training job fails with 'ResourceLimitExceeded' error. Which TWO actions should the company take to resolve this?

Accepted Answer

Use a different instance type that is not at its limit.. The 'ResourceLimitExceeded' error indicates that the requested instance type has reached its concurrent usage limit in the current AWS region. Switching to a different instance type that is not at its limit allows the training job to proceed without exceeding the service quota. Option E is correct because requesting a service limit increase for SageMaker training job resources directly raises the cap on the number of concurrent instances or total instance count, resolving the underlying quota issue.

Answer

Launch the training job in a different AWS region.

Answer

Use SageMaker Managed Spot Training to reduce cost.

Answer

Compress the training data to reduce storage requirements.

Question 10

A data scientist is building a recommender system using Amazon SageMaker. The dataset contains user-item interactions with implicit feedback (clicks). Which THREE evaluation metrics are appropriate for this use case?

Accepted Answer

Precision@k. For implicit feedback (e.g., clicks) in recommender systems, evaluation metrics should focus on ranking quality without requiring explicit negative labels. Precision@k measures the proportion of relevant items among top-k recommendations, Recall@k measures the proportion of relevant items retrieved in top-k, and Mean Average Precision (MAP) averages precision across multiple recall levels, all suitable for top-k recommendation tasks. RMSE is for explicit rating prediction and requires ground truth ratings, not applicable to implicit feedback. AUC-ROC is for binary classification with both positive and negative samples, but implicit feedback lacks explicit negatives, making it unsuitable.

Answer

Root Mean Squared Error (RMSE)

Answer

Area Under the ROC Curve (AUC-ROC)

Question 11

A training job fails with the error shown. The training script expects a file named 'train.csv' in the 'training' channel. What is the most likely cause?

Accepted Answer

The 'train.csv' file is located inside a subfolder within 's3://my-bucket/data/', and the script expects it directly in the channel path.. The error indicates that the training script cannot find 'train.csv' in the expected location. When SageMaker copies data from an S3 channel path (e.g., 's3://my-bucket/data/') to the training instance, it places the contents of that S3 prefix directly into the channel directory (e.g., '/opt/ml/input/data/training/'). If the CSV file is inside a subfolder (e.g., 's3://my-bucket/data/subfolder/train.csv'), the script will not find it at the top level of the channel path, causing a 'file not found' error.

Answer

The S3 bucket policy denies access to the 'train.csv' file.

Answer

The channel name in the input data configuration does not match the script's expected channel name.

Answer

The training script has a bug that prevents it from reading the file.

Question 12

A CloudFormation stack creation failed. The SageMaker endpoint resource shows CREATE_FAILED. What is the most likely issue?

Accepted Answer

The S3 bucket 'my-bucket' does not contain the object 'model.tar.gz'.. A CREATE_FAILED status on a SageMaker endpoint resource during CloudFormation stack creation most commonly indicates that the model artifact specified in the Model definition cannot be located. SageMaker requires the S3 bucket and object path (e.g., 's3://my-bucket/model.tar.gz') to exist and be accessible at the time of model creation. If the object is missing, the model resource fails, cascading to the endpoint creation failure.

Answer

The IAM role used by CloudFormation lacks permissions to create endpoints.

Answer

The SageMaker endpoint configuration is invalid.

Answer

The instance type specified for the endpoint is not available in the region.

Question 13

A company is building a binary classifier to detect fraudulent transactions. The dataset is highly imbalanced with only 0.1% positive cases. The data scientist uses logistic regression and obtains 99.9% accuracy on the test set. Which metric should the data scientist use to evaluate the model's performance?

Accepted Answer

Precision-recall curve. With only 0.1% positive cases, accuracy is misleading because a model that always predicts 'not fraudulent' achieves 99.9% accuracy. The precision-recall curve focuses on the positive class and is robust to extreme class imbalance, showing the trade-off between precision and recall across thresholds. This makes it the best choice for evaluating a binary classifier on highly imbalanced fraud detection data.

Answer

ROC AUC

Answer

Precision

Answer

F1 score

Question 14

A data scientist trains a gradient boosting model on a large dataset using SageMaker. The training completes successfully, but when deploying the model to a real-time endpoint, inference latency is too high. Which change is MOST likely to reduce latency without significant accuracy loss?

Accepted Answer

Prune the trees by removing nodes with low importance. Pruning trees by removing nodes with low importance reduces the model's complexity, which directly decreases inference latency because fewer decision paths need to be evaluated. In gradient boosting, this can be done with minimal accuracy loss if the removed nodes correspond to splits that contribute little to the overall prediction, as measured by feature importance or gain.

Answer

Use a larger instance type for the endpoint

Answer

Increase the number of trees in the ensemble

Answer

Use SageMaker Batch Transform instead of real-time

Question 15

A data scientist is training a linear regression model and notices that the model performs well on training data but poorly on validation data. Which technique should be applied to reduce overfitting?

Accepted Answer

Apply L2 regularization (Ridge). L2 regularization (Ridge) adds a penalty term proportional to the square of the magnitude of the coefficients to the loss function. This shrinks the weights toward zero, reducing the model's sensitivity to individual features and preventing it from fitting noise in the training data, which directly addresses overfitting.

Answer

Increase the number of epochs

Answer

Add more features

Answer

Remove training examples