MLS-C01 Practice Test 16 — 15 Questions

Question 1

A data scientist is building a model to predict insurance claim amounts. The target variable is right-skewed with many small claims and a few very large claims. The scientist wants to minimize the impact of outliers. Which loss function or transformation is MOST appropriate?

Accepted Answer

Apply a log transformation to the target variable. Using a log transformation or modeling with a log-link function can reduce skewness and impact of outliers. Option A (Mean squared error) is sensitive to outliers. Option B (Quantile loss) is robust but less common for mean prediction. Option D (Poisson loss) is for count data. Option C (Log transformation of target) is standard for skewed continuous targets.

Answer

Use mean squared error loss without any transformation

Answer

Use quantile loss to predict the median

Answer

Use Poisson loss assuming the target follows a Poisson distribution

Question 2

A machine learning team is deploying a model using Amazon SageMaker. The model receives requests with sparse high-dimensional features. The team wants to minimize inference latency. Which SageMaker endpoint configuration is MOST suitable?

Accepted Answer

Use a multi-model endpoint on a GPU instance. Option D is correct because multi-model endpoints on GPU instances allow multiple models to be loaded into memory on a single GPU-backed instance, reducing cold-start latency for sparse high-dimensional features by keeping models warm and leveraging GPU parallelism for inference. This minimizes inference latency compared to other configurations by avoiding the overhead of separate endpoint invocations and optimizing resource utilization for high-dimensional data.

Answer

Use a multi-variant endpoint with two variants

Answer

Use a serverless endpoint with provisioned concurrency

Answer

Use a single model endpoint with a large instance type

Question 3

Which TWO metrics are appropriate for evaluating a binary classification model trained on imbalanced data? (Select TWO.)

Accepted Answer

F1 score. The F1 score is appropriate for imbalanced binary classification because it balances precision and recall, making it robust when the positive class is rare. Unlike accuracy, it does not get inflated by a majority negative class, and it directly penalizes models that predict the majority class for all instances.

Answer

Log loss

Answer

Accuracy

Answer

ROC-AUC

Question 4

A data scientist ran a hyperparameter tuning job for an XGBoost model. The tuning job completed, but the best validation RMSE is 2.34. The data scientist believes the model can perform better. Based on the exhibit, which change to the tuning strategy is most likely to improve the model's performance?

Accepted Answer

Increase the static num_round hyperparameter to 500. Option D is correct because increasing the static `num_round` hyperparameter to 500 allows the model to train for more boosting rounds, which can reduce underfitting and lower the RMSE further. The current best validation RMSE of 2.34 suggests the model may not have converged, and additional rounds can help the XGBoost model learn more complex patterns, provided overfitting is monitored with early stopping.

Answer

Use random search instead of Bayesian optimization

Answer

Change the objective to binary:logistic

Answer

Increase the maximum value of eta to 1.0

Question 5

A data scientist is training a decision tree classifier and notices that the model performs well on training data but poorly on test data. Which technique should the data scientist use to address this issue?

Accepted Answer

Prune the tree. Pruning the tree reduces overfitting by removing branches that have little statistical significance or that capture noise in the training data. This technique improves generalization to unseen test data, which directly addresses the symptom of high training accuracy and low test accuracy.

Answer

Use a different split criterion

Answer

Apply L1 regularization

Answer

Increase tree depth

Question 6

A company uses Amazon SageMaker to train a model for fraud detection. The dataset has 1 million transactions, with 0.1% fraud. The data scientist trains a random forest model and achieves 99.9% accuracy but 0% recall on the fraud class. Which technique is most likely to improve recall without significantly reducing precision?

Accepted Answer

Apply SMOTE to generate synthetic fraud samples. Option C is correct because SMOTE (Synthetic Minority Oversampling Technique) generates synthetic fraud samples by interpolating between existing minority class instances, which directly addresses the extreme class imbalance (0.1% fraud). This increases the representation of the fraud class in the training data, allowing the random forest model to learn decision boundaries that capture fraud patterns, thereby improving recall without introducing the noise or information loss associated with other methods.

Answer

Tune the classification threshold

Answer

Use cost-sensitive learning with a high cost for fraud misclassification

Answer

Undersample the majority class

Question 7

Refer to the exhibit. A data scientist is evaluating a binary classification model for spam detection. The exhibit shows a single prediction instance. What is the model's prediction for this instance?

Accepted Answer

Spam. The model's prediction is 'Spam' because the prediction instance shows a probability of 0.95 for the 'Spam' class, which exceeds the typical decision threshold of 0.5 used in binary classification. Since the probability for 'Spam' is higher than for 'Ham' (0.05), the model assigns the instance to the class with the highest probability, which is Spam.

Answer

Ham

Answer

0.95

Answer

The model is unsure because probability is not 1.0

Question 8

Refer to the exhibit. A data scientist runs the above CLI command to create a SageMaker training job. The job fails with an error 'Unable to read data from s3://bucket/train/'. What is the MOST likely cause?

Accepted Answer

The IAM role does not have permissions to read from the S3 bucket. The error 'Unable to read data from s3://bucket/train/' indicates that the SageMaker training job cannot access the S3 input data. The most common cause is that the IAM role specified in the command does not have the necessary s3:GetObject permission on the S3 bucket or objects. SageMaker uses the IAM role to assume permissions for reading training data, and without proper S3 read access, the job fails at the data loading stage.

Answer

The training image is not accessible

Answer

The instance type does not support the required memory

Answer

The training job is in a different region than the S3 bucket

Question 9

A data scientist is training a deep learning model on Amazon SageMaker using a PyTorch estimator. The training job runs on a single ml.p3.2xlarge instance but is taking too long. The scientist wants to reduce training time by using distributed data parallelism across multiple GPUs. Which change to the training script and SageMaker estimator is required?

Accepted Answer

Modify the script to use torch.nn.parallel.DistributedDataParallel and set instance_count to 2 in the estimator.. Option C is correct because to achieve distributed data parallelism across multiple GPUs on multiple instances with PyTorch, you must modify the training script to use `torch.nn.parallel.DistributedDataParallel` (DDP), which handles gradient synchronization across nodes. Additionally, you must set `instance_count` to 2 (or more) in the SageMaker PyTorch estimator to launch multiple instances, each with its own GPU, enabling true multi-node distributed training.

Answer

Add the SageMaker distributed data parallelism configuration in the estimator and modify the script to use the SageMaker distributed library.

Answer

Change the framework to TensorFlow and use tf.distribute.MirroredStrategy with instance_count=2.

Answer

Modify the script to use torch.nn.DataParallel and keep instance_count as 1.

Question 10

A data scientist is using Amazon SageMaker to train a model with a custom Docker container. The training job fails with an error: 'Container exited with code 137'. What is the most likely cause?

Accepted Answer

The training instance ran out of memory.. Exit code 137 (128+9) indicates the container was killed by the SIGKILL signal, which typically occurs when the Linux Out-Of-Memory (OOM) killer terminates a process that has exceeded its memory allocation. In Amazon SageMaker, training instances have finite memory, and if the training algorithm or data loading exceeds that limit, the OOM killer forcibly stops the container, resulting in exit code 137.

Answer

The training data was corrupted.

Answer

The training job exceeded the maximum runtime.

Answer

The Docker entrypoint script was not found.

Question 11

A data scientist is assigned an IAM policy as shown. The data scientist attempts to create a SageMaker endpoint to deploy a model, but the request fails. What is the most likely reason?

Accepted Answer

The data scientist does not have permission to create an endpoint.. The IAM policy shown does not include the `sagemaker:CreateEndpoint` action, which is required to create a SageMaker endpoint. Even if the data scientist has permissions for other SageMaker actions like `CreateModel` or `CreateEndpointConfig`, the explicit absence of `CreateEndpoint` in the policy will cause the request to fail with an access denied error. AWS IAM policies must explicitly grant each action needed for the operation.

Answer

The data scientist does not have permission to upload the model to S3.

Answer

The data scientist does not have permission to create a training job.

Answer

The data scientist does not have permission to pass roles.

Question 12

A data scientist is tuning a gradient boosting model using Amazon SageMaker Automatic Model Tuning (AMT). Which THREE hyperparameters should the scientist consider tuning to reduce overfitting? (Select THREE.)

Accepted Answer

Learning rate (eta). Learning rate (eta) controls the contribution of each tree to the ensemble. A lower learning rate forces the model to learn more slowly, requiring more trees but reducing the risk of overfitting by preventing any single tree from having too much influence on the final prediction.

Answer

Subsample ratio

Answer

Maximum depth (max_depth)

Question 13

A company is building a real-time fraud detection system using Amazon SageMaker. The model must have low latency (under 10ms) and high throughput (thousands of predictions per second). The team has trained a gradient boosting model using XGBoost. Which SageMaker inference option is MOST suitable?

Accepted Answer

Deploy the model on a SageMaker real-time endpoint with a multi-model endpoint.. A multi-model endpoint (MME) on SageMaker is the most suitable option because it allows you to host multiple XGBoost models on a single endpoint, sharing the underlying instance to maximize throughput and minimize latency. MMEs keep models loaded in memory and route requests to the correct model with sub-10ms overhead, meeting the low-latency and high-throughput requirements for real-time fraud detection.

Answer

Use SageMaker asynchronous inference.

Answer

Deploy the model on a SageMaker serverless endpoint.

Answer

Use a batch transform job.

Question 14

A company is building a recommendation system using matrix factorization. The training data contains user-item interactions. The model performs well on the training set but poorly on the test set. Which regularization technique should be applied to improve generalization?

Accepted Answer

Add L2 regularization to the user and item latent factors. L2 regularization (weight decay) penalizes large values in the user and item latent factor matrices, which helps prevent overfitting by encouraging the model to learn smoother, more generalizable representations. This is the standard regularization technique used in matrix factorization for collaborative filtering, as it directly controls the magnitude of the latent vectors without inducing sparsity.

Answer

Add L1 regularization to the user and item latent factors

Answer

Apply dropout to the latent factors during training

Answer

Use batch normalization on the factors

Question 15

A data scientist is training a linear regression model to predict house prices. The dataset contains 10 features. After training, the data scientist notices that the model has high bias (underfitting). Which action should the data scientist take to reduce bias?

Accepted Answer

Add more features, such as polynomial features. High bias (underfitting) means the model is too simple to capture the underlying patterns in the data. Adding more features, such as polynomial features, increases model complexity, allowing the linear regression model to fit non-linear relationships and reduce bias. This directly addresses the underfitting issue by giving the model more expressive power.

Answer

Reduce the amount of training data

Answer

Increase the regularization strength

Answer

Use a simpler model, such as ridge regression