MLS-C01 Practice Test 2 — 25 Questions

Question 1

A data scientist is training a binary classification model using Amazon SageMaker. The dataset is highly imbalanced (99% negative class, 1% positive class). The model currently achieves 99% accuracy but fails to detect most positive cases. Which metric should the data scientist primarily use to evaluate model performance?

Accepted Answer

F1 score. In highly imbalanced datasets (99% negative, 1% positive), accuracy is misleading because a model can achieve 99% accuracy by simply predicting the majority class for all instances, failing to detect any positive cases. The F1 score (option B) is the harmonic mean of precision and recall, providing a balanced measure that penalizes models that trade off recall for precision or vice versa. This makes it the primary metric for evaluating binary classification performance on imbalanced data, as it directly reflects the model's ability to correctly identify positive cases while minimizing false positives.

Answer

ROC AUC

Answer

Recall

Answer

Accuracy

Question 2

A team is building a product recommendation system using matrix factorization in Amazon SageMaker. They notice that the model's training loss decreases steadily but validation loss starts increasing after 5 epochs. What is the most likely cause?

Accepted Answer

Overfitting. In matrix factorization for recommendation systems, a decreasing training loss with an increasing validation loss after several epochs is a classic sign of overfitting. The model is memorizing the training data (including noise) rather than learning generalizable patterns, which degrades its performance on unseen validation data.

Answer

Underfitting

Answer

Not enough training data

Answer

Learning rate too high

Question 3

A company is using Amazon SageMaker to train a deep learning model on a large dataset. The training job is taking too long. The team wants to reduce training time without changing the model architecture. Which action should they take?

Accepted Answer

Use SageMaker's distributed training with multiple instances. SageMaker's distributed training with multiple instances splits the dataset and model computations across several machines, enabling parallel processing that significantly reduces wall-clock training time. This approach leverages data parallelism or model parallelism without altering the model architecture, directly addressing the need for faster training.

Answer

Increase the learning rate by a factor of 10

Answer

Reduce the number of epochs

Answer

Reduce the batch size

Question 4

A data scientist is deploying a regression model in Amazon SageMaker that predicts housing prices. The model shows high bias (underfitting). Which action is most likely to reduce bias?

Accepted Answer

Add more features or increase model complexity. High bias (underfitting) means the model is too simple to capture the underlying patterns in the data. Adding more features or increasing model complexity (e.g., using polynomial features, deeper trees, or a more flexible algorithm) directly addresses underfitting by giving the model greater capacity to learn from the data. In Amazon SageMaker, this could involve using a more complex built-in algorithm like XGBoost with deeper trees or adding feature engineering transformations in a processing job.

Answer

Reduce the amount of training data

Answer

Increase regularization strength

Answer

Use a simpler model

Question 5

A machine learning engineer is training a neural network on Amazon SageMaker using a custom Docker container. The training job fails with an error: 'CUDA out of memory.' The training instance is an ml.p3.2xlarge with 16 GB GPU memory. The model and data fit into memory when using batch size 32, but the engineer wants to maximize GPU utilization. Which approach should the engineer use to fix the out-of-memory error while maintaining efficient training?

Accepted Answer

Implement gradient accumulation with a larger effective batch size. Gradient accumulation allows the engineer to simulate a larger effective batch size by accumulating gradients over multiple forward/backward passes before performing an optimizer step. This keeps the per-step memory footprint low (avoiding CUDA out-of-memory) while maintaining training dynamics similar to a larger batch, thus maximizing GPU utilization without crashing.

Answer

Enable mixed precision training

Answer

Reduce batch size to 1

Answer

Use a CPU-only instance

Question 6

A data scientist is training a binary classifier on an imbalanced dataset where the positive class represents 1% of the data. The model currently achieves 99% accuracy but a recall of only 10% on the positive class. Which metric combination should the data scientist prioritize to evaluate model improvements?

Accepted Answer

F1 score and AUC-ROC. With a highly imbalanced dataset (1% positive class), 99% accuracy is misleading because the model can achieve it by simply predicting the majority class. The low recall (10%) indicates the model fails to identify most positive instances. The F1 score balances precision and recall, providing a single metric for minority class performance, while AUC-ROC evaluates the model's ability to distinguish between classes across all thresholds, making it robust to class imbalance.

Answer

Precision and recall at 90% precision

Answer

Accuracy and RMSE

Answer

Precision and RMSE

Question 7

Refer to the exhibit. An ML engineer creates a CloudFormation stack with this template. The stack creation succeeds, but when the engineer tries to invoke the endpoint, it returns a ModelError. The CloudWatch logs show that the container exited with error. What is the MOST likely cause?

Accepted Answer

The execution role does not have permissions to pull the Docker image from ECR.. Option A is correct because the CloudFormation template likely does not grant the SageMaker execution role the necessary `ecr:GetDownloadUrlForLayer` and `ecr:BatchGetImage` permissions to pull the container image from Amazon ECR. Without these permissions, the SageMaker service cannot download the Docker image to the ML compute instances, causing the container to fail with a ModelError and exit error in CloudWatch logs.

Answer

The initial instance count is set to 2, which is insufficient for the model size.

Answer

The endpoint is not deployed in a VPC and cannot access the S3 bucket.

Answer

The EndpointConfig references the model but the model is not yet created.

Question 8

A team uses AWS Glue ETL jobs to preprocess data for SageMaker training. The job runs successfully but the output data is empty. What is the most likely cause?

Accepted Answer

The filter transformation condition is too restrictive, removing all rows. Option C is correct because a filter transformation in AWS Glue ETL jobs can remove all rows if the condition is too restrictive, resulting in an empty output dataset. This is a common logic error where the filter predicate (e.g., `df.filter("value > 100")`) matches no records, causing the DynamicFrame to be empty after transformation. The job succeeds because no runtime error occurs, but the output is empty.

Answer

There is a data type mismatch between source and target

Answer

The source data is partitioned and only a subset of partitions is read

Answer

The Glue job runs out of memory and fails silently

Question 9

A data scientist runs a SageMaker training job that fails with the above error. The S3 bucket and object exist, and the IAM role has s3:GetObject permission. What is the MOST likely cause?

Accepted Answer

The S3 object was uploaded with incorrect checksum or is corrupted. The error indicates that SageMaker cannot read the S3 object, even though the bucket and object exist and the IAM role has s3:GetObject permission. The most likely cause is that the object was uploaded with an incorrect checksum or is corrupted, which prevents SageMaker from verifying the integrity of the data during the training job initialization. SageMaker uses ETag (MD5 checksum) validation when reading objects, and a mismatch triggers a failure.

Answer

The training instance does not have internet access

Answer

The S3 bucket has versioning enabled and the object version is not specified

Answer

The IAM role lacks kms:Decrypt permission for an encrypted S3 object

Question 10

A data scientist is trying to run a SageMaker training job that writes output to an S3 bucket 'my-bucket'. The IAM policy is shown. The training job fails with an AccessDenied error when trying to write to S3. What is the reason?

Accepted Answer

The S3 bucket is encrypted with AWS KMS and the policy does not include kms:GenerateDataKey. The correct answer is A because the training job fails with an AccessDenied error when writing to an S3 bucket that is encrypted with AWS KMS. The IAM policy shown must include the `kms:GenerateDataKey` permission to allow the SageMaker training job to generate a data key for encrypting the output objects. Without this KMS permission, the S3 PutObject operation is denied even if the policy allows `s3:PutObject`, as KMS encryption requires explicit authorization to use the customer master key (CMK).

Answer

The policy does not allow s3:ListBucket

Answer

The policy does not allow s3:PutObject

Answer

The policy does not allow s3:PutObjectAcl

Question 11

A company uses SageMaker to train a model. The training job is taking too long and the data scientist wants to speed it up. Which THREE strategies should the data scientist consider? (Select THREE.)

Accepted Answer

Use a GPU instance type like ml.p3.2xlarge. Option B is correct because GPU instances like ml.p3.2xlarge are optimized for parallel computation, significantly accelerating the training of deep learning models by handling matrix operations more efficiently than CPUs. SageMaker supports a range of GPU instances (e.g., ml.p3, ml.p4, ml.g5) that can reduce training time for compute-intensive workloads.

Answer

Reduce the number of training epochs

Answer

Increase the batch size in the training script

Question 12

Which TWO metrics are MOST appropriate for evaluating a regression model that predicts house prices, where the business is most sensitive to large errors?

Accepted Answer

Root Mean Squared Error (RMSE). RMSE is most appropriate because it squares the errors before averaging, which heavily penalizes large errors. Since the business is most sensitive to large errors in house price predictions, RMSE directly aligns with this requirement by amplifying the impact of outliers, making it a suitable metric for evaluating model performance in this context.

Answer

Accuracy

Answer

Mean Absolute Error (MAE)

Answer

R-squared

Question 13

A data scientist is training a neural network for image classification. The training loss is not decreasing significantly, and the validation loss is high. Which TWO actions should the scientist take to address potential vanishing gradients?

Accepted Answer

Use ReLU activation functions in hidden layers. ReLU activation functions help mitigate vanishing gradients because they output a constant gradient of 1 for positive inputs, preventing the gradient from shrinking as it propagates backward through many layers. This avoids the exponential decay of gradients that occurs with saturating activations like sigmoid or tanh, enabling effective training of deep networks.

Answer

Increase the learning rate

Answer

Switch activation functions from ReLU to sigmoid

Answer

Remove dropout layers

Question 14

A company is deploying a real-time fraud detection system using a gradient boosting model on AWS SageMaker. The model uses 200 features and is trained on 50 GB of data. The inference latency requirement is under 10 ms per request. During load testing, the endpoint shows average latency of 15 ms. Which change is MOST likely to reduce latency below 10 ms?

Accepted Answer

Reduce the number of features to the top 50 based on feature importance. Reducing the number of features from 200 to the top 50 directly decreases the amount of data each inference request must process, which lowers both feature engineering overhead and model evaluation time. For gradient boosting models on SageMaker, fewer features mean fewer decision tree splits to traverse per prediction, which can significantly reduce latency without requiring hardware changes. This is the most direct and cost-effective way to meet the 10 ms requirement.

Answer

Switch to a GPU-based instance type

Answer

Increase the number of trees in the model

Answer

Use a larger batch size for inference

Question 15

A data scientist is training a binary classifier to predict customer churn. The dataset has 10,000 samples, with 500 churners (positive class). The scientist trains a logistic regression model and obtains an F1-score of 0.6. To improve the F1-score, which approach is MOST likely to be effective?

Accepted Answer

Apply SMOTE to oversample the minority class. The dataset is highly imbalanced (500 churners out of 10,000 samples, a 5% positive rate). Logistic regression trained on such imbalance tends to bias toward the majority class, resulting in low recall for the minority class and a poor F1-score. SMOTE (Synthetic Minority Oversampling Technique) generates synthetic samples for the minority class by interpolating between existing minority instances, which balances the class distribution and allows the model to learn a better decision boundary, directly improving recall and F1-score.

Answer

Increase the regularization strength (C)

Answer

Apply PCA to reduce feature dimensionality

Answer

Use the original dataset without any modification

Question 16

Drag and drop the steps to train a model using Amazon SageMaker built-in algorithm in the correct order.

Accepted Answer

Prepare training data and upload to S3 → Choose built-in algorithm → Create training job specifying algorithm, input/output S3 paths, and compute resources → Execute training job. Training involves data preparation, job creation, algorithm selection, input/output paths, and execution.

Answer

Choose built-in algorithm → Prepare training data and upload to S3 → Create training job specifying algorithm, input/output S3 paths, and compute resources → Execute training job

Answer

Prepare training data and upload to S3 → Create training job specifying input/output S3 paths and compute resources → Choose built-in algorithm → Execute training job

Answer

Create training job specifying algorithm, input/output S3 paths, and compute resources → Prepare training data and upload to S3 → Execute training job → Choose built-in algorithm

Question 17

A company is training a deep learning model on SageMaker using multiple GPUs. The training is slow due to inefficient data loading. Which TWO actions can improve I/O performance?

Accepted Answer

Use Pipe mode input for training data.. Pipe mode streams training data directly from Amazon S3 to the training algorithm, bypassing the need to download data to disk before training begins. This eliminates disk I/O bottlenecks and reduces data loading latency, which is critical for GPU-intensive training where GPUs may otherwise idle waiting for data.

Answer

Use instance store volumes for data.

Answer

Increase the instance count to a single large instance.

Answer

Use Amazon EBS volumes attached to training instances.

Question 18

Drag and drop the steps to deploy a model as a SageMaker endpoint for real-time inference in the correct order.

Accepted Answer

1. Create model, 2. Create endpoint configuration, 3. Create endpoint, 4. Test endpoint. Deployment requires model creation, endpoint configuration, endpoint creation, and testing.

Answer

1. Create endpoint configuration, 2. Create model, 3. Create endpoint, 4. Test endpoint

Answer

1. Create model, 2. Create endpoint, 3. Create endpoint configuration, 4. Test endpoint

Answer

1. Test endpoint, 2. Create model, 3. Create endpoint configuration, 4. Create endpoint

Question 19

Drag and drop the steps to create a data processing job using Amazon SageMaker Processing in the correct order.

Accepted Answer

Step 1: Create a processing script, Step 2: Upload script and data to Amazon S3, Step 3: Configure the ProcessingJob, Step 4: Execute the ProcessingJob, Step 5: Verify the job output. Processing requires script creation, data upload, job configuration, execution, and verification.

Answer

Step 1: Upload script and data to Amazon S3, Step 2: Create a processing script, Step 3: Configure the ProcessingJob, Step 4: Execute the ProcessingJob, Step 5: Verify the job output

Answer

Step 1: Configure the ProcessingJob, Step 2: Create a processing script, Step 3: Upload script and data to Amazon S3, Step 4: Execute the ProcessingJob, Step 5: Verify the job output

Answer

Step 1: Create a processing script, Step 2: Configure the ProcessingJob, Step 3: Upload script and data to Amazon S3, Step 4: Execute the ProcessingJob, Step 5: Verify the job output

Question 20

A company is using Amazon SageMaker to train a large language model. The training job is taking too long. The data scientist wants to reduce training time without sacrificing model accuracy. Which THREE strategies are MOST appropriate?

Accepted Answer

Use mixed precision training (float16). Mixed precision training (float16) reduces memory usage and accelerates computation by using half-precision floating-point numbers for most operations, while maintaining a single-precision copy of critical parameters to preserve accuracy. This directly reduces training time on compatible GPUs (e.g., NVIDIA V100, A100) without sacrificing model quality, as the loss scaling technique prevents underflow in gradients.

Answer

Switch from GPU instance to CPU instance

Answer

Increase the maximum sequence length