MLS-C01 Modeling Practice Test 1 — 15 Questions

Question 1

A company is building a recommendation system for an e-commerce platform. They have user-item interaction data and want to use matrix factorization. However, the dataset is sparse (99% missing interactions). Which approach should the data scientist take to train the model effectively?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use alternating least squares (ALS) with implicit feedback and assign lower confidence to unobserved interactions. Alternating Least Squares (ALS) with implicit feedback is specifically designed to handle sparse implicit feedback datasets by assigning lower confidence to unobserved interactions (e.g., confidence = 1 + alpha * r_ui, where r_ui is 0 for unobserved). This avoids the pitfalls of treating missing values as zeros (which distorts the factorization) and scales well to 99% sparsity by leveraging weighted regularization.

Answer

Impute missing values with zeros and use singular value decomposition (SVD)

Answer

Remove all users and items with fewer than 10 interactions to reduce sparsity

Answer

Use item-based collaborative filtering with cosine similarity

Question 2

A company is using Amazon SageMaker to train a deep learning model. The training job is taking a long time, and the data scientist wants to reduce training time without sacrificing accuracy. Which technique should they use?

Accepted Answer

Use a larger instance type. Using a larger instance type provides more computational resources (e.g., more GPUs, faster CPUs, more memory), which directly reduces training time without altering the model architecture or training process, thus preserving accuracy. Option D (managed spot training) is designed to reduce cost, not necessarily training time; while checkpointing can help with fault tolerance, spot instances may be interrupted, potentially increasing total time. Option B (smaller instance) would increase training time. Option C (reducing epochs) risks underfitting and lower accuracy.

Answer

Use a smaller instance type

Answer

Reduce the number of epochs

Answer

Use managed spot training with checkpointing

Question 3

An ML team trains a deep learning model using Amazon SageMaker with a custom Docker container. Training completes successfully, but the model's accuracy on the test set is significantly lower than expected. The team suspects overfitting. Which two actions should they take to mitigate overfitting? (Choose TWO.)

Accepted Answer

Use data augmentation. Dropout and data augmentation are effective regularization techniques to reduce overfitting. Option A (increasing epochs) would worsen overfitting. Option B (reducing batch size) can introduce noise but is not a primary regularization method. Option D (adding more layers) increases model capacity, likely worsening overfitting.

Answer

Reduce the batch size

Answer

Increase the number of layers

Answer

Increase the number of training epochs

Question 4

A data scientist is building a time series forecasting model for daily sales data. The data exhibits strong seasonality with a weekly pattern and a yearly trend. The scientist wants to use Amazon SageMaker's built-in algorithm. Which algorithm is most appropriate?

Accepted Answer

Amazon SageMaker DeepAR. DeepAR is a built-in SageMaker algorithm specifically designed for time series forecasting, capable of capturing seasonality and trends. It outperforms general-purpose algorithms on such data. Option B (Linear Learner) is wrong because it does not handle seasonality natively. Option C (K-Means) is wrong as it is a clustering algorithm. Option D (XGBoost) is wrong because it is not specialized for time series and does not inherently model temporal dependencies.

Answer

Linear Learner

Answer

K-Means

Answer

XGBoost

Question 5

A data scientist ran an XGBoost training job in SageMaker and it failed with the error shown in the exhibit. Which hyperparameter change is most likely to resolve the numeric overflow?

Accepted Answer

Reduce max_depth to a lower value. Reducing max_depth prevents trees from growing too deep, which limits leaf weight magnitudes and avoids numeric overflow. Option B is incorrect because increasing subsample does not directly affect tree depth or leaf weights; it affects the fraction of data used per tree. Option C is incorrect because increasing eta (learning rate) increases the step size for updates, which can lead to larger leaf weights and potential overflow. Option D is incorrect because increasing num_round adds more boosting iterations but does not control the size of leaf weights; it may even exacerbate overflow if leaf weights are already large.

Answer

Increase subsample

Answer

Increase eta (learning rate)

Answer

Increase num_round

Question 6

A data scientist is training a gradient boosting model on a large dataset (100 GB) stored in Amazon S3. The training job uses a SageMaker built-in XGBoost algorithm with a single ml.p3.2xlarge instance. The job fails with a memory error. Which solution should the data scientist adopt to resolve the memory issue?

Accepted Answer

Increase the number of training instances to use distributed training with XGBoost.. The memory error occurs because a single ml.p3.2xlarge instance cannot hold the entire 100 GB dataset in memory during training. Option C resolves this by distributing the data across multiple instances, reducing the per-instance memory load. Option B (Pipe mode) streams data from S3 without downloading it fully, but XGBoost still requires the entire dataset to be loaded into memory if not using distributed training, so it may not fully resolve the memory issue. Option A (PCA) reduces features but may not sufficiently reduce memory usage and risks information loss. Option D (BlazingText) is a different algorithm not suited for gradient boosting. Option E (Linear Learner) also requires significant memory for large datasets and is not a direct fix for memory errors in XGBoost.

Answer

Reduce the number of features by using PCA before training.

Answer

Use SageMaker Pipe mode to stream data directly from S3 instead of downloading it.

Answer

Use SageMaker BlazingText algorithm with negative sampling.

Answer

Switch to SageMaker Linear Learner algorithm, which requires less memory.

Question 7

A data scientist wants to evaluate the performance of a multiclass classification model. The model outputs probabilities for 10 classes. Which metric is most appropriate for evaluating the model's ranking performance across all classes?

Accepted Answer

Log loss (cross-entropy). Log loss measures the performance of a classification model where the prediction is a probability value, and it penalizes false classifications. Option A (F1 score macro-averaged) is not appropriate because it focuses on threshold-based metrics rather than probability ranking. Option B (Accuracy) ignores probability calibration. Option C (Mean Absolute Error) is for regression, not classification. Option E (ROC AUC one-vs-rest macro-averaged) can be used for multiclass ranking, but log loss is more appropriate when the model outputs probabilities and you want to evaluate the overall probability quality across all classes.

Answer

F1 score (macro-averaged)

Answer

Accuracy

Answer

Mean Absolute Error

Answer

ROC AUC (one-vs-rest macro-averaged)

Question 8

Which TWO metrics are appropriate for evaluating a binary classification model when the cost of false negatives is high? (Choose 2)

Accepted Answer

Recall. When the cost of false negatives is high, the primary goal is to minimize false negatives by capturing as many actual positives as possible. Recall (Option A) directly measures this by calculating TP/(TP+FN). Specificity (Option B) measures TN/(TN+FP) and focuses on false positives, not false negatives; it is irrelevant when FN cost is high. Accuracy (C) can be misleading with imbalanced data. F1 score (D) balances precision and recall but does not specifically target FN. Precision (E) focuses on false positives, not false negatives. Therefore, only Recall is appropriate.

Answer

Specificity

Answer

Accuracy

Answer

F1 score

Answer

Precision

Question 9

A team is deploying a real-time inference endpoint using Amazon SageMaker. The model is a large ensemble of 10 deep learning models, each 500 MB. The inference latency requirement is under 200 ms. Currently, the endpoint using a single ml.p3.2xlarge instance takes 1.5 seconds per request. Which approach is MOST likely to meet the latency requirement?

Accepted Answer

Use model parallelism to split the ensemble across multiple GPUs on a single instance.. Model parallelism distributes the ensemble across multiple GPUs on a single ml.p3.2xlarge instance (which has 1 GPU) or across multiple GPUs on a larger instance, reducing per-device load and inference time. Option A is wrong because increasing batch size increases per-request latency, not reduces it. Option B is wrong because compute-optimized instances like c5.18xlarge have no GPU, which is needed for deep learning models. Option C is wrong because SageMaker Neo compiles models for edge devices or specific hardware, not for model parallelism or latency reduction on a single instance.

Answer

Increase the batch size to process more requests per invocation.

Answer

Switch to a compute-optimized instance like c5.18xlarge.

Answer

Use SageMaker Neo to compile the model for the target instance.

Question 10

A data scientist is training a regression model. The training loss is decreasing but the validation loss starts to increase after a few epochs. Which technique should the scientist use to address this issue?

Accepted Answer

Implement early stopping based on validation loss.. Early stopping halts training when validation loss stops improving, which prevents overfitting by not allowing the model to memorize noise after it has generalized. Option A is incorrect because decreasing batch size introduces more noise into gradient estimates but does not directly address overfitting; it may even slow convergence. Option C is incorrect because adding more layers increases model capacity, which typically worsens overfitting by allowing the model to fit training data even more closely. Option D is incorrect because increasing the learning rate can cause the optimizer to diverge or overshoot minima, and it does not mitigate overfitting.

Answer

Decrease the batch size.

Answer

Add more layers to the model.

Answer

Increase the learning rate.

Question 11

A data scientist has this IAM policy attached to their role. When trying to create a SageMaker endpoint using the AWS CLI, they get an 'AccessDenied' error. What is the most likely reason?

Accepted Answer

The role does not have permission to access the S3 bucket containing the model artifacts. The role does not have permission to access the S3 bucket containing the model artifacts. When creating a SageMaker endpoint, the service needs to access the model artifacts stored in S3. The IAM role attached to the data scientist must have s3:GetObject permission on the bucket. Without this, the CreateEndpoint API call will fail with AccessDenied. Option B is incorrect because sagemaker:InvokeEndpoint is not required to create an endpoint; it is needed for invoking the endpoint after creation. Option A is less likely because KMS decryption is only needed if the model artifacts are encrypted with a customer-managed KMS key. Option D is not a typical cause because using a wildcard '*' for resources is acceptable and does not cause AccessDenied errors.

Answer

The role does not have permission to use KMS to decrypt the model artifacts

Answer

The policy does not include 'sagemaker:InvokeEndpoint' action

Answer

The policy uses a wildcard '*' for resources instead of specific ARNs

Question 12

A company is using SageMaker to train a TensorFlow model for image classification. The training is slow on a single GPU instance. Which TWO strategies can reduce training time? (Choose TWO.)

Accepted Answer

Use SageMaker Pipe Mode for data ingestion. Options B and D are correct. SageMaker Pipe Mode streams training data directly from S3, reducing data download time. Distributed training with multiple GPUs parallelizes the computation, speeding up training. Option A (increase image size) increases computational load and slows training. Option C (increase epochs) adds more iterations, increasing time. Option E (decrease batch size) can reduce GPU utilization efficiency, often slowing training.

Answer

Increase the image size

Answer

Increase the number of training epochs

Answer

Decrease the batch size

Question 13

A company wants to use Amazon SageMaker to train a model, but the training data contains personally identifiable information (PII). The data scientist needs to ensure that the PII is not accessible during training. The data is stored in S3. What is the MOST secure approach?

Accepted Answer

Use SageMaker's built-in mechanisms to encrypt data at rest and in transit, and ensure the training container does not have direct S3 access.. Using SageMaker with a SageMaker-managed KMS key encrypts data at rest and in transit, and the training container cannot access S3 directly. Option A is wrong because encryption alone does not prevent access. Option B is wrong because Redshift adds complexity. Option C is wrong because a VPC alone does not encrypt data.

Answer

Configure a VPC for the SageMaker notebook and training job.

Answer

Load the data into Amazon Redshift and use Redshift ML.

Answer

Use server-side encryption with S3-managed keys (SSE-S3).

Question 14

A data scientist is trying to create a SageMaker endpoint using an IAM role with the attached policy. The operation fails with 'AccessDenied'. What is the MOST likely cause?

Accepted Answer

The policy does not allow sagemaker:CreateEndpoint.. The error 'AccessDenied' when creating a SageMaker endpoint is most likely due to missing sagemaker:CreateEndpoint permission. To create an endpoint, the IAM role must have the sagemaker:CreateEndpoint action. While InvokeEndpoint is needed for invoking the endpoint after creation, it is not required during creation. S3 bucket access may be needed for model artifacts, but that would typically cause a different error or occur earlier in the process (when creating the model). Therefore, option C is the most likely cause.

Answer

The policy does not allow sagemaker:InvokeEndpoint.

Answer

The policy does not allow access to the S3 bucket for model artifacts.

Answer

The policy does not allow sagemaker:CreateModel.

Question 15

Refer to the exhibit. A data scientist ran a SageMaker training job using a built-in XGBoost algorithm. The job failed with the error shown. Which step should the data scientist take to fix the issue?

Accepted Answer

Remove the metric definition from the training job configuration. The error indicates that SageMaker's built-in XGBoost algorithm does not support a custom metric named 'accuracy' because XGBoost's built-in objective functions do not compute accuracy natively. Removing the metric definition from the training job configuration resolves the issue by allowing SageMaker to use the default metrics that XGBoost does support. Changing the metric to 'validation:rmse' is not the correct step because the error specifically indicates that the metric definition is the problem; removing it is the proper fix.

Answer

Write a custom algorithm that calculates accuracy

Answer

Change the metric to 'validation:rmse'

Answer

Use a different built-in algorithm that supports accuracy