MLA-C01 Practice Test 16 — 15 Questions

Question 1

A model deployed on SageMaker is returning inaccurate predictions for certain customer segments. The team suspects data drift. Which SageMaker feature should they use to continuously monitor input data distribution?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor continuously monitors input data for drift, alerting when distributions change. Other features serve different purposes: Clarify for bias and explainability, Debugger for training, Feature Store for feature management.

Answer

SageMaker Clarify

Answer

SageMaker Debugger

Answer

SageMaker Feature Store

Question 2

A team is using Amazon SageMaker to train a neural network. They want to minimize training time while effectively exploring the hyperparameter space. Which approach should they use?

Accepted Answer

Bayesian optimization. Bayesian optimization is efficient as it builds a probabilistic model and selects hyperparameters that are likely to improve performance, making it faster than exhaustive methods like grid search.

Answer

Random search

Answer

Grid search

Answer

Manual tuning

Question 3

Refer to the exhibit. A data scientist receives an AccessDenied error when trying to create a training job using SageMaker. What is the most likely cause?

Accepted Answer

Missing s3:PutObject permission. The policy allows sagemaker:CreateTrainingJob and s3:GetObject, but training jobs also need to write logs and output to S3 (s3:PutObject). The missing s3:PutObject permission causes the AccessDenied error.

Answer

Missing sagemaker:CreateTrainingJob permission

Answer

Missing sagemaker:DescribeTrainingJob permission

Answer

Using wrong AWS region

Question 4

A data science team is using Amazon SageMaker Pipelines to orchestrate a multi-step workflow that includes data preprocessing, training, and model evaluation. They want to reuse the preprocessed data across multiple pipeline executions without re-running the preprocessing step if the source data hasn't changed. What should they configure?

Accepted Answer

Use SageMaker Processing steps with caching. SageMaker Pipelines supports step caching, which allows reusing the output of a step if its inputs and parameters are unchanged. SageMaker Feature Store is for feature storage and serving, not for pipeline step reuse. Checkpointing is for training resumption, not step caching. Data Wrangler preprocesses but caching is a pipeline feature.

Answer

Use SageMaker Training steps with checkpointing

Answer

Use SageMaker Feature Store to store the preprocessed features

Answer

Use SageMaker Data Wrangler for the preprocessing

Question 5

A data scientist wants to evaluate the performance of a binary classification model. The dataset is highly imbalanced with only 5% positive class. Which metric should be used to evaluate the model?

Accepted Answer

F1-score. F1-score balances precision and recall, making it suitable for imbalanced datasets. Accuracy can be misleading (e.g., 95% if always predicting negative). Mean Squared Error and R-squared are for regression.

Answer

Accuracy

Answer

Mean Squared Error

Answer

R-squared

Question 6

Refer to the exhibit. A data scientist configured an automatic model tuning job for a classification model. The tuning job completed after 20 training jobs, but the best validation accuracy was only 0.65. What is the most effective way to potentially improve the result?

Accepted Answer

Increase MaxNumberOfTrainingJobs to 100. With only 20 training jobs, Bayesian optimization may not have fully explored the hyperparameter space. Increasing the maximum number of training jobs allows more exploration and increases the chance of finding better hyperparameters. Changing to random search could help but Bayesian is generally more efficient. Changing the objective to training accuracy would not improve generalization. Increasing parallel jobs does not increase total exploration.

Answer

Change the strategy to Random

Answer

Change the objective metric to training:accuracy

Answer

Increase MaxParallelTrainingJobs to 10

Question 7

An ML engineer creates a SageMaker inference pipeline with two containers: a preprocessor and a predictor. The preprocessor is a lightweight Python script that transforms input data. How should the engineer structure the endpoints to ensure both containers run sequentially?

Accepted Answer

Create a PipelineModel in SageMaker with both containers listed in order: first preprocessor, then predictor.. Option A is correct because SageMaker inference pipelines chain containers in the order specified in the pipeline model. You create a PipelineModel with a list of containers. Option B is wrong because Lambda is not needed; the pipeline handles sequencing. Option C is wrong because batch transform is for batch processing, not sequential inference. Option D is wrong because using a single container would require bundling, which is less modular.

Answer

Use batch transform with two transform jobs chained together.

Answer

Use an AWS Lambda function as a proxy to invoke the preprocessor and then the predictor separately.

Answer

Combine the preprocessor and predictor into a single Docker container.

Question 8

Refer to the exhibit. A data scientist reviews the output of a SageMaker training job. The model has 95% training accuracy and 92% validation accuracy. Which statement is true?

Accepted Answer

The model has acceptable performance with a small generalization gap. Option C is correct because a 3% gap between training and validation accuracy is typically considered a small generalization gap and indicates acceptable performance. Option A (overfitting) would be a larger gap. Option B (underfitting) would show low accuracy on both sets. Option D (more epochs) may not help if the model is already converging.

Answer

The model is underfitting because the validation accuracy is too low

Answer

The model needs more epochs to improve validation accuracy

Answer

The model is overfitting because the training accuracy is higher than validation accuracy

Question 9

A data scientist is using Amazon SageMaker Data Wrangler for data preparation. Which two tasks can be performed using Data Wrangler's built-in transforms? (Choose two.)

Accepted Answer

Encoding categorical variables. Data Wrangler includes built-in transforms for handling missing values and encoding categorical variables. Running SQL queries is possible via custom import, but not a built-in transform. Training models and creating ensembles are not part of Data Wrangler.

Answer

Running a SQL query on the data

Answer

Creating an ensemble of models

Answer

Training a custom machine learning model

Question 10

A trained model needs to be deployed for real-time inference with low latency. Which AWS service is best suited for this?

Accepted Answer

SageMaker endpoints. SageMaker endpoints provide managed, scalable, and low-latency real-time inference. Batch Transform is for offline inference, Hyperparameter Tuning is for training, and Lambda is for serverless but lacks native ML optimizations.

Answer

SageMaker Batch Transform

Answer

SageMaker Hyperparameter Tuning

Answer

AWS Lambda with model packaged

Question 11

Refer to the exhibit. A SageMaker training job logs show training AUC increasing but validation AUC plateauing at 0.880. What is the most likely issue?

Accepted Answer

Overfitting. Training AUC continues to increase while validation AUC stops improving and even drops slightly, indicating overfitting. Underfitting would show both low, high learning rate would cause erratic behavior, and insufficient data typically causes high variance but not this pattern.

Answer

Learning rate too high

Answer

Underfitting

Answer

Insufficient training data

Question 12

A data scientist is preparing text data for a sentiment analysis model using Amazon SageMaker. Which two data preprocessing techniques are commonly used when working with text data for natural language processing? (Choose two.)

Accepted Answer

Tokenization. Stop word removal and tokenization are standard text preprocessing steps. One-hot encoding of all words leads to high dimensionality and is rarely used directly. Image resizing is for images, and PCA is for numerical dimensionality reduction.

Answer

One-hot encoding of all words

Answer

Image resizing

Answer

Principal component analysis (PCA)

Question 13

During model training on Amazon SageMaker, the training job fails with a 'ResourceLimitExceeded' error. What is the most likely cause?

Accepted Answer

The account's instance limit for the chosen instance type has been reached. The ResourceLimitExceeded error typically indicates that the AWS account has reached its service limit for the specified instance type. A syntax error would cause a different error (e.g., ClientError). A dataset too large might cause an out-of-memory error but not this specific error. Learning rate does not cause resource limits.

Answer

The algorithm's learning rate is too high

Answer

The dataset is too large for the instance

Answer

The training script has a syntax error

Question 14

Refer to the exhibit. A data scientist configured SageMaker Debugger to monitor training for overfitting. However, the rule never triggers even though the model appears to be overfitting. What is the most likely reason?

Accepted Answer

The debug hook is not collecting the validation loss. The DebugHookConfig only collects losses and gradients. The overfitting rule likely compares training loss to validation loss, but validation loss is not being collected. Without a collection for validation loss (e.g., validation:loss), the rule cannot evaluate the condition for overfitting. The rule evaluator image, instance type, and S3 path are less likely causes; the image is a placeholder but might be valid.

Answer

The instance type for the rule is too small

Answer

The S3 output path is not writable

Answer

The rule evaluator image is incorrect

Question 15

Refer to the exhibit. A data scientist ran a SageMaker training job using a built-in algorithm. The job failed with the above error. What is the most likely cause?

Accepted Answer

The input CSV file has missing or mismatched column headers.. Option B is correct because the error explicitly states the data format is incorrect; the CSV headers do not match the expected schema. Option A is wrong because the error does not mention memory. Option C is wrong because the error is about data format, not permissions. Option D is wrong because the algorithm is built-in and should support CSV if headers match.

Answer

The S3 bucket lacks proper permissions for SageMaker to read the training data.

Answer

The built-in algorithm does not support CSV input format.

Answer

The training instance ran out of memory.