MLA-C01 ML Model Development Practice Test 4 — 15 Questions

Question 1

A model deployed on SageMaker is returning inaccurate predictions for certain customer segments. The team suspects data drift. Which SageMaker feature should they use to continuously monitor input data distribution?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor continuously monitors input data for drift, alerting when distributions change. Other features serve different purposes: Clarify for bias and explainability, Debugger for training, Feature Store for feature management.

Answer

SageMaker Clarify

Answer

SageMaker Debugger

Answer

SageMaker Feature Store

Question 2

A data scientist is using Amazon SageMaker Data Wrangler for data preparation. Which two tasks can be performed using Data Wrangler's built-in transforms? (Choose two.)

Accepted Answer

Encoding categorical variables. Data Wrangler includes built-in transforms for handling missing values and encoding categorical variables. Running SQL queries is possible via custom import, but not a built-in transform. Training models and creating ensembles are not part of Data Wrangler.

Answer

Running a SQL query on the data

Answer

Creating an ensemble of models

Answer

Training a custom machine learning model

Question 3

A data scientist is building a text classification model using Amazon SageMaker. The dataset is large and includes imbalanced classes. Which three techniques can help improve model performance? (Choose three.)

Accepted Answer

Using cost-sensitive learning. Oversampling, SMOTE, and cost-sensitive learning are standard approaches to handle class imbalance. Using a linear classifier only is limiting, and TF-IDF is a feature extraction method that does not address imbalance directly.

Answer

Performing feature extraction using TF-IDF

Answer

Using a linear classifier only

Question 4

A company uses SageMaker to train a model. They want to ensure that training data is encrypted at rest and in transit, and that only authorized users can access the training artifacts. Which three steps should they take? (Choose three.)

Accepted Answer

Configure IAM policies to restrict access to SageMaker resources. Option A is correct because IAM policies allow you to define fine-grained permissions to control which users or roles can create, describe, or delete SageMaker resources (e.g., training jobs, endpoints). By restricting access via IAM, you ensure that only authorized principals can interact with training artifacts, such as model output in S3 or logs in CloudWatch. This directly addresses the requirement of limiting access to authorized users.

Answer

Use SageMaker Model Monitor

Answer

Use SageMaker Network Isolation

Question 5

Refer to the exhibit. The training job failed. What is the MOST likely cause?

Accepted Answer

The training data size exceeds the available EBS volume size. The log indicates no usable scratch space, meaning the default EBS volume is full. The retry with local SSD suggests that the instance might not have local SSD, or it also fails. The most likely cause is that the training data or intermediate files exceed the available EBS volume size.

Answer

The learning rate is too high

Answer

The instance type does not have SSD storage

Answer

The instance type does not have enough memory

Answer

The number of epochs is too low

Question 6

Refer to the exhibit. The data scientist wants to update the endpoint to use a new model version without downtime. Which approach should they use?

Accepted Answer

Create a new endpoint configuration with a second variant and update the endpoint. To update an endpoint without downtime, create a new endpoint configuration that includes both the old and new model variants, then update the endpoint to use that configuration. The endpoint gradually shifts traffic to the new variant. Deleting the endpoint causes downtime, direct model name update is not allowed, and Model Monitor does not handle deployments.

Answer

Delete the existing endpoint and create a new one

Answer

Update the endpoint's model name directly

Answer

Use SageMaker Model Monitor to automatically switch

Question 7

Refer to the exhibit. A user launches a SageMaker notebook instance with this lifecycle configuration. What happens?

Accepted Answer

The script runs every time the notebook is started. SageMaker lifecycle configurations run every time the notebook instance is started (including after stop and start). They do not run only on first start or after kernel resets.

Answer

The script runs after each kernel reset

Answer

The script runs only on the first start

Answer

The script runs only when creating the instance

Question 8

A data scientist is building a model to predict customer churn based on historical data. The dataset has 10 features and 100,000 records, and the target is binary. Which algorithm is most appropriate for this binary classification problem?

Accepted Answer

Logistic regression. Logistic regression is a standard algorithm for binary classification, providing probabilistic outputs and interpretability. Linear regression is for regression, K-means is for clustering, and PCA is for dimensionality reduction.

Answer

Principal component analysis

Answer

K-means clustering

Answer

Linear regression

Question 9

A machine learning engineer trains a binary classifier and obtains an accuracy of 95% on the test set. The dataset is imbalanced with 95% positive class. What is the most important metric to evaluate the model's performance?

Accepted Answer

F1 score. Accuracy is misleading on imbalanced datasets because a model that always predicts the majority class achieves high accuracy. F1 score balances precision and recall, providing a more reliable measure.

Answer

R-squared

Answer

Accuracy

Answer

RMSE

Question 10

Which technique is commonly used to handle missing values in a categorical feature?

Accepted Answer

Mode imputation. Mode imputation (replacing missing values with the most frequent category) is a standard method for categorical data. Mean imputation is for numerical data, standard scaling is for feature scaling, and one-hot encoding encodes categories without handling missing values.

Answer

One-hot encoding

Answer

Mean imputation

Answer

Standard scaling

Question 11

A team is using Amazon SageMaker to train a neural network. They want to minimize training time while effectively exploring the hyperparameter space. Which approach should they use?

Accepted Answer

Bayesian optimization. Bayesian optimization is efficient as it builds a probabilistic model and selects hyperparameters that are likely to improve performance, making it faster than exhaustive methods like grid search.

Answer

Random search

Answer

Grid search

Answer

Manual tuning

Question 12

A trained model needs to be deployed for real-time inference with low latency. Which AWS service is best suited for this?

Accepted Answer

SageMaker endpoints. SageMaker endpoints provide managed, scalable, and low-latency real-time inference. Batch Transform is for offline inference, Hyperparameter Tuning is for training, and Lambda is for serverless but lacks native ML optimizations.

Answer

SageMaker Batch Transform

Answer

SageMaker Hyperparameter Tuning

Answer

AWS Lambda with model packaged

Question 13

Which feature scaling method is most robust to outliers in the data?

Accepted Answer

Robust scaling. Robust scaling uses median and interquartile range, making it less sensitive to outliers than standardization (mean and standard deviation) or min-max scaling (range dependent on extremes).

Answer

Normalization (L2)

Answer

Standardization (Z-score)

Answer

Min-max scaling

Question 14

A model has high training accuracy but low validation accuracy. Which action is least likely to reduce overfitting?

Accepted Answer

Increase model complexity. Increasing model complexity (e.g., adding more layers or parameters) makes the model more flexible, which typically exacerbates overfitting by allowing it to memorize noise in the training data. Since the goal is to reduce overfitting, this action is counterproductive and therefore the least likely to help.

Answer

Use dropout

Answer

Increase regularization strength

Answer

Add more training data

Question 15

A company wants to forecast monthly sales that show clear seasonality. Which algorithm is most suitable?

Accepted Answer

ARIMA (Seasonal ARIMA). ARIMA (specifically SARIMA for seasonality) is designed for univariate time series forecasting with seasonal patterns. Linear regression may capture trends but not seasonality, random forest can be used but not optimal for time series, K-means is clustering.

Answer

Random forest

Answer

K-means clustering

Answer

Linear regression