MLA-C01 Practice Test 14 — 15 Questions

Question 1

A company deploys a deep learning model to a real-time SageMaker endpoint. After deployment, users report high inference latency. Which action is the MOST effective first step to reduce latency?

Accepted Answer

Compile the model using SageMaker Neo to optimize for the target instance.. SageMaker Neo compiles the trained model to optimize it for the target instance hardware, reducing inference latency without requiring additional resources. This is the most effective first step because it directly addresses model execution efficiency, often yielding significant speedups for deep learning models.

Answer

Switch to a larger instance type with more GPU memory.

Answer

Enable SageMaker Model Monitor to capture inference data.

Answer

Increase the number of instances in the endpoint to handle more requests.

Question 2

A company is training a binary classifier in SageMaker and observes that the training loss decreases but validation loss increases after a few epochs. What is the most likely issue?

Accepted Answer

Overfitting. The training loss decreasing while validation loss increasing after a few epochs is the classic signature of overfitting. The model is memorizing the training data (including noise) rather than learning generalizable patterns, which causes it to perform poorly on unseen validation data. In SageMaker, this often occurs when the model has too many parameters relative to the dataset size, or when regularization techniques like dropout or L2 weight decay are insufficient.

Answer

Learning rate too high

Answer

Underfitting

Answer

Data imbalance

Question 3

A hospital deploys a model to predict patient readmission risk. To comply with regulations, they must ensure that the model's predictions do not show bias against any demographic group over time. Which service should they use for ongoing monitoring?

Accepted Answer

SageMaker Clarify. SageMaker Clarify is the correct service because it is specifically designed to detect bias in ML model predictions and can be configured for ongoing monitoring. It provides bias metrics (e.g., difference in positive proportion, disparate impact) and can run on a schedule to continuously evaluate predictions against demographic groups, ensuring regulatory compliance over time.

Answer

AWS Audit Manager

Answer

SageMaker Model Monitor

Answer

Amazon Macie

Question 4

A company has 200 small PyTorch models that are each used infrequently but need to be available for real-time inference. To minimize costs, they want to host all models on a single endpoint. Which SageMaker feature should they use?

Accepted Answer

Multi-model endpoint (MME). SageMaker multi-model endpoints (MME) let a single endpoint host hundreds or thousands of models behind one container, loading each model into memory or disk on demand and unloading idle ones. This is purpose-built for the scenario of many small, infrequently used models that must still serve real-time inference, because you pay for one endpoint's worth of instances rather than one endpoint per model. The SageMaker SDK/API passes a TargetModel parameter on each InvokeEndpoint call so the endpoint knows which model to load and serve.

Answer

Multi-container endpoint

Answer

Batch Transform job

Answer

Asynchronous inference endpoint

Question 5

A data scientist is working on a binary classification problem and wants to use AWS Glue for data preparation. The dataset has missing values in several numeric columns. Which imputation strategy is MOST appropriate for the scientist to apply in AWS Glue ETL?

Accepted Answer

Use the FillMissingValues transform to replace missing values with the mean of each column. AWS Glue ETL (PySpark) supports the `Imputer` transformer which can impute missing numeric values using the mean or median of the column. This is a built-in, straightforward approach.

Answer

Use a machine learning model to predict missing values

Answer

Drop all rows with missing values using the Drop transform

Answer

Set missing values to zero

Question 6

A data science team uses SageMaker to train and deploy models. They need to track model lineage, including datasets, training jobs, and model versions, to ensure reproducibility. Which THREE actions should they take? (Select THREE)

Accepted Answer

Enable SageMaker ML Lineage Tracking. A is correct because SageMaker ML Lineage Tracking automatically captures the relationships between datasets, training jobs, and model versions, creating a directed acyclic graph (DAG) of the ML workflow. This enables full reproducibility by allowing you to trace which data and code produced a specific model, without manual intervention.

Answer

Store trained models in a public S3 bucket

Answer

Use SageMaker Experiments to organize training runs

Question 7

A company trains a model daily using Amazon SageMaker and uses the model for real-time inference. They want to detect data drift between the training data and the inference data to decide when to retrain. Which AWS service should they use for this purpose?

Accepted Answer

Amazon SageMaker Model Monitor. Amazon SageMaker Model Monitor is the correct service because it is specifically designed to continuously monitor machine learning models in production for data drift, feature attribution drift, and quality issues. It compares the distribution of live inference data against the baseline training data statistics and alerts when drift exceeds defined thresholds, enabling timely retraining decisions.

Answer

Amazon Athena

Answer

AWS Glue

Answer

AWS Lambda

Question 8

A data science team is using Amazon SageMaker to train and deploy a binary classification model. They want to continuously monitor the model for data drift in production. Which combination of AWS services and SageMaker features should they use to implement automated drift detection with minimal operational overhead?

Accepted Answer

SageMaker Model Monitor and Amazon CloudWatch. SageMaker Model Monitor is the native SageMaker feature designed specifically for continuously monitoring deployed models for data drift, bias drift, and feature attribution drift. It automatically captures inference requests and responses, computes statistics, and publishes metrics to Amazon CloudWatch, which can trigger alarms for drift detection. This combination provides automated drift detection with minimal operational overhead because it requires no custom infrastructure or manual scheduling.

Answer

SageMaker Debugger and Amazon SNS

Answer

SageMaker Pipelines and AWS Lambda

Answer

SageMaker Clarify and AWS Config

Question 9

A gaming company uses a SageMaker endpoint for real-time player churn prediction. The model is updated weekly. After a recent retraining, the team notices that the endpoint's predicted probabilities for churn have shifted dramatically: the average predicted probability dropped from 0.3 to 0.05. The team suspects concept drift (the relationship between features and target changed) rather than data drift. They have SageMaker Model Monitor set up for data drift and quality metrics, but not for bias or explainability. The team needs to confirm concept drift and take corrective action. Which approach should the team take FIRST?

Accepted Answer

Configure SageMaker Model Monitor's model quality monitoring to compare predictions against actual outcomes collected from a week of production traffic. To detect concept drift, the team needs to compare the model's predictions against actual observed outcomes (ground truth). SageMaker Model Monitor's quality monitoring can track prediction accuracy over time if ground truth is provided. Option A (Configure SageMaker Model Monitor's model quality monitoring) is the correct first step. Option B (retrain with more recent data) might help but does not confirm drift. Option D (investigate data drift by reviewing feature distribution) checks feature distribution, not concept drift. Option C (use Clarify for SHAP values) is for feature importance, not drift detection.

Answer

Immediately retrain the model using the most recent month of data and redeploy to the endpoint

Answer

Use Amazon SageMaker Clarify to compute SHAP values and understand which features are driving the new predictions

Answer

Investigate data drift by reviewing the Model Monitor feature distribution constraints and comparing recent input data to the baseline

Question 10

A data scientist is using Amazon SageMaker Data Wrangler to prepare a dataset. The dataset contains a column with date strings in the format 'YYYY-MM-DD'. The data scientist wants to extract the year, month, and day as separate features. Which Data Wrangler transform should be used?

Accepted Answer

Parse date transform.. The 'Parse date' transform in Amazon SageMaker Data Wrangler is specifically designed to convert date strings into structured datetime components. By applying this transform to the 'YYYY-MM-DD' column, the data scientist can automatically extract year, month, and day as separate features, enabling downstream feature engineering without manual string parsing.

Answer

Encode categorical transform.

Answer

Scale values transform.

Answer

Handle missing transform.

Question 11

A company deploys a model for fraud detection. They need to monitor for bias after deployment, specifically whether the model's false positive rate changes across demographic groups over time. Which SageMaker feature should they use?

Accepted Answer

SageMaker Clarify (post-deployment bias monitoring). SageMaker Clarify provides post-deployment bias monitoring by analyzing predictions against ground truth labels for defined facets. It can track metrics like false positive rate differences over time.

Answer

SageMaker Model Monitor – Model Quality

Answer

SageMaker Model Monitor – Feature Attribution Drift

Answer

SageMaker Model Monitor – Data Quality

Question 12

A team wants to automatically retrain a model when new labeled data arrives. Which SageMaker feature can orchestrate this workflow?

Accepted Answer

SageMaker Pipelines. SageMaker Pipelines is a purpose-built CI/CD service for machine learning that allows you to define, orchestrate, and automate end-to-end ML workflows, including retraining models when new labeled data arrives. You can create a pipeline that triggers on new data events (e.g., via an S3 event notification or a Lambda function) and automatically executes steps such as data processing, training, evaluation, and model registration. This makes it the correct choice for orchestrating an automated retraining workflow.

Answer

SageMaker Model Monitor

Answer

SageMaker Debugger

Answer

SageMaker Autopilot

Question 13

Which TWO tools are specifically designed for debugging and analyzing training jobs in SageMaker?

Accepted Answer

SageMaker Experiments. SageMaker Debugger (C) is specifically designed for debugging and analyzing training jobs: it captures tensors, metrics, and system resource data during training, and its built-in rules detect issues like vanishing gradients, overfitting, or GPU/CPU bottlenecks in real time. SageMaker Experiments (B) is designed for tracking, organizing, comparing, and analyzing training runs (trials, metrics, parameters, and artifacts), which makes it a core tool for analyzing training jobs. The other options serve different purposes: SageMaker Autopilot (A) automates model building and hyperparameter tuning, SageMaker Clarify (D) provides bias detection and explainability, and SageMaker Model Monitor (E) monitors deployed endpoints for data and model drift rather than debugging training jobs.

Answer

SageMaker Autopilot

Answer

SageMaker Clarify

Answer

SageMaker Model Monitor

Question 14

A data engineer needs to split a time-series dataset into training and validation sets for a forecasting model. Which split method should be used to avoid data leakage?

Accepted Answer

Temporal split where training uses data up to a cutoff date and validation uses later data.. Time-series data has temporal dependencies, and random splits or k-fold cross-validation with shuffling would cause data leakage by allowing future information to influence training. A temporal split ensures that the model is trained only on past data and validated on future data, preserving the chronological order and preventing leakage.

Answer

Use k-fold cross-validation with random shuffling.

Answer

Use feature importance scores to weight the splitting process.

Answer

Random split with 80% training and 20% validation.

Question 15

An ML engineer is using Amazon SageMaker Automatic Model Tuning (AMT) to optimize hyperparameters for a gradient boosting model. The tuning job is taking a long time and has completed many training jobs. The engineer wants to stop training jobs that are unlikely to improve the objective metric. What should they configure?

Accepted Answer

Enable early stopping in the hyperparameter tuning job. Enabling early stopping in the Amazon SageMaker Automatic Model Tuning (AMT) job allows the tuning job to automatically stop training jobs that are unlikely to improve the objective metric based on intermediate results. This reduces the total time and compute cost by terminating poorly performing trials early, which directly addresses the engineer's goal of stopping unpromising training jobs.

Answer

Reduce the number of hyperparameter ranges

Answer

Use a random search strategy instead of Bayesian

Answer

Increase the maximum number of training jobs