MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 7 — 15 Questions

Question 1

A company wants to automatically trigger a retraining pipeline when concept drift is detected in their deployed model. Which combination of services should they use?

Accepted Answer

SageMaker Model Monitor → CloudWatch Alarm → SNS → Lambda. SageMaker Model Monitor detects concept drift by analyzing model predictions against a baseline, then publishes metrics to CloudWatch. A CloudWatch Alarm triggers when drift exceeds a threshold, sending a notification via SNS to invoke a Lambda function, which starts the retraining pipeline. This end-to-end integration ensures automated, event-driven retraining without manual intervention.

Answer

SageMaker Model Monitor → Lambda

Answer

CloudWatch Events → SageMaker Training Job

Answer

SageMaker Clarify → SNS → Step Functions

Question 2

A machine learning team needs to ensure that all model training and inference jobs within SageMaker Studio run in a private network without internet access. The team also requires that inter-container traffic within the same training job be encrypted. Which configurations should they combine?

Accepted Answer

Enable network isolation mode and inter-container traffic encryption. To run SageMaker jobs in a private network without internet access, you enable network isolation mode, which prevents containers from accessing the internet. To encrypt inter-container traffic within the same training job, you enable inter-container traffic encryption. These two configurations together meet both requirements.

Answer

Configure SageMaker Studio in VPC-only mode and use KMS encryption

Answer

Use a VPC with a NAT gateway and enable network isolation

Answer

Enable inter-container traffic encryption and use a VPC with VPC endpoints

Question 3

A data scientist uses SageMaker Model Monitor to track feature attribution drift. Which technique does SageMaker Model Monitor use to compute feature attributions?

Accepted Answer

SHAP. SageMaker Model Monitor uses SHAP (SHapley Additive exPlanations) to compute feature attributions for model explainability and drift detection. SHAP provides a unified measure of feature importance based on cooperative game theory, ensuring consistent and locally accurate attributions across all features.

Answer

Permutation Feature Importance

Answer

Partial Dependence Plots

Answer

LIME

Question 4

A company wants to reduce costs for a SageMaker real-time endpoint that has variable traffic. Which feature allows the endpoint to automatically adjust instance count based on demand?

Accepted Answer

Auto Scaling for SageMaker endpoints. Auto Scaling for SageMaker endpoints is the native capability that dynamically adjusts the number of instances behind a real-time endpoint based on CloudWatch metrics such as InvocationsPerInstance or ModelLatency. It uses Application Auto Scaling policies (target tracking or step scaling) to add instances during traffic spikes and remove them during lulls, directly reducing cost for variable workloads. Savings Plans and Inference Recommender do not perform runtime scaling.

Answer

SageMaker Savings Plans

Answer

SageMaker Inference Recommender

Answer

SageMaker Model Monitor

Question 5

A hospital deploys a model to predict patient readmission risk. To comply with regulations, they must ensure that the model's predictions do not show bias against any demographic group over time. Which service should they use for ongoing monitoring?

Accepted Answer

SageMaker Clarify. SageMaker Clarify is the correct service because it is specifically designed to detect bias in ML model predictions and can be configured for ongoing monitoring. It provides bias metrics (e.g., difference in positive proportion, disparate impact) and can run on a schedule to continuously evaluate predictions against demographic groups, ensuring regulatory compliance over time.

Answer

AWS Audit Manager

Answer

SageMaker Model Monitor

Answer

Amazon Macie

Question 6

A team uses SageMaker ML Lineage Tracking to capture the metadata of their ML workflow. They want to query the lineage to see which model version was trained from a specific dataset. Which Lineage Tracking entity represents the dataset?

Accepted Answer

Artifact. In SageMaker ML Lineage Tracking, an Artifact represents a tangible object or data — datasets, models, model versions, and endpoints are all artifacts. The dataset is therefore represented as an Artifact, and lineage queries traverse associations between artifacts, actions, and contexts to trace which model version was trained from it.

Answer

Association

Answer

Action

Answer

Context

Question 7

An organization wants to schedule a retraining pipeline to run every Sunday night. Which AWS service should they use to trigger the pipeline on a schedule?

Accepted Answer

Amazon EventBridge. Amazon EventBridge is the correct choice because it provides a scheduled event source using cron or rate expressions to trigger target services at specified times, such as every Sunday night. It can directly invoke an AWS Step Functions state machine or a Lambda function to start the retraining pipeline, making it the native scheduling service for event-driven workflows in AWS.

Answer

AWS Lambda

Answer

AWS Step Functions

Answer

Amazon SQS

Question 8

A team monitors a production endpoint and notices a sudden increase in 5XXError count. Which of the following is the most likely cause?

Accepted Answer

The model container is out of memory or crashing. A sudden increase in 5XX errors, particularly HTTP 503 or 502, typically indicates that the model container is failing to process requests due to resource exhaustion (e.g., OOM kills) or a crash in the inference process. In a production ML endpoint, such errors often stem from the container running out of memory, leading to the container being terminated by the orchestrator (e.g., Kubernetes OOMKill) or the application crashing internally, which directly causes 5XX responses.

Answer

The endpoint is under-provisioned and requests are throttled

Answer

The input data format has changed

Answer

The model is returning predictions with high latency

Question 9

A machine learning team needs to monitor a deployed model for both data drift and concept drift. Which TWO approaches should they implement? (Select TWO.)

Accepted Answer

Set up SageMaker Model Monitor for data quality monitoring. Option A is correct because SageMaker Model Monitor's data quality monitoring detects data drift by comparing the statistical properties of incoming inference requests against the baseline statistics captured from the training dataset, alerting when feature distributions shift. Option D is correct because Model Monitor's model quality monitoring evaluates concept drift by comparing predicted values against actual ground-truth labels, tracking metrics such as accuracy, precision, and recall over time to detect degradation in the model's real-world performance. Option B is not correct because SageMaker Clarify is used for bias detection and explainability (e.g., SHAP values), not for detecting data or concept drift. Option C is not correct because CloudWatch Logs Insights merely queries and analyzes log data; it does not provide built-in drift detection statistics or baseline comparisons. Option E is not correct because SageMaker Debugger is designed to debug training jobs by capturing tensors and monitoring training metrics, not to monitor deployed models for drift.

Answer

Use SageMaker Clarify for bias monitoring

Answer

Configure CloudWatch Logs Insights to query inference logs

Answer

Enable SageMaker Debugger during inference

Question 10

An ML engineer monitors a SageMaker endpoint for data drift. They set up SageMaker Model Monitor to compare inference data against a baseline created from the training dataset. The monitoring schedule runs daily and reports violations. Which monitoring type should be configured to detect if the distribution of a numerical feature in real-time inference data differs significantly from the training distribution?

Accepted Answer

Data quality monitoring. SageMaker Model Monitor's data quality monitoring detects feature distribution drift (statistical drift) between baseline and live data. Model quality monitoring requires ground truth labels, bias drift monitors fairness metrics, and feature attribution drift monitors SHAP values.

Answer

Feature attribution drift monitoring

Answer

Bias drift monitoring

Answer

Model quality monitoring

Question 11

A company wants to automate remediation when a SageMaker endpoint's latency exceeds a threshold for more than 5 minutes. The team needs to be notified and a Lambda function should be invoked to scale up the endpoint. Which combination of services should be used?

Accepted Answer

CloudWatch Alarm → SNS topic → Lambda function. CloudWatch Alarms evaluate SageMaker endpoint metrics (e.g., ModelLatency) against thresholds over evaluation periods; when the alarm fires after 5 minutes of breach, it publishes to an SNS topic, which can fan out to both email/SMS subscribers for notification and a Lambda function for automated scaling. This is the canonical AWS pattern for metric-driven remediation.

Answer

EventBridge rule to trigger Lambda

Answer

CloudWatch Logs subscription filter → Lambda function

Answer

SageMaker Model Monitor → Lambda function

Question 12

A machine learning team trains a model in SageMaker and wants to track every step — from dataset version to hyperparameters to final model artifact — for reproducibility and audit compliance. Which SageMaker feature should they use?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the correct choice because it is specifically designed to create a directed acyclic graph (DAG) of every step in the ML workflow, including dataset versions, hyperparameters, training jobs, and model artifacts. This enables full reproducibility and audit compliance by capturing the provenance of each entity and their relationships, which is exactly what the question requires.

Answer

SageMaker Feature Store

Answer

SageMaker Experiments

Answer

SageMaker Model Registry

Question 13

A company deploys a model for fraud detection. They want to monitor if the model's predictions become less accurate over time due to changes in the underlying data distribution, but they do not have immediate access to ground truth labels. Which type of drift should they monitor as a proxy?

Accepted Answer

Data drift. Data drift (option C) is the correct proxy to monitor when ground truth labels are unavailable because it detects changes in the input feature distribution over time. If the underlying data distribution shifts, the model's predictions are likely to become less accurate even if the relationship between features and labels remains stable. This allows teams to trigger retraining or investigation before model quality degrades.

Answer

Feature attribution drift

Answer

Model quality drift

Answer

Concept drift

Question 14

An organization needs to ensure that all data used for inference on a SageMaker endpoint is encrypted at rest. The endpoint uses a SageMaker-provided container. Which configuration should be applied?

Accepted Answer

Specify a KMS key in the endpoint configuration. SageMaker endpoints use AWS KMS for encryption at rest. By specifying a KMS key in the endpoint configuration, the data in the attached ML storage volume is encrypted. Inter-container traffic encryption is for encryption in transit.

Answer

Use a custom container with built-in encryption

Answer

Enable network isolation mode

Answer

Enable inter-container traffic encryption

Question 15

A company deploys a real-time inference endpoint with auto-scaling using a target tracking policy based on average Invocations per instance. They notice that during a traffic spike, the endpoint scales out too late, causing increased latency. They want to scale proactively before the spike. Which strategy should they implement?

Accepted Answer

Use a scheduled scaling action to add capacity before the expected spike. A scheduled scaling action adds capacity at a predetermined time before the expected traffic spike, allowing the endpoint to be ready proactively. Target tracking reacts after metrics breach thresholds, which is inherently reactive and too slow for sharp spikes. Scheduled scaling is the correct strategy when spikes are predictable (e.g., known business hours or events).

Answer

Enable provisioned concurrency on the endpoint

Answer

Pre-warm the endpoint by sending dummy requests

Answer

Switch to a step scaling policy with a higher cooldown period