MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 6 — 15 Questions

Question 1

An ML engineer wants to be notified when the average inference latency of a SageMaker endpoint exceeds 500 ms for 2 consecutive evaluation periods. Which AWS service combination should they use?

Accepted Answer

CloudWatch Alarm + SNS. CloudWatch Alarms evaluate metrics over specified periods and can trigger SNS notifications when a threshold is breached for a defined number of consecutive periods. SageMaker endpoints automatically publish inference latency metrics to CloudWatch, so an alarm on the latency metric with a 500 ms threshold and 2 evaluation periods, wired to an SNS topic, delivers exactly the required notification.

Answer

SageMaker Model Monitor + SNS

Answer

EventBridge + Lambda

Answer

SageMaker Clarify + SNS

Question 2

A financial institution uses SageMaker to train and deploy models. They need to track every experiment, model version, and deployment step for audit purposes. Which SageMaker feature should they use to capture the full lineage of artifacts, actions, and contexts?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the feature designed to capture the full lineage of artifacts, actions, and contexts, providing an end-to-end audit trail of the machine learning workflow. It automatically records relationships between data, models, and experiments.

Answer

SageMaker Clarify

Answer

SageMaker Model Registry

Answer

SageMaker Experiments

Question 3

A team wants to use SageMaker Clarify to monitor bias in their production model predictions. They have configured a bias drift monitor. What does SageMaker Clarify compare to detect bias drift?

Accepted Answer

Current bias metrics against a baseline bias metrics computed from training data. SageMaker Clarify bias drift monitoring compares the current bias metrics (e.g., disparate impact) computed on live data against a baseline bias metric computed from the training data. This detects if bias has drifted over time.

Answer

Current input data distribution against the training data distribution

Answer

Current SHAP feature attributions against baseline SHAP values

Answer

Current predictions against ground truth labels collected in real-time

Question 4

A machine learning engineer wants to deploy a pre-trained foundation model for text summarization using SageMaker JumpStart. Which of the following is a primary cost consideration when deploying such a model?

Accepted Answer

The cost of GPU instances required for low-latency inference. When deploying a pre-trained foundation model via SageMaker JumpStart, the model is already trained, so fine-tuning cost is optional and not primary. The main ongoing cost is the compute instance used for inference, especially GPU instances needed for low-latency, high-throughput text summarization. Data transfer for inference requests is typically negligible compared to compute, and S3 storage for model artifacts is a minor one-time cost.

Answer

The cost of fine-tuning the model on custom data

Answer

The cost of data transfer for inference requests

Answer

The cost of storing the model artifacts in S3

Question 5

A company uses SageMaker endpoints for real-time inference. They want to automatically scale the number of instances based on the number of outstanding requests. Which auto-scaling policy type should they choose?

Accepted Answer

Target tracking scaling. Target tracking scaling is the correct choice because it lets you specify a target value for a metric — such as a custom metric for outstanding requests per instance — and SageMaker automatically adjusts instance count to keep that metric at the target. This is the recommended policy for metrics that correlate directly with capacity needs, like request backlog or invocations per instance. It handles both scale-out and scale-in automatically without manual threshold tuning.

Answer

Scheduled scaling

Answer

Step scaling

Answer

Simple scaling

Question 6

An organization needs to ensure that all data transmitted between containers in a SageMaker training job is encrypted. In the training job configuration, which setting should they enable?

Accepted Answer

Enable inter-container traffic encryption. SageMaker training jobs support inter-container traffic encryption, which ensures that data transmitted between containers (e.g., distributed training workers) is encrypted in transit. This setting uses TLS to protect the communication channel, meeting the organization's requirement for encrypted data transmission between containers.

Answer

Use a KMS key for data encryption

Answer

Configure the training job in VPC-only mode

Answer

Enable network isolation mode

Question 7

An ML team uses SageMaker to deploy a model for real-time inference. They want to monitor and improve cost efficiency. Which THREE actions should they take? (Select THREE.)

Accepted Answer

Use SageMaker Inference Recommender to find the optimal instance type and count. Option A is correct because SageMaker Inference Recommender runs load tests and benchmarks to recommend the optimal instance type and instance count for a real-time endpoint, directly improving cost efficiency by avoiding over-provisioning. Option B is correct because configuring auto-scaling on a SageMaker endpoint adjusts the number of instances to match traffic demand, so the team pays only for capacity actually needed during peaks and troughs. Option E is correct because SageMaker Savings Plans offer discounted pricing (up to 64% off) in exchange for a committed hourly spend, reducing the cost of steady-state real-time inference workloads. Option C is not a cost-efficiency action; a CloudWatch dashboard for latency is an observability tool and does not by itself reduce spend. Option D is incorrect because Managed Spot Training applies to training jobs, not to real-time inference endpoint instances, which cannot use spot capacity for persistent endpoints.

Answer

Create a CloudWatch dashboard to monitor endpoint latency

Answer

Use SageMaker Managed Spot Training for endpoint instances

Question 8

A machine learning team deploys a model for loan approval. They want to monitor data drift on the real-time endpoint using SageMaker Model Monitor. Which set of actions should they take to set up data quality monitoring?

Accepted Answer

Enable data capture on the endpoint, generate a baseline from training data, create a data quality monitoring schedule, and set up a CloudWatch Alarm on violations. SageMaker Model Monitor requires a baseline from training data, then schedules monitoring jobs that compare live endpoint captures against that baseline. Alerts are sent via CloudWatch Alarms.

Answer

Use SageMaker Clarify to detect data drift on the endpoint

Answer

Create a model quality monitoring schedule directly on the endpoint without any baseline

Answer

Enable data capture and rely on SageMaker Model Monitor to automatically infer drift without a baseline

Question 9

A data scientist notices that a production model's accuracy has degraded over the past week. The training data distribution remains unchanged, but the relationship between features and the target has shifted. Which type of drift is occurring, and which monitoring approach should be used?

Accepted Answer

Concept drift; use SageMaker Model Monitor model quality monitoring with ground truth labels. Concept drift occurs when the underlying relationship between features and target changes. Model quality monitoring (comparing predictions against ground truth) detects this. Data drift monitors feature distribution changes, which are not present here.

Answer

Bias drift; use SageMaker Clarify post-deployment bias monitoring

Answer

Data drift; use SageMaker Model Monitor data quality monitoring

Answer

Feature attribution drift; use SageMaker Clarify

Question 10

A company wants to reduce costs for a real-time inference endpoint that experiences predictable traffic spikes during business hours and low traffic at night. Which auto-scaling policy is MOST cost-effective while maintaining performance?

Accepted Answer

Scheduled scaling that increases instances before business hours and decreases after. Scheduled scaling directly aligns capacity with the predictable traffic pattern (business hours vs. night), allowing you to proactively add instances before demand increases and remove them afterward. This avoids the cost of over-provisioning during low-traffic periods and the latency of reactive scaling, making it the most cost-effective approach for a known, recurring schedule.

Answer

Step scaling based on CPU utilization

Answer

Manual scaling by the operations team

Answer

Target tracking with a custom metric for response time

Question 11

A data scientist deploys a model and wants to monitor the endpoint's invocation latency. They notice that the CloudWatch metric 'ModelLatency' is high, but 'OverheadLatency' is low. Which statement correctly interprets these metrics?

Accepted Answer

The model inference time is the bottleneck; consider optimizing the model or using a faster instance type. The 'ModelLatency' metric measures the time taken by the SageMaker model container to process a single request, including inference and any preprocessing/postprocessing within the container. 'OverheadLatency' measures the time spent on SageMaker infrastructure (e.g., network I/O, request queuing, and response handling). When ModelLatency is high and OverheadLatency is low, the bottleneck is clearly the model inference time itself, not the infrastructure overhead. Therefore, optimizing the model (e.g., quantization, pruning) or upgrading to a faster instance type (e.g., GPU vs. CPU) is the correct remediation.

Answer

The SageMaker overhead is causing the delay; check endpoint configuration

Answer

The endpoint is overloaded; increase the number of instances

Answer

The network latency is high; move the endpoint closer to clients

Question 12

An organization uses SageMaker Studio and needs to restrict Studio's internet access while allowing users to install custom packages from a private PyPI mirror hosted in a VPC. Which networking configuration should they use?

Accepted Answer

Disable internet access for Studio and configure VPC-only mode, then use a VPC endpoint to the private PyPI mirror. To restrict Studio's internet access while allowing access to a private PyPI mirror in a VPC, you must disable internet access for Studio and configure VPC-only mode, then create a VPC endpoint (e.g., an interface endpoint or a private link) to the mirror. This keeps all traffic within the VPC and avoids public internet exposure. The VPC endpoint provides private connectivity to the mirror without a NAT gateway or internet gateway.

Answer

Use a NAT gateway to allow outbound traffic to the private PyPI mirror

Answer

Disable internet access for Studio and rely on SageMaker's default VPC configuration

Answer

Enable internet access for Studio and use a VPC endpoint to the private PyPI mirror

Question 13

A company uses SageMaker Inference Recommender to select the optimal endpoint configuration. After running the recommender, they receive a recommendation for a specific instance type and initial instance count. What should they do next to optimize costs over time?

Accepted Answer

Set up auto-scaling with a target tracking policy based on the recommended metric. Inference Recommender provides an initial right-sized instance type and count based on load testing, but traffic varies over time. Setting up auto-scaling with a target tracking policy (e.g., on InvocationsPerInstance or CPU utilization) lets the endpoint scale in during low traffic and out during peaks, optimizing cost continuously rather than paying for a static over-provisioned fleet.

Answer

Use the recommended configuration without changes, as it is already optimal

Answer

Purchase a Savings Plan for the recommended instance type to reduce hourly cost

Answer

Manually adjust the instance count daily based on observed traffic

Question 14

A data science team wants to automate the retraining of a model when data drift is detected. Which TWO AWS services should they use in combination to achieve this? (Choose TWO)

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor (C) is the correct service for detecting data drift: it continuously monitors a deployed model's endpoint, compares incoming inference data against a baseline, and can emit CloudWatch metrics/alerts when drift (or data quality, bias, or feature attribution issues) is detected. AWS Lambda (D) is the correct companion service because it can be triggered by those CloudWatch alarms/events to run the retraining logic — for example, invoking a SageMaker training job or pipeline to retrain and redeploy the model automatically. Together they form the detect-then-retrain automation loop the team needs. AWS Cloud9 (A) is only a cloud IDE and provides no monitoring or automation capability. Amazon DynamoDB (B) is a NoSQL database and does not detect drift or orchestrate retraining. Amazon Kinesis Data Analytics (E) is for real-time stream processing with SQL/Flink, not for model drift detection or triggering retraining workflows.

Answer

AWS Cloud9

Answer

Amazon DynamoDB

Answer

Amazon Kinesis Data Analytics

Question 15

A machine learning engineer wants to monitor a deployed model for data drift. Which SageMaker feature should they use to automatically detect drift in the input data distribution compared to the training data baseline?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor can be configured to run monitoring jobs that compare live inference data against a baseline created from training data to detect data drift.

Answer

SageMaker Pipelines

Answer

SageMaker Debugger

Answer

SageMaker Clarify