mediumMultiple Choice
MLA-C01 Practice Question: A company uses Amazon SageMaker to train and…
A company uses Amazon SageMaker to train and deploy a machine learning model. After deployment, they notice that the model's accuracy drops significantly over time due to changes in the underlying data distribution. Which monitoring solution should they implement to detect this issue automatically?
⚠ Common exam trap
Watch out — candidates often confuse operational monitoring (latency, logs, API activity) with data quality monitoring, leading candidates to pick options that track infrastructure or performance rather than the underlying data distribution that causes model decay.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Set up Amazon SageMaker Model Monitor with data quality monitoring.
Amazon SageMaker Model Monitor with data quality monitoring is the correct solution because it automatically detects deviations in the input data distribution compared to a baseline, which directly addresses the problem of model accuracy degradation due to data drift. It continuously monitors the statistical properties of inference requests and alerts when drift is detected, enabling proactive retraining.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Set up Amazon SageMaker Model Monitor with data quality monitoring.
Why this is correct
SageMaker Model Monitor's data quality monitoring compares live inference traffic against the baseline statistics captured from the training dataset, raising alerts when feature distributions drift. This directly detects the covariate shift causing the accuracy decline, satisfying the requirement for automatic detection of data distribution changes.
- ✗
Configure AWS Config rules to check the model accuracy metric.
Why it's wrong here
AWS Config evaluates resource configuration compliance, not model prediction quality, so it cannot detect accuracy degradation from distribution shift. It is tempting because Config provides automated monitoring and alerting, and it would be correct for tracking whether SageMaker endpoint or bucket settings drift from a defined configuration baseline.
- ✗
Use AWS CloudTrail to monitor changes to the model's S3 bucket.
Why it's wrong here
CloudTrail records API activity and control-plane events, not the statistical properties of inference requests, so it cannot reveal distribution shift. It is tempting because CloudTrail is the default audit service for detecting unauthorised changes, and it would be correct if the accuracy drop stemmed from someone altering the model artefacts in S3.
- ✗
Enable Amazon CloudWatch Logs on the endpoint and set alarms on inference latency.
Why it's wrong here
CloudWatch Logs with latency alarms tracks endpoint responsiveness, not data drift; latency can stay flat while input distributions shift. It is tempting because CloudWatch underpins operational monitoring, and latency alarms would be right for detecting slow inference or throttling, but drift detection requires SageMaker Model Monitor's data quality baseline comparisons.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLA-C01 question from scratch — 665 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.