mediumMultiple Choice
MLA-C01 Practice Question: A gaming company uses a SageMaker endpoint for…
A gaming company uses a SageMaker endpoint for real-time player churn prediction. The model is updated weekly. After a recent retraining, the team notices that the endpoint's predicted probabilities for churn have shifted dramatically: the average predicted probability dropped from 0.3 to 0.05. The team suspects concept drift (the relationship between features and target changed) rather than data drift. They have SageMaker Model Monitor set up for data drift and quality metrics, but not for bias or explainability. The team needs to confirm concept drift and take corrective action. Which approach should the team take FIRST?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Configure SageMaker Model Monitor's model quality monitoring to compare predictions against actual outcomes collected from a week of production traffic
To detect concept drift, the team needs to compare the model's predictions against actual observed outcomes (ground truth). SageMaker Model Monitor's quality monitoring can track prediction accuracy over time if ground truth is provided. Option A (Configure SageMaker Model Monitor's model quality monitoring) is the correct first step. Option B (retrain with more recent data) might help but does not confirm drift. Option D (investigate data drift by reviewing feature distribution) checks feature distribution, not concept drift. Option C (use Clarify for SHAP values) is for feature importance, not drift detection.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Configure SageMaker Model Monitor's model quality monitoring to compare predictions against actual outcomes collected from a week of production traffic
Why this is correct
Model quality monitoring compares live predictions with ground-truth outcomes, directly measuring whether the feature-target relationship has shifted. This confirms concept drift rather than merely detecting input distribution changes, which data drift monitoring already covers. It is the prerequisite step before retraining or recalibration.
- ✗
Immediately retrain the model using the most recent month of data and redeploy to the endpoint
Why it's wrong here
Retraining on one month of data cannot confirm concept drift; it masks the symptom and risks overfitting to a short window. Retraining is the corrective action taken after drift is verified. First the team must measure whether the feature-to-target relationship actually changed.
- ✗
Use Amazon SageMaker Clarify to compute SHAP values and understand which features are driving the new predictions
Why it's wrong here
SHAP values explain individual predictions, not whether the feature-target relationship drifted over time. Clarify suits debugging model behaviour or bias auditing. Confirming concept drift requires comparing recent label outcomes against baseline performance, which SHAP cannot provide.
- ✗
Investigate data drift by reviewing the Model Monitor feature distribution constraints and comparing recent input data to the baseline
Why it's wrong here
Model Monitor already tracks data drift, and the team explicitly suspects concept drift rather than data drift. Reviewing feature distribution constraints re-checks the input distribution, which is unchanged. Concept drift needs label-based performance comparison, so this investigation addresses the wrong axis.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLA-C01 question from scratch — 665 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.