mediumMultiple Choice
MLA-C01 Practice Question: A machine learning model is deployed on SageMaker…
A machine learning model is deployed on SageMaker and its predictions are used in a production application. The model's accuracy has degraded over time. What is the most likely cause?
⚠ Common exam trap
Test-takers frequently confuse performance degradation due to resource constraints (e.g., instance size) with accuracy degradation caused by data distribution shifts, which is a core concept in ML monitoring.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The model experienced concept drift.
Concept drift occurs when the statistical properties of the target variable change over time, causing the model's predictions to become less accurate. In production ML systems on SageMaker, this is a common issue as real-world data distributions evolve, and the model does not automatically adapt without retraining.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The training data was not shuffled properly.
Why it's wrong here
Shuffling affects training convergence and generalisation at build time, but accuracy degrading after deployment points to distribution drift between training and live data. It is tempting because poor shuffling genuinely harms model quality, so it would be correct if accuracy had been poor from the outset rather than decaying over time.
- ✗
The model was not compiled for inference.
Why it's wrong here
Compilation for inference affects runtime and hardware targeting, not predictive quality; an uncompiled model would fail to deploy or run slowly, not drift. It is tempting because SageMaker compilation optimises inference latency and cost, so it is the right answer when the question asks about deployment performance rather than accuracy decay.
- ✓
The model experienced concept drift.
Why this is correct
Concept drift occurs when the statistical relationship between input features and target labels changes in production, so the model's learned mapping no longer matches reality. This degrades accuracy over time even though the model and code are unchanged.
- ✗
The endpoint instance type is too small.
Why it's wrong here
An undersized instance type causes latency, throttling or timeouts, not gradual accuracy loss; predictions remain as accurate as the model allows. Accuracy degradation over time points to data drift, where production input distributions diverge from the training data the model learned from.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLA-C01 question from scratch — 665 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.