easyMultiple Choice
PMLE Practice Question: A data science team deploys a regression model to…
A data science team deploys a regression model to predict house prices. After one month, the mean absolute error (MAE) on the serving data increases by 20% compared to the test set. Which monitoring strategy should the team implement first to diagnose the issue?
⚠ Common exam trap
Google Cloud often tests the misconception that the first step in diagnosing model degradation is to check for data drift (Option D), when in fact the correct first step is to confirm and quantify the performance drop itself using serving-time metrics like sliding-window MAE.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Monitor prediction residuals and compute serving-time MAE over sliding windows.
The first step in diagnosing a 20% MAE increase on serving data is to monitor prediction residuals over sliding windows. This directly tracks how model errors evolve in production, allowing the team to detect whether performance degradation is sudden or gradual, and to correlate it with specific time windows or data slices. Computing serving-time MAE on sliding windows provides an immediate, interpretable signal of model health without assuming the root cause.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Retrain the model daily with the latest data to adapt to changing patterns.
Why it's wrong here
Retraining daily treats the symptom without diagnosing it, and could bake drifted or corrupted inputs into the model, masking the root cause. Retraining suits confirmed, sustained distribution shift after drift analysis has identified which features changed.
- ✓
Monitor prediction residuals and compute serving-time MAE over sliding windows.
Why this is correct
Monitoring residuals and computing serving-time MAE over sliding windows directly quantifies the 20% degradation against the test-set baseline, exposing whether error drifts gradually or spikes. Residual distributions also reveal bias or variance shifts in predictions, pinpointing covariate or concept drift as the root cause before invoking retraining or feature audits.
- ✗
Compare the distribution of training labels with serving labels using a two-sample t-test.
Why it's wrong here
Serving labels are typically unavailable at prediction time, so comparing training and serving label distributions is not feasible for immediate diagnosis. This comparison suits offline evaluation or delayed-label audits, not first-line monitoring of a live regression endpoint.
- ✗
Monitor input feature distributions for drift using the Kolmogorov-Smirnov test.
Why it's wrong here
Monitoring input feature distributions with the Kolmogorov-Smirnov test detects covariate drift, but the stem asks for the first diagnostic step after MAE rose; label or prediction drift must also be checked to locate the cause. KS testing suits ongoing feature-drift surveillance once the failure mode is understood.
Go deeper
Related to this question
About these practice questions
Courseiva writes every PMLE question from scratch — 775 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PMLE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PMLE exam.