A data science team deploys a regression model to Amazon SageMaker for real-time inference. After one month, the model's prediction errors increase significantly, but data distributions remain unchanged. Which monitoring approach is MOST suitable for detecting this issue?
Because distributions are unchanged, the issue is performance degradation rather than data drift. Model Monitor comparing predictions against ground truth labels detects declining accuracy, satisfying the need to catch error increases when input distributions remain stable.
Why this answer
Amazon SageMaker Model Monitor can be configured to track model performance metrics (e.g., regression error metrics like RMSE or MAE) against ground truth labels as they arrive. Since the question states that data distributions remain unchanged but prediction errors increase, the issue is likely model degradation (e.g., concept drift or model staleness) rather than data drift. Monitoring ground truth labels directly captures this performance degradation, making option A the most suitable approach.
Exam trap
The trap here is that candidates often confuse data drift (changes in input features) with concept drift (changes in the relationship between features and target), and mistakenly choose data drift monitoring (option D) even though the question explicitly states data distributions are unchanged, while the correct approach is to monitor ground truth performance metrics (option A).
How to eliminate wrong answers
Option B is wrong because Amazon SageMaker Clarify is designed for detecting bias and explaining model predictions, not for monitoring model performance degradation over time; it focuses on feature attribution drift, which is a form of explainability monitoring, not a direct measure of prediction error increase. Option C is wrong because Amazon CloudWatch monitoring of endpoint latency tracks infrastructure performance (e.g., response times, invocation counts), not the accuracy or error rate of model predictions; latency issues do not explain increased prediction errors when data distributions are unchanged. Option D is wrong because Amazon SageMaker Model Monitor configured for data drift tracks changes in the input feature distribution, but the question explicitly states that data distributions remain unchanged, so data drift monitoring would not detect the issue; the problem is model performance degradation despite stable input data.