MLA-C01 Practice Question: A team uses SageMaker Model Monitor to track data…

A team uses SageMaker Model Monitor to track data quality. They notice that the monitor's constraint violations are increasing but the model performance remains good. What should they do?

⚠ Common exam trap

Candidates often assume increasing constraint violations always mean the model is failing, leading them to choose retraining (Option C) or threshold relaxation (Option B), when the correct first step is to investigate the specific features to distinguish benign drift from harmful drift.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Investigate the specific features that are violating constraints to see if they are still relevant.

Increasing constraint violations in SageMaker Model Monitor do not necessarily indicate model degradation; they may reflect benign data drift where feature distributions shift but the model's predictive performance remains intact. Investigating specific violating features allows the team to determine whether the drift is meaningful (e.g., due to a real-world change that the model should adapt to) or irrelevant (e.g., a feature that is no longer used in the inference pipeline). This aligns with the monitoring best practice of separating data quality alerts from model performance metrics to avoid unnecessary retraining or threshold tuning.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Disable the monitor because it is not affecting performance.
Why it's wrong here
Disabling the monitor discards the drift signal rather than investigating it, so emerging data quality issues go undetected until they degrade predictions. It is tempting because current performance looks fine, but Model Monitor exists precisely to catch input distribution shifts before they affect outcomes; disabling it would only suit a decommissioned model.
✗
Relax the constraint thresholds to reduce alerts.
Why it's wrong here
Relaxing constraint thresholds would mask the increasing violations without addressing the underlying data drift, which could eventually degrade model performance. This is tempting because threshold tuning is a standard method to reduce noise from benign fluctuations in production data. It would be correct only if the violations were false positives caused by overly sensitive static rules, not genuine drift.
✗
Retrain the model using the latest data.
Why it's wrong here
Retraining addresses model parameters, not the input data distribution that Model Monitor flagged, so the constraint violations would persist. It is tempting because retraining is the usual remedy for degraded performance, and would be correct once drift has actually harmed accuracy, but here the task is investigating the data quality violation itself.
✓
Investigate the specific features that are violating constraints to see if they are still relevant.
Why this is correct
Investigating the specific violating features is right because Model Monitor's data quality constraints compare live traffic against the training baseline's statistical properties, not model accuracy. A drifting feature may be irrelevant to predictions, so examining which features breach constraints distinguishes harmless drift from genuine data issues, satisfying the need to act despite stable performance.

Go deeper

MLA-C01 Practice Question: A team uses SageMaker Model Monitor to track data…

Why each option matters

Option-by-option breakdown

Related to this question