PMLE Monitoring ML Solutions Practice Question

You are configuring Vertex AI Model Monitoring for a deployed model on a Vertex AI Endpoint. The model uses a mix of numerical and categorical features. You want to ensure that the monitoring job effectively detects drift while minimizing false alerts. Which two actions should you take? (Choose two.)

⚠ Common exam trap

The trap here is to use a one-size-fits-all approach with default thresholds or to monitor all features indiscriminately, which can lead to alert fatigue or missed critical drifts.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Configure separate drift thresholds for each feature based on its historical variability and importance to the model.

Aligning monitoring frequency with the data's rate of change and setting feature-specific thresholds based on variability and importance are key to effective drift detection. These actions reduce false alerts while ensuring critical drifts are caught, balancing sensitivity and operational efficiency.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Set the sampling rate to 1.0 to analyze every prediction request for drift.
Why it's wrong here
A sampling rate of 1.0 analyzes all requests, which can be costly and may not be necessary if the data distribution is stable. Higher sampling can increase detection sensitivity but also costs. A balanced sampling rate, such as 0.1 or 0.2, often suffices for drift detection while managing costs. Sampling rate should be chosen based on traffic volume and drift detection needs.
✗
Enable monitoring for all features, including those with low importance, to ensure comprehensive coverage.
Why it's wrong here
Monitoring all features, especially low-importance ones, can increase false alerts and monitoring overhead. Low-importance features may drift without affecting model performance, leading to unnecessary investigations. It is better to prioritize features based on their impact on predictions, focusing monitoring efforts where they matter most.
✓
Configure separate drift thresholds for each feature based on its historical variability and importance to the model.
Why this is correct
Using feature-specific thresholds accounts for differences in natural variability and importance. A feature like age might fluctuate more than a binary flag, so a single global threshold could cause false alerts for age or miss drift in the flag. Tailoring thresholds improves detection accuracy and reduces noise, making alerts more meaningful.
✓
Set the monitoring frequency to be aligned with the expected rate of change in the underlying data distribution.
Why this is correct
Aligning the monitoring frequency with the expected rate of change ensures that drift is detected in a timely manner without excessive monitoring costs or false alerts from natural fluctuations. For example, if data changes slowly, a daily frequency may suffice; if rapidly, hourly might be needed. This setting directly impacts the relevance and actionability of alerts.
✗
Use the default drift threshold provided by Vertex AI Model Monitoring for all features to simplify configuration.
Why it's wrong here
Default thresholds may not suit all features, as they do not account for varying variability and importance. Using defaults could lead to false alerts for naturally volatile features or missed drifts for stable but critical features. Customizing thresholds per feature is more effective for accurate drift detection.

Go deeper

PMLE Monitoring ML Solutions Practice Question

Why each option matters

Option-by-option breakdown

Related to this question