PMLE Monitoring ML Solutions Practice Question

An ML engineer has set up Vertex AI Model Monitoring on an endpoint with a sampling rate of 0.1 (10%). They notice that the monitoring job runs hourly but the reported drift metrics seem inconsistent. What is the most likely cause?

⚠ Common exam trap

The trap is assuming the monitoring configuration itself (frequency or metric choice) is broken, when the real issue is statistical — candidates overlook that sampling rate directly controls the sample size feeding drift calculations.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

The sampling rate is too low, leading to insufficient data for reliable drift statistics.

A sampling rate of 0.1 means only 10% of prediction requests are logged and analyzed, which can produce statistically noisy or inconsistent drift metrics, especially for low-traffic endpoints. Drift detection relies on sufficient sample size to compute reliable distribution comparisons (e.g., JS divergence), so a low sampling rate is the most likely cause of the inconsistency.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
The sampling rate is too low, leading to insufficient data for reliable drift statistics.
Why this is correct
A 0.1 sampling rate means only 10% of prediction requests feed the drift calculation, so hourly windows may contain too few samples for statistically reliable distribution comparisons. Raising the sampling rate satisfies the need for sufficient data volume behind each reported drift metric.
✗
Prediction drift monitoring is not enabled; only feature drift is configured.
Why it's wrong here
Prediction drift being disabled would not make feature drift metrics inconsistent; it would simply omit prediction outputs. Vertex AI reports feature drift independently, so the inconsistency stems from the sampling rate producing too few requests per hourly window for stable statistics, not from a missing monitoring type.
✗
The drift detection algorithm is not suited for this model; try changing from JS divergence to L-infinity distance.
Why it's wrong here
JS divergence and L-infinity distance both compute drift from the sampled data; switching algorithms cannot fix metrics that fluctuate because the sample itself is too small. Algorithm choice matters when distributions have specific shapes, but here the sampling rate, not the distance metric, drives the inconsistency.
✗
The monitoring frequency is too low; it should be set to every 5 minutes.
Why it's wrong here
Running every five minutes reduces the window further, so each job samples even fewer requests and drift becomes noisier, not steadier. Frequency controls how often drift is evaluated; the sampling rate controls how much data each evaluation uses, and 10% of a short interval is the actual cause.

Go deeper

PMLE Monitoring ML Solutions Practice Question

Why each option matters

Option-by-option breakdown

Related to this question