PMLE Monitoring ML Solutions Practice Question

An ML engineer wants to monitor the performance of a Vertex AI Endpoint. Which TWO metrics are available in Cloud Monitoring for Vertex AI Endpoints? (Choose 2)

⚠ Common exam trap

Watch out — candidates often confuse model-quality metrics (accuracy, skew, SHAP) with operational serving metrics (error count, latency); candidates often assume that because Vertex AI offers Model Monitoring, those quality metrics are automatically available in Cloud Monitoring for every endpoint.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Error count

Vertex AI Endpoints automatically publish request-level metrics to Cloud Monitoring, including aiplatform.googleapis.com/endpoint/error_count, which tracks failed prediction requests and is essential for detecting serving problems. Option E (Prediction latency (p50, p95, p99)) is correct because Vertex AI Endpoints expose latency metrics such as aiplatform.googleapis.com/endpoint/prediction_latencies, which Cloud Monitoring reports as percentiles (p50, p95, p99) to characterize response-time distribution. Option A (Model accuracy) is not a built-in Cloud Monitoring metric for Endpoints; accuracy must be computed separately, for example via Vertex AI Model Monitoring or custom evaluation jobs. Option C (Feature skew score) belongs to Vertex AI Model Monitoring's skew/drift detection outputs, not to the standard Endpoint metrics in Cloud Monitoring. Option D (SHAP values) are explainability artifacts produced by Vertex Explainable AI, not time-series metrics available for an Endpoint in Cloud Monitoring.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Model accuracy
Why it's wrong here
Model accuracy is not emitted by Vertex AI Endpoints into Cloud Monitoring; the platform cannot compute it without ground-truth labels. It is tempting because accuracy is the natural quality measure, but it belongs to Model Monitoring's skew/drift or custom label-based evaluation jobs, not the Endpoint metric surface.
✓
Error count
Why this is correct
Error count is exported automatically to Cloud Monitoring for every Vertex AI Endpoint, aggregated per deployed model and response code. It satisfies the stem's monitoring requirement by surfacing failed prediction requests, letting the engineer alert on serving errors without configuring custom logging or additional instrumentation.
✗
Feature skew score
Why it's wrong here
Feature skew score belongs to Vertex AI Model Monitoring's training-serving skew detection, not to the Cloud Monitoring metric set exposed for Endpoints. It is tempting because skew monitoring genuinely tracks feature distribution drift, but that requires configuring a Model Monitor job against a deployed model rather than reading Endpoint metrics.
✗
SHAP values
Why it's wrong here
SHAP values are produced by Vertex AI Explainable AI on a model's predictions, not published as Cloud Monitoring metrics for Endpoints. They are tempting because explainability genuinely quantifies per-feature prediction attribution, but that output is retrieved through the explain endpoint, not the monitoring metric pipeline.
✓
Prediction latency (p50, p95, p99)
Why this is correct
Vertex AI Endpoints publish prediction latency percentiles to Cloud Monitoring, capturing the distribution of request response times rather than only the mean. This satisfies the stem's monitoring requirement, exposing tail latency that average measurements hide and directly affecting user experience.

Go deeper

PMLE Monitoring ML Solutions Practice Question

Why each option matters

Option-by-option breakdown

Related to this question