PMLE Practice Exam 51 — 20 Questions

Question 1

A team wants to monitor prediction drift on a Vertex AI Endpoint for a classification model. They have configured Vertex AI Model Monitoring with default settings. Which metric will be used to detect prediction drift?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Jensen-Shannon divergence. Vertex AI Model Monitoring uses Jensen-Shannon divergence as the default algorithm for prediction drift detection. It compares the distribution of predicted classes over time against a reference distribution (e.g., training predictions).

Answer

Kullback-Leibler divergence

Answer

L-infinity distance

Answer

Population Stability Index (PSI)

Question 2

An organization runs a Vertex AI pipeline that includes a model evaluation step. Team members want to reuse previously computed evaluation metrics when re-running the pipeline with unchanged code and hyperparameters. Which feature should they enable?

Accepted Answer

Enable pipeline caching (default behavior). Vertex AI Pipelines automatically caches component outputs based on a cache key derived from the component image, code, and input parameters. If the cache key matches a previous run, the cached output is reused, saving time and cost.

Answer

Manually store outputs in Cloud Storage and check for existence

Answer

Use the importer component to fetch previous results

Answer

Disable caching for the evaluation component

Question 3

A company has deployed a model to Vertex AI Endpoints and wants to monitor for feature drift using Jensen-Shannon divergence. They have set a threshold of 0.1. After one week, the monitoring job reports a divergence of 0.15 for a feature. What should the engineer do next to diagnose which features are contributing to the drift?

Accepted Answer

Use Vertex AI Explainability to compute feature attributions and identify drifted features. To identify which features are drifting, the engineer can use Vertex AI Explainability to compute feature attributions (e.g., SHAP values) and correlate them with drift metrics.

Answer

Deploy a new model version immediately

Answer

Check the model's confusion matrix in BigQuery

Answer

Increase the sampling rate to capture more data

Question 4

A team is using Vertex AI Model Monitoring and wants to set up automated retraining when drift is detected. Which THREE services are needed to implement this pipeline? (Choose three.)

Accepted Answer

Cloud Functions. The typical pipeline: Cloud Monitoring alert on drift → Pub/Sub message → Cloud Function or Cloud Run → triggers Vertex AI Pipeline for retraining. Optionally, Cloud Scheduler is not needed as it's event-driven.

Answer

Cloud Scheduler

Answer

Vertex AI Model Monitoring

Question 5

A team is monitoring a model and observes that the error rate (prediction failures) has increased. They have enabled request/response logging on the Vertex AI Endpoint. How can they set up a metric and alert for prediction error rate?

Accepted Answer

Create a log-based metric in Cloud Logging for error logs and set up an alert in Cloud Monitoring. Vertex AI Endpoint logs contain information about failed predictions. You can create a log-based metric in Cloud Logging that counts error logs, and then create an alert in Cloud Monitoring based on that metric.

Answer

Configure Cloud Monitoring to pull error rate from Cloud Endpoints

Answer

Use Vertex AI Model Monitoring to monitor error rate directly

Answer

Enable Vertex AI Pipelines to track errors

Question 6

An MLOps engineer needs to collect ground truth labels for a deployed classification model to compare predictions against actuals. Where should the engineer store the ground truth data to enable Vertex AI model quality monitoring?

Accepted Answer

BigQuery. Vertex AI Model Monitoring expects ground truth data to be uploaded to BigQuery tables, which can then be used to compute confusion matrices and other quality metrics over time.

Answer

Firestore

Answer

Cloud Spanner

Answer

Cloud Storage

Question 7

An ML team wants to automatically retrain a model when data drift is detected. They have set up a Cloud Monitoring alert on drift. What service should they use to trigger a retraining pipeline in response to the alert?

Accepted Answer

Cloud Functions. Cloud Monitoring alerts can send notifications to Pub/Sub topics. A Pub/Sub message can then trigger a Cloud Function or Cloud Run service that starts a Vertex AI Pipeline for retraining.

Answer

Cloud Scheduler

Answer

Vertex AI Feature Store

Answer

Vertex AI Model Monitoring

Question 8

A team develops a pipeline that trains a model and evaluates it. They want to pass the test accuracy (a float) from the evaluation component to a subsequent deployment component. Which KFP SDK type should the evaluation component output be annotated with?

Accepted Answer

Output[float]. For simple numeric values like floats, the appropriate output type is a pipeline parameter (e.g., float). KFP artifacts are used for larger data like models or datasets, not for scalar metrics.

Answer

Output[Metrics]

Answer

Output[Artifact]

Answer

Output[ClassificationMetrics]

Question 9

A team is monitoring a model on Vertex AI Endpoints and wants to track the p99 latency of online predictions. Which approach should they use to set up latency monitoring and alerting?

Accepted Answer

Use Cloud Logging to create log-based metrics from prediction logs and set up alerts in Cloud Monitoring. Vertex AI Endpoints automatically export request/response logs to Cloud Logging, which can be used to create log-based metrics for latency percentiles. These metrics can then be visualized in Cloud Monitoring dashboards and used for alerting.

Answer

Enable Vertex AI Model Monitoring and select 'latency' as a metric

Answer

Enable Vertex AI Explainable AI to output latency statistics

Answer

Configure Cloud Monitoring to scrape Prometheus metrics from the endpoint

Question 10

You are building a Vertex AI pipeline using the KFP SDK v2. One component processes a large dataset and outputs a metrics artifact. You notice that the component is being cached even when the dataset changes, because the component code and image remain the same. How can you force the component to always re-execute when the dataset changes?

Accepted Answer

Set caching_strategy.max_cache_staleness = "0s" on the component.. In KFP SDK v2, caching is keyed on the component code, image digest, and all input values. If the dataset is passed as a URI string, changing the URI will invalidate the cache. If the dataset changes without a URI change, you can disable caching per component by setting caching_strategy.max_cache_staleness to 0 or using the disable_cache method.

Answer

Use the dsl.CacheKey annotation to explicitly set cache keys.

Answer

Change the component's image tag to :latest.

Answer

Add a random integer parameter to the component to vary the inputs.

Question 11

A data scientist notices that the model's prediction latency has increased over the last week. They need to investigate the root cause by examining request and response logs for the Vertex AI Endpoint. What is the recommended way to capture these logs?

Accepted Answer

Configure request/response logging by specifying a BigQuery destination in the endpoint deployment. Vertex AI Endpoint can be configured to log request/response data to BigQuery via a log sink. This data can then be analyzed to understand latency issues.

Answer

Enable Vertex AI Model Monitoring with sampling rate 100%

Answer

Export logs to Cloud Storage using a sink and use gcloud logging read

Answer

Use Cloud Monitoring custom metrics to capture latency per request

Question 12

An ML team is using Population Stability Index (PSI) to monitor feature drift on a Vertex AI Endpoint. The PSI value for a feature is 0.25, which exceeds the alert threshold of 0.2. The feature has high SHAP importance. The team wants to automatically retrain the model. What is the correct end-to-end setup?

Accepted Answer

Cloud Monitoring alert → Pub/Sub → Cloud Functions → Vertex AI Pipeline (with training and deployment steps). The correct setup involves Cloud Monitoring alert (based on PSI metric) → Pub/Sub → Cloud Function → Vertex AI Pipeline to trigger retraining with updated data.

Answer

Vertex AI Model Monitoring alert → Cloud Scheduler → Vertex AI Training

Answer

Cloud Functions → Pub/Sub → Cloud Monitoring → Vertex AI Pipeline

Answer

Cloud Monitoring alert → Cloud Logging → Cloud Functions → Vertex AI Training

Question 13

A company wants to monitor fairness of a model by evaluating performance metrics across demographic subgroups. They have ground truth labels stored in BigQuery. Which Vertex AI service should they use?

Accepted Answer

Vertex AI Model Evaluation. Vertex AI Model Evaluation supports sliced evaluation, allowing you to compute metrics per subgroup (e.g., by demographic) using data in BigQuery.

Answer

Vertex AI Model Monitoring

Answer

Vertex AI Prediction

Answer

Vertex AI Explainability

Question 14

A company wants to be alerted when the prediction error rate on their Vertex AI Endpoint exceeds 5% in any 5-minute window. What is the best way to set up this alert?

Accepted Answer

Create a Cloud Monitoring alert on the metric 'prediction/error_count' with a threshold of 5% of total predictions. Cloud Monitoring can create an alerting policy based on the 'Prediction error count' metric with a rolling 5-minute window and a threshold of 5% (or absolute count).

Answer

Set up a scheduled Cloud Function to query logs and check error rate

Answer

Enable BigQuery log sink and create a scheduled query alert

Answer

Use Vertex AI Model Monitoring to detect prediction drift

Question 15

Which TWO are best practices when deploying AutoML models to production?

Accepted Answer

Monitor for data drift. Monitoring for data drift (Option A) is a best practice because production models can degrade over time as the statistical properties of input data change. Vertex AI provides a Model Monitoring service that automatically detects skew and drift by comparing serving data distribution against training data distribution, triggering alerts when anomaly thresholds are breached. This ensures model reliability and performance in production.

Answer

Train the model on a disk to reduce latency

Answer

Deploy on sole-tenant nodes

Answer

Use TPUs for model serving

Question 16

A company is experiencing high prediction costs on Vertex AI Endpoints. They want to monitor and optimize costs. Which THREE actions should they take? (Choose 3)

Accepted Answer

Use Cloud Billing reports to track Vertex AI endpoint costs per hour and per request. Cost monitoring involves tracking per-hour and per-request costs, setting budget alerts, and possibly adjusting scaling to reduce unnecessary compute.

Answer

Use Vertex AI Explainability on every prediction

Answer

Enable Vertex AI Model Monitoring to track prediction latency

Question 17

A team is using Vertex AI Explainability with a deployed model. They need to generate explanations for image classification predictions. Which explanation method should they configure in the ExplanationSpec?

Accepted Answer

XRAI. XRAI (eXplanation with Ranked Area Integrals) is specifically designed for image models to highlight regions that contribute to the prediction.

Answer

SHAP with KernelExplainer

Answer

Sampled Shapley

Answer

Integrated Gradients

Question 18

An ML engineer wants to monitor the latency of online predictions from a Vertex AI Endpoint. They need to track p50, p95, and p99 latency over time and set up alerts if p99 exceeds 1 second. Which approach should they take?

Accepted Answer

Configure a Cloud Monitoring dashboard using the built-in Vertex AI Endpoint metrics for latency. Cloud Monitoring can ingest metrics from Vertex AI Endpoints, including prediction latency distributions (p50, p95, p99). Alerts can be configured on these metrics.

Answer

Use Cloud Monitoring custom metrics and publish latency percentiles from the application code

Answer

Enable request/response logging to BigQuery and calculate latency percentiles using SQL queries

Answer

Use Vertex AI Model Monitoring to track prediction latency

Question 19

A data science team uses Vertex AI Model Monitoring to detect data quality issues in a production model. Which TWO metrics should they enable to identify problems with missing values in predictions? (Select TWO.)

Accepted Answer

Feature value distribution skew (distance metrics).. Vertex AI Model Monitoring's feature value distribution skew detection uses distance metrics (e.g., Jenson-Shannon divergence, L-infinity) to compare the distribution of feature values in the serving data against the training data. A sudden increase in missing values in a feature will shift its distribution, triggering a skew alert. This allows the team to detect missing value problems indirectly by monitoring distributional drift.

Answer

Training-serving skew detection for all features.

Answer

Total count of missing values across all features.

Answer

Prediction confidence score.

Question 20

A company wants to use Vertex AI for hyperparameter tuning. Which three components are required to configure a hyperparameter tuning job? (Choose THREE.)

Accepted Answer

Algorithm (e.g., Bayesian, grid, random). Vertex AI hyperparameter tuning requires specifying the search algorithm (Bayesian, grid, or random) to determine how the hyperparameter space is explored. Bayesian optimization is the default and most efficient for continuous spaces, while grid search is exhaustive and random search is simple. Without an algorithm, Vertex AI cannot decide how to sample trials.

Answer

Machine type for each trial

Answer

Training container image