PMLE Practice Test 33 — 15 Questions

Question 1

A team uses Vertex AI Experiments to track ML training runs. They want to automatically trigger a retraining pipeline when new labeled data arrives in BigQuery, and ensure the pipeline uses only approved libraries from a central artifact registry. Which combination of services should they use?

Accepted Answer

Vertex AI Pipelines with a scheduled trigger, and use Cloud Build to pull libraries from Artifact Registry.. Option B is correct because Vertex AI Pipelines provides a managed orchestration service for ML workflows, and a scheduled trigger can be set to run the pipeline when new labeled data arrives in BigQuery (e.g., via a Cloud Scheduler or Eventarc trigger). Cloud Build is used to pull approved libraries from Artifact Registry, ensuring only vetted dependencies are used during pipeline execution, which meets the security and compliance requirement.

Answer

Cloud Composer to orchestrate, with Cloud Storage for libraries.

Answer

Cloud Functions triggered by BigQuery, Cloud Build to run training, and Artifact Registry for libraries.

Answer

Vertex AI Experiments with continuous evaluation, and a Cloud Run job for training.

Answer

Dataflow to preprocess, then trigger a Cloud Run job.

Question 2

Refer to the exhibit. A team leader applies this IAM policy on a Vertex AI model resource. What does the condition accomplish?

Accepted Answer

Limits access to models whose resource name starts with 'dev-'. The condition in the IAM policy uses the `resource.name.startsWith('dev-')` condition expression, which restricts access to Vertex AI model resources whose resource name begins with the prefix 'dev-'. This is a common pattern for environment-based access control, allowing the data scientist to only interact with models designated for development.

Answer

Allows the data scientist to access model evaluations only

Answer

Limits access to models owned by the data scientist

Answer

Limits access to models only in the us-central1 region

Answer

Limits access to models created after a certain date

Question 3

A company uses a custom container on Vertex AI Prediction. They want to send custom metrics from their prediction container to Cloud Monitoring. Which method should they use?

Accepted Answer

OpenCensus or OpenTelemetry SDK. Option A is correct because OpenCensus and OpenTelemetry are the recommended open-source frameworks for exporting custom metrics from custom containers on Vertex AI Prediction to Cloud Monitoring. They provide a standardized way to instrument your application code, collect metrics, and send them directly to Cloud Monitoring via the Cloud Monitoring API, without requiring additional agents or log-based workarounds.

Answer

Vertex AI built-in metrics

Answer

Stackdriver Monitoring agent installed in the container

Answer

Cloud Logging log-based metrics

Question 4

After setting up model monitoring on Vertex AI for a classification model, the engineer sees a high number of anomaly alerts for the "age" feature. Upon investigation, the age distribution in recent predictions is similar to training data. What might be the cause?

Accepted Answer

The monitoring threshold for age is too low. Option C is correct because the high number of anomaly alerts despite the age distribution being similar to training data indicates that the monitoring threshold for the 'age' feature is set too low. In Vertex AI Model Monitoring, anomaly detection compares recent prediction distributions against a baseline using statistical tests (e.g., the Kolmogorov-Smirnov test for numerical features). If the threshold is too sensitive, even minor, statistically insignificant deviations can trigger alerts, leading to false positives even when the distribution is essentially unchanged.

Answer

The feature importance of age has changed

Answer

The monitoring baseline was incorrectly set

Answer

The model is overfitting to age

Question 5

A company uses Vertex AI Model Monitoring to detect data drift. They have a model that predicts house prices. Which dataset should they compare against the training data to detect drift?

Accepted Answer

The latest batch of predictions. Option C is correct because Vertex AI Model Monitoring compares the training data (serving as the baseline) against the latest batch of predictions to detect data drift. This batch represents the most recent inference requests, allowing the monitoring service to compute statistical distribution differences (e.g., Jensen-Shannon divergence) and trigger alerts when drift exceeds a configured threshold. Using the latest batch ensures timely detection of shifts in the production data distribution.

Answer

The entire historical prediction data

Answer

A random sample of recent predictions

Answer

The validation data used during training

Question 6

A team deploys a model using Vertex AI and wants to monitor for concept drift. What should they track?

Accepted Answer

Changes in the relationship between inputs and outputs. Concept drift refers to a change in the underlying relationship between the input features and the target variable over time, which degrades model performance. In Vertex AI, monitoring this requires tracking the statistical relationship between inputs and outputs (e.g., via prediction residuals or model performance metrics), not just the input distribution alone. Option D correctly identifies this need, as concept drift is fundamentally about the input-output mapping shifting, even if the input distribution remains stable.

Answer

Number of prediction requests

Answer

Model prediction latency

Answer

Changes in input data distribution

Question 7

Your ML pipeline uses Vertex AI Feature Store to serve features for online predictions. You need to monitor the freshness of features in the online store. Which approach is most effective?

Accepted Answer

Create a custom metric in Cloud Monitoring that tracks the time since last feature update, and set an alert threshold.. Option C is correct because Cloud Monitoring custom metrics allow you to track the timestamp of the last feature update in Vertex AI Feature Store and set an alert threshold for staleness. This directly measures feature freshness, which is critical for online predictions where stale features can degrade model accuracy. Other options either measure unrelated metrics (entity count), are too slow (nightly batch), or focus on auditing rather than real-time monitoring.

Answer

Set up a Cloud Monitoring alert for feature store entity count.

Answer

Schedule a nightly BigQuery batch job to compare feature values.

Answer

Enable detailed audit logs in Feature Store and export to BigQuery.

Question 8

A team is monitoring a production ML system that includes multiple models and data processing pipelines. They want to set up a comprehensive alerting strategy that minimizes false positives while ensuring critical issues are promptly addressed. Which approach is the most effective?

Accepted Answer

Use AIOps with anomaly detection to dynamically adjust thresholds. Option D is correct because AIOps with anomaly detection uses machine learning to dynamically adjust alert thresholds based on real-time system behavior, reducing false positives while ensuring critical issues are detected promptly. This approach adapts to changing data distributions and traffic patterns, unlike static thresholds that require manual tuning and often miss subtle anomalies. It is the most effective strategy for complex ML production systems where multiple models and pipelines interact, as it can correlate signals across components to identify genuine incidents.

Answer

Set up alerts for all possible error conditions

Answer

Use static thresholds based on historical data

Answer

Rely on manual monitoring during business hours

Question 9

Refer to the exhibit. An alert policy is configured to trigger when prediction latency exceeds 500 ms for 5 consecutive minutes. The team is experiencing many false positive alerts during brief latency spikes. Which adjustment would most effectively reduce false positives while still detecting prolonged latency issues?

Accepted Answer

Increase the duration to 30 minutes. Increasing the duration from 5 to 30 minutes (Option C) directly addresses the problem of false positives from brief latency spikes by requiring the latency to exceed 500 ms for a longer continuous period before triggering an alert. This ensures that only sustained, prolonged latency issues—not transient spikes—activate the policy, aligning with the goal of detecting genuine degradation while ignoring noise.

Answer

Change the comparison to less than

Answer

Add a condition that CPU utilization is also high

Answer

Increase the threshold to 1000 ms

Question 10

A machine learning model deployed on Vertex AI is returning erroneous predictions. The team needs to investigate the root cause by examining the prediction request and response details. Which Google Cloud tool is best suited for this?

Accepted Answer

Cloud Logging. Cloud Logging is the correct tool because it captures detailed logs of prediction requests and responses, including input features, model outputs, and any errors. By examining these logs, the team can trace the exact data flow and identify discrepancies causing erroneous predictions, such as data preprocessing issues or model version mismatches.

Answer

Cloud Monitoring

Answer

Cloud Debugger

Answer

Cloud Trace

Question 11

An ML engineer is building a monitoring dashboard for a Vertex AI pipeline that includes training, evaluation, and batch prediction. Which THREE components should be included to provide comprehensive observability? (Select THREE.)

Accepted Answer

Pipeline execution status, duration, and failure rates for each component.. Option A is correct because pipeline execution status, duration, and failure rates are fundamental metrics for monitoring the health and performance of a Vertex AI pipeline. These metrics allow the ML engineer to quickly identify bottlenecks, track overall workflow progress, and detect failures in training, evaluation, or batch prediction steps, which is essential for comprehensive observability.

Answer

Compute engine CPU and memory logs for each pipeline step.

Answer

Online prediction latency and request count from the deployed model endpoint.

Question 12

An ML team wants to monitor their recommendation model for fairness. Which TWO metrics should they track to detect potential bias? (Select TWO.)

Accepted Answer

Pair-wise fairness metrics such as equal opportunity difference.. Pair-wise fairness metrics like equal opportunity difference directly compare model outcomes (e.g., true positive rates) across sensitive groups, making them a standard tool for detecting bias in classification tasks. This metric measures the difference in true positive rates between privileged and unprivileged groups, where a value close to zero indicates fairness. Tracking such metrics aligns with the core principle of monitoring for disparate impact in ML systems.

Answer

Recall for the minority group only.

Answer

Overall accuracy on the test set.

Answer

Average prediction confidence per request.

Question 13

Refer to the exhibit. What is the purpose of this query?

Accepted Answer

To find prediction errors in Cloud Logging. The query filters Cloud Logging entries for the string 'prediction failed', which directly indicates prediction errors logged by the ML prediction service. This is a common pattern for monitoring model inference failures in production, not for measuring drift, counting requests, or measuring latency.

Answer

To detect data drift

Answer

To count all prediction requests

Answer

To monitor model latency

Question 14

You have deployed a text classification model using Vertex AI Endpoints. The model is performing well, but the operations team wants to be alerted if the endpoint returns an excessive number of HTTP 503 errors. What is the simplest way to achieve this?

Accepted Answer

Create a Cloud Monitoring alert based on the metric 'prediction/failed_request_count' with a condition on 5xx errors.. Option B is correct because Vertex AI Endpoints automatically export the 'prediction/failed_request_count' metric to Cloud Monitoring, which includes a label for HTTP status codes. By creating an alert on this metric with a filter for 5xx errors, you can directly monitor excessive 503 responses without additional infrastructure or custom code.

Answer

Configure a Cloud Monitoring uptime check on the endpoint URL.

Answer

Add a logging statement in the custom prediction routine to count errors manually.

Answer

Export Cloud Logging to BigQuery and run a scheduled query for 503s.

Question 15

Your team manages multiple ML models on Vertex AI. You need to implement a centralized monitoring solution to track model performance over time. Which TWO approaches should you consider? (Choose two.)

Accepted Answer

Create Cloud Monitoring dashboards and alerts based on Vertex AI metrics.. Option C is correct because Cloud Monitoring provides centralized dashboards and alerting for Vertex AI metrics such as prediction latency, request count, and error rates, enabling you to track model performance over time without additional infrastructure. Option D is correct because Vertex AI Model Monitoring is purpose-built to detect training-serving skew and feature drift by comparing serving data distributions to training data, which is essential for maintaining model performance in production.

Answer

Store all prediction logs in BigQuery and analyze using SQL.

Answer

Use Cloud Source Repositories to track model code versions.

Answer

Enable Cloud Billing budgets to track cost per model.