PMLE Practice Test 33 — 15 Questions

Question 1

A financial institution has deployed a fraud detection model on a Vertex AI Endpoint. The model uses both numerical and categorical features. They have enabled Vertex AI Model Monitoring with training data and configured drift thresholds. After several weeks, they notice that the model's recall has dropped significantly, but the overall prediction distribution remains stable. They suspect that a specific subgroup of transactions is being misclassified. Which approach should they use to identify the subgroup and diagnose the issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Export the serving data and predictions to BigQuery, then use slice-based evaluation to compare performance metrics across different subgroups.. Exporting serving data and predictions to BigQuery enables slice-based evaluation, which can reveal performance disparities across subgroups. Since Vertex AI Model Monitoring does not provide subgroup-level metrics, this manual analysis is necessary to identify the problematic subgroup and diagnose the recall drop.

Answer

Use Vertex AI Model Monitoring's feature attribution analysis to identify which features contribute most to the misclassifications.

Answer

Configure Vertex AI Model Monitoring to monitor the 'transaction_amount' feature with a lower threshold to detect subtle drifts that might affect recall.

Answer

Retrain the model immediately using the most recent data to improve recall across all subgroups.

Question 2

A credit-risk team runs a tabular model on a Vertex AI Endpoint. They configured Vertex AI Model Monitoring with a training dataset and skew detection using the default threshold. After a week, they receive alerts that many features have high training-serving skew, but the model's business metrics (approval rate, default rate) are unchanged. They suspect the alerts are false positives due to a recent change in an upstream data pipeline that shifted feature distributions. What should they do to reduce these false alerts while still monitoring for real skew?

Accepted Answer

Recreate the monitoring job with a new training dataset that reflects the recent pipeline change, and adjust the skew threshold based on observed variance.. Training-serving skew compares live data to the training baseline. When an upstream pipeline changes distributions legitimately, the old baseline becomes invalid and triggers false skew alerts. Updating the monitoring configuration with a representative training dataset and a threshold based on observed variance realigns detection with current production behavior. Disabling monitoring or making it more sensitive does not solve the underlying baseline mismatch.

Answer

Disable skew detection and rely only on prediction drift monitoring.

Answer

Switch from skew detection to outlier detection and set the outlier threshold to a very low value.

Answer

Increase the monitoring frequency and lower the skew threshold to capture more data points.

Question 3

Your team has deployed a tabular model to a Vertex AI Endpoint and enabled Vertex AI Model Monitoring with training-serving skew detection. You configured the monitoring job to run hourly and store statistics in a Cloud Storage bucket. After the first run, you notice that the job did not produce any drift metrics. You want to determine the cause of the missing metrics. What should you do first?

Accepted Answer

Check that the endpoint's deployed model has a reference dataset specified and that the monitoring job's input schema matches the endpoint's prediction schema.. Vertex AI Model Monitoring requires a reference dataset and a matching schema to compute training-serving skew. Without these, the job cannot generate metrics. Checking the configuration of the reference dataset and schema alignment is the logical first step to resolve missing metrics.

Answer

Increase the monitoring job's sampling rate to 1.0 to ensure that all prediction requests are analyzed.

Answer

Re-deploy the model to the endpoint with a larger machine type to handle the monitoring workload.

Answer

Verify that the Cloud Storage bucket has the correct IAM permissions for the Vertex AI service account to write objects.

Question 4

A team is using Vertex AI Model Monitoring to detect prediction drift on a deployed model. They have configured the monitoring job to run every 6 hours. After a week, they notice that the drift metric has been consistently high but no alerts have been triggered. They have verified that the alerting policy in Cloud Monitoring is correctly configured. What is the most likely cause?

Accepted Answer

The drift threshold is set higher than the observed drift metric values.. Alerts are triggered when the drift metric exceeds the configured threshold. If the threshold is set higher than the observed drift values, no alert will fire. The team should review and lower the threshold to match their tolerance for drift, ensuring alerts are generated when drift is significant.

Answer

The sampling rate is too low, causing the drift metric to be inaccurate.

Answer

The monitoring job is not writing the drift metrics to Cloud Monitoring.

Answer

The monitoring frequency is too high, causing the system to ignore the drift.

Question 5

You are using Vertex AI Model Monitoring to detect prediction drift on a deployed model that serves online predictions. You want to ensure that the monitoring job can correctly compute drift metrics for numerical features. Which two configurations are required for the monitoring job to compute drift for a numerical feature? (Choose two.)

Accepted Answer

Specify the feature's distribution as a numerical distribution and provide the training dataset as the reference.. To compute drift for numerical features, Vertex AI Model Monitoring needs a reference dataset to establish the baseline distribution and a correct input schema to parse the feature as numerical. These two configurations are essential; sampling rate, explainability, and thresholds are not required for the computation itself.

Answer

Enable explainability on the endpoint to generate feature attributions for drift analysis.

Answer

Configure the monitoring job to use a default threshold for drift detection, such as 0.3 for the Jensen-Shannon distance.

Answer

Set the monitoring job's sampling rate to at least 0.5 to ensure sufficient data for statistical tests.

Question 6

A company has deployed a model for image classification and wants to monitor for feature drift using XRAI attributions. However, they notice that the XRAI attribution maps are too large and are causing high latency in the monitoring pipeline. What is the most effective way to reduce the overhead of explainability monitoring for image models?

Accepted Answer

Reduce the sampling rate for the explainability feature. Vertex AI Explainable AI supports XRAI for image models, but generating XRAI attributions can be computationally expensive. Sampling a subset of predictions reduces the number of explanations generated, lowering latency and cost.

Answer

Disable XRAI and use integrated gradients instead

Answer

Use a smaller image size for the model

Answer

Increase the number of replicas on the endpoint

Question 7

An organization runs a Vertex AI pipeline that includes a model evaluation step. Team members want to reuse previously computed evaluation metrics when re-running the pipeline with unchanged code and hyperparameters. Which feature should they enable?

Accepted Answer

Enable pipeline caching (default behavior). Vertex AI Pipelines has caching enabled by default. When a pipeline is re-run with unchanged code, hyperparameters, and inputs, the evaluation step will reuse the cached output from the previous run, saving time and cost. Team members do not need to manually store outputs or disable caching.

Answer

Manually store outputs in Cloud Storage and check for existence

Answer

Use the importer component to fetch previous results

Answer

Disable caching for the evaluation component

Question 8

An ML engineer is configuring Vertex AI Model Monitoring for a deployed model that receives a mix of numerical and categorical features. The engineer wants to ensure that the monitoring job can detect both data drift and training-serving skew. Which two configurations are required to enable both types of detection? (Choose two.)

Accepted Answer

Enable skew detection and specify the training dataset used to train the model.. To detect both data drift and training-serving skew in Vertex AI Model Monitoring, you must enable each detection type and provide the appropriate baseline. Skew detection requires the training dataset to compare against serving data. Drift detection requires a baseline dataset, often a sample of recent serving data. These two configurations are independent and both necessary to achieve the desired monitoring.

Answer

Set the monitoring objective to 'drift' and provide a training dataset for baseline.

Answer

Set the monitoring frequency to at least once per hour to capture both types of drift.

Answer

Configure alerting via Cloud Monitoring to notify when either drift or skew exceeds thresholds.

Question 9

You have deployed a model to a Vertex AI Endpoint and enabled Vertex AI Model Monitoring with a monitoring frequency of every 24 hours. The model serves predictions with a feature called 'transaction_amount' that has a skewed distribution. After several days, you notice that the drift metrics for this feature are consistently below the threshold, but you suspect that the feature distribution has actually shifted. You want to improve the sensitivity of drift detection for this feature. What should you do?

Accepted Answer

Adjust the drift threshold for 'transaction_amount' to a lower value to trigger alerts more easily.. To increase sensitivity to drift for a specific feature, you can lower the drift threshold for that feature. This makes the monitoring job trigger alerts at smaller deviations from the reference distribution. Changing frequency or sampling rate affects data volume or timeliness, not the sensitivity of the drift metric itself.

Answer

Enable explainable AI on the endpoint to get feature attributions for 'transaction_amount'.

Answer

Change the monitoring frequency to every hour to capture more data points.

Answer

Increase the sampling rate to analyze a larger proportion of prediction requests.

Question 10

You are configuring Vertex AI Model Monitoring for a deployed model on a Vertex AI Endpoint. The model uses a mix of numerical and categorical features. You want to ensure that the monitoring job effectively detects drift while minimizing false alerts. Which two actions should you take? (Choose two.)

Accepted Answer

Configure separate drift thresholds for each feature based on its historical variability and importance to the model.. Aligning monitoring frequency with the data's rate of change and setting feature-specific thresholds based on variability and importance are key to effective drift detection. These actions reduce false alerts while ensuring critical drifts are caught, balancing sensitivity and operational efficiency.

Answer

Set the sampling rate to 1.0 to analyze every prediction request for drift.

Answer

Enable monitoring for all features, including those with low importance, to ensure comprehensive coverage.

Answer

Use the default drift threshold provided by Vertex AI Model Monitoring for all features to simplify configuration.

Question 11

A media company uses a Vertex AI Endpoint to serve a video recommendation model. They have enabled Vertex AI Model Monitoring for prediction drift. After a major news event, they observe a significant increase in prediction drift alerts, but the model's recommendations remain relevant and user engagement is stable. They want to reduce unnecessary alerts without losing the ability to detect true model degradation. What should they do?

Accepted Answer

Increase the prediction drift threshold to a higher value based on historical variance.. Prediction drift alerts are based on changes in the model's output distribution. A major news event can legitimately shift user behavior and thus prediction distributions without degrading model quality. Raising the drift threshold based on historical variance reduces false alerts while preserving the ability to detect significant deviations. Disabling monitoring, changing frequency, or switching to skew detection do not appropriately address the root cause.

Answer

Switch from prediction drift to training-serving skew monitoring.

Answer

Reduce the monitoring frequency from daily to weekly to decrease the number of alerts.

Answer

Disable prediction drift monitoring and rely on business metrics instead.

Question 12

A financial institution wants to detect fraudulent transactions in real-time. They have a labeled dataset of historical transactions and want to build a custom model with minimal coding. They also need to integrate the model into an existing application that expects a REST API. Which Google Cloud service should they use to train and deploy the model with the least effort?

Accepted Answer

Vertex AI AutoML Tabular to train a model, then deploy it to a Vertex AI endpoint.. Vertex AI AutoML Tabular provides a low-code solution for training custom models on tabular data. It automatically handles feature engineering and model selection, and the trained model can be deployed to a Vertex AI endpoint, which offers a REST API for real-time predictions. This integration minimizes effort and meets the need for a custom model with REST API access. Other options either require more coding, add unnecessary complexity, or do not support real-time serving natively.

Answer

AI Platform Training with a custom TensorFlow model, then deploy to AI Platform Prediction.

Answer

Cloud Functions to call the BigQuery ML model directly via a REST API.

Answer

BigQuery ML to train a logistic regression model, then export the model to Vertex AI for deployment.

Question 13

Your team is using Vertex AI Pipelines to train a model weekly. You want to monitor the pipeline for failures and receive a notification when a pipeline run fails. You have configured the pipeline to send logs to Cloud Logging. What should you do to receive an alert on pipeline failure?

Accepted Answer

Create a log-based metric that counts error log entries and an alerting policy on that metric.. To alert on pipeline failures, you can create a log-based metric that filters for error logs from Vertex AI Pipelines and then create a Cloud Monitoring alerting policy on that metric. This leverages existing logs and integrates with Cloud Monitoring's alerting system without custom code.

Answer

Set up a Cloud Function that triggers on pipeline completion and sends an email if the status is failed.

Answer

Use Cloud Monitoring's built-in Vertex AI Pipeline failure metric to create an alert.

Answer

Configure the pipeline to publish a custom metric to Cloud Monitoring on failure and create an alert on that metric.

Question 14

A team uses Vertex AI Workbench notebooks for collaborative model development. They want to ensure that code changes are version-controlled, that multiple data scientists can work on the same notebook without conflicts, and that the environment is reproducible across team members. Which approach should they take?

Accepted Answer

Use Vertex AI Workbench managed notebooks with Git integration and a custom container image for environment reproducibility.. Vertex AI Workbench managed notebooks support Git integration for version control and custom container images for reproducible environments. This combination allows multiple data scientists to collaborate on notebooks with version history and ensures the runtime environment is consistent across team members.

Answer

Use a shared JupyterLab instance launched on a single VM; data scientists connect simultaneously.

Answer

Store notebooks in Cloud Storage and share the bucket; each user edits their own copy.

Answer

Use Vertex AI Pipelines to run all code as pipelines; data scientists only view results in notebooks.

Question 15

You are optimizing a model for deployment on Vertex AI using NVIDIA Triton Inference Server. Which TWO actions can you take to improve inference performance?

Accepted Answer

Use TensorRT to quantize the model to FP16 or INT8.. Option B is correct because TensorRT can quantize a model to FP16 or INT8, reducing precision and memory bandwidth requirements while leveraging NVIDIA GPU tensor cores, which directly lowers inference latency and increases throughput on Triton. Option D is correct because Triton's dynamic batching aggregates multiple incoming inference requests into a single batch at runtime, improving GPU utilization and throughput without requiring client-side batching. Option A is not appropriate because simply maxing out replicas can waste resources and does not guarantee better performance if the model is not compute-bound or if the machine lacks capacity. Option C is wrong because disabling model caching forces reloads and increases latency rather than improving inference performance. Option E is not ideal because adding vCPUs does not help GPU-bound inference and may not improve Triton throughput.

Answer

Increase the number of model replicas to the maximum.

Answer

Disable model caching to reduce memory usage.

Answer

Use a larger machine type with more vCPUs.