PMLE Practice Test 15 — 15 Questions

Question 1

A data scientist notices that the prediction distribution of a deployed model has changed significantly over the past week. They want to identify which features are contributing most to the drift. Which approach should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use Vertex AI Explainability to get feature importance and identify which features have high importance and significant drift. Vertex AI Explainability provides feature attributions (e.g., SHAP values) that can be used to correlate feature drift with model prediction changes.

Answer

Compute Pearson correlation between each feature's drift score and the model's prediction drift

Answer

Use Vertex AI Model Monitoring to compare training and serving distributions for each feature

Answer

Enable request/response logging to BigQuery and manually analyze feature distributions

Question 2

An organization wants to collect ground truth labels for model quality monitoring and store them in BigQuery. They also want to compute and visualize a confusion matrix over time. Which TWO actions should they take? (Choose 2)

Accepted Answer

Use Vertex AI Model Evaluation to compare predictions with ground truth and generate confusion matrix. Ground truth labels must be uploaded to BigQuery. Vertex AI Model Evaluation can then be used to compute metrics like confusion matrices.

Answer

Configure Vertex AI Model Monitoring for prediction drift

Answer

Use Vertex AI Model Evaluation to run sliced evaluation

Answer

Enable request/response logging to Cloud Logging

Question 3

An ML engineer is troubleshooting why a Vertex AI Endpoint is returning high prediction latency. They have enabled request/response logging and see that some requests take >1 second while most are fast. Which THREE actions should they take to diagnose the issue?

Accepted Answer

Check CPU and GPU utilisation metrics on the endpoint.. Check Cloud Monitoring metrics for p99 latency trends, CPU/GPU utilisation to identify bottlenecks, and review request/response logs for patterns (e.g., large payloads causing slow inference).

Answer

Disable Vertex AI Model Monitoring to reduce overhead.

Answer

Increase the number of replicas to resolve the issue immediately.

Question 4

An engineer wants to set up request/response logging for a Vertex AI Endpoint to analyze prediction behavior. Which TWO resources must be configured? (Choose 2)

Accepted Answer

A Cloud Logging log sink that routes endpoint logs to BigQuery. Request/response logging requires a BigQuery table as the sink destination and the log entry must include the prediction input/output.

Answer

Vertex AI Model Monitoring enabled

Answer

A Cloud Pub/Sub topic for streaming logs

Answer

Cloud Functions to transform logs

Question 5

A company wants to automatically retrain their model when data drift is detected. Which THREE components are needed to implement this pipeline?

Accepted Answer

Cloud Function to invoke Vertex AI Pipeline. The typical flow: Cloud Monitoring alert on drift → Pub/Sub topic → Cloud Function → Vertex AI Pipeline for training. Model Registry stores the new model after training.

Answer

Vertex AI Feature Store

Answer

Cloud Storage bucket for storing training data

Question 6

A company deploys a model on Vertex AI Endpoints and configures Vertex AI Model Monitoring with a sampling rate of 0.1 and monitoring frequency of every hour. They notice that the monitoring alert fires only after several hours of drift. What is the most likely cause?

Accepted Answer

The sampling rate is too low, causing the drift detection algorithm to require more time to accumulate a representative distribution. A low sampling rate (0.1) means only 10% of predictions are analyzed, which reduces the statistical power to detect drift quickly. Increasing the sampling rate improves detection speed.

Answer

The alerting threshold is set too high; it should be lowered

Answer

The monitoring frequency is too low; it should be every minute

Answer

The model endpoint is not receiving enough traffic to generate a statistically significant sample

Question 7

A data science team uses Vertex AI Experiments to compare multiple model training runs. They want to capture and compare hyperparameters, metrics, and code versions for each run. Which TWO steps should they take?

Accepted Answer

Log hyperparameters and metrics using the Vertex AI SDK's experiment logging functions. Using the Vertex AI SDK to log parameters and metrics enables comparison. Integrating with a version control system like Git ensures code version tracking.

Answer

Use Cloud Logging to capture all training outputs

Answer

Store code versions in Cloud Storage and link them to experiments manually

Answer

Export experiment data to BigQuery for comparison

Question 8

An engineer wants to configure alerting when the data distribution of a serving feature deviates from the training data distribution. The model is deployed on Vertex AI Endpoints. Which divergence metric should they use to compare the training and serving distributions?

Accepted Answer

Jensen-Shannon divergence. Vertex AI Model Monitoring supports Jensen-Shannon divergence for comparing distributions. It is a symmetric and bounded metric suitable for detecting feature skew between training and serving data.

Answer

Kullback-Leibler divergence

Answer

Population Stability Index (PSI)

Answer

Chi-squared test

Question 9

A real-time recommendation model deployed on Vertex AI Endpoints is experiencing increased latency, especially during peak hours. The model is hosted on a single machine with 4 CPUs. Which set of actions should you take to diagnose and resolve the issue?

Accepted Answer

Enable autoscaling on the endpoint and analyze request patterns to set min/max instances.. Enabling autoscaling on a Vertex AI Endpoint allows the deployment to dynamically adjust the number of serving instances based on real-time traffic, directly addressing peak-hour latency. Analyzing request patterns to set appropriate min/max instances ensures that the endpoint scales proactively without over-provisioning, which is the standard diagnostic and resolution approach for latency issues caused by insufficient capacity under variable load.

Answer

Increase the machine type to with 32 CPUs and disable autoscaling.

Answer

Switch the endpoint to use GPUs and enable batch requests.

Answer

Change the serving framework to use TensorFlow Serving with gRPC.

Question 10

An ML engineer needs to run batch predictions on 10 TB of data stored in BigQuery using a TensorFlow model. The predictions must be written to BigQuery. Which service should they use?

Accepted Answer

Use Vertex AI Batch Prediction with BigQuery source and sink.. Vertex AI Batch Prediction supports BigQuery as both source and sink, enabling direct batch prediction without additional infrastructure.

Answer

Create a Dataflow pipeline to read from BigQuery, run the model using Python, and write results to BigQuery.

Answer

Export BigQuery data to GCS, run batch prediction on GCS, then load results back to BigQuery.

Answer

Use Vertex AI online prediction with batch requests.

Question 11

A company is deploying a machine learning model for real-time inference on Vertex AI. Which TWO practices improve serving performance and reliability?

Accepted Answer

Enable autoscaling to handle traffic variations.. Vertex AI's autoscaling dynamically adjusts the number of replicas based on incoming request traffic, ensuring low latency during spikes and cost savings during lulls. This is critical for real-time inference, where consistent response times are required and manual scaling would either over-provision or under-provision resources. Autoscaling uses metrics like CPU utilization or request count to scale up or down, directly improving serving performance and reliability.

Answer

Use batch prediction for all requests.

Answer

Use manual scaling with a fixed number of replicas.

Answer

Deploy all models on the same machine type for consistency.

Question 12

A team wants to implement CI/CD for their ML models using Cloud Build. They have a pipeline that trains a model and deploys it. What is the best practice for triggering the pipeline when a new commit is pushed to the source repository?

Accepted Answer

Configure a Cloud Build trigger on the source repository (e.g., Cloud Source Repositories, GitHub). Cloud Build natively supports triggers that automatically start a pipeline when a new commit is pushed to a connected source repository (e.g., Cloud Source Repositories, GitHub, Bitbucket). This is the simplest, most event-driven approach, requiring no polling, custom services, or additional messaging infrastructure. It directly maps the git push event to a build invocation, ensuring near-instantaneous pipeline execution.

Answer

Set up a Cloud Scheduler job to poll the repository periodically

Answer

Deploy a custom web service on App Engine to call Cloud Build API

Answer

Use Pub/Sub to notify Cloud Build of new commits

Question 13

A team wants to monitor features in Vertex AI Feature Store for drift. Which TWO configurations are required?

Accepted Answer

Enable feature monitoring on the feature view. Enable feature monitoring on the feature view and configure an alerting channel (e.g., email, Pub/Sub) for drift notifications.

Answer

Set up a BigQuery sink for monitoring results

Answer

Create a Cloud Scheduler job to trigger monitoring

Answer

Deploy a monitoring model to an endpoint

Question 14

A machine learning engineer is building a Vertex AI pipeline that uses a pre-built Google Cloud Pipeline Components (GCPC) to train a custom model. Which component should the engineer use to submit a custom training job to Vertex AI?

Accepted Answer

CustomJob. The CustomJob component is the correct choice because it is the pre-built GCPC component specifically designed to submit a custom training job to Vertex AI. It allows the engineer to specify a custom container image or a Python training script, along with machine configuration and hyperparameters, directly within a Vertex AI pipeline. Other components serve different purposes, such as hyperparameter tuning, batch predictions, or model deployment.

Answer

HyperparameterTuningJob

Answer

BatchPredictionJob

Answer

ModelDeploy

Question 15

A travel booking company has a real-time recommendation system that suggests hotels and flights to users. The model is served using TensorFlow Serving on a Google Kubernetes Engine (GKE) cluster with auto-scaling enabled. The cluster uses n1-standard-4 machine types. The team has set up Cloud Monitoring dashboards and alerts. Last week, during a major holiday promotion, the team noticed that the model's inference latency P99 increased from 150 ms to 450 ms over a 30-minute period, while the request throughput increased from 500 to 1,200 requests per second. CPU utilization across the cluster rose to 95%, but memory utilization remained at 60%. The model version and the serving infrastructure configuration have not changed since the last deployment. Which action should the team take to mitigate the latency issue?

Accepted Answer

Add more nodes to the GKE cluster to increase the total CPU resources available for serving.. The latency spike is caused by CPU saturation (95% utilization) under increased load (500 to 1,200 RPS). Adding more nodes to the GKE cluster directly increases the total CPU resources available, allowing the existing TensorFlow Serving pods to handle the higher throughput without contention. This is the most immediate and infrastructure-appropriate fix because the model version and serving configuration have not changed, ruling out model-level or code-level optimizations.

Answer

Implement a feature engineering pipeline that compresses the input features to reduce data size and inference time.

Answer

Deploy a newer version of the model that uses a more efficient architecture to reduce computational complexity.

Answer

Increase the number of TensorFlow Serving instances by reducing the CPU request per pod in GKE to allow more pods per node.