Courseiva

CCNA Monitoring ML Solutions Questions

59 questions · Monitoring ML Solutions · All types, answers revealed

Practice these questionsDomain overviewAll questions
1
MCQhard

A team uses Vertex AI Explainable AI with integrated gradients for a deep learning model. They want to reduce the computational cost of explanations without significantly reducing explanation quality. Which configuration change should they make?

A.Switch from integrated gradients to XRAI.
B.Reduce the number of integral approximation steps.
C.Apply feature attribution to only a random subset of predictions.
D.Use sampled Shapley instead, as it is always cheaper.
AnswerB

Fewer steps lower computation; optimal steps can be tuned.

Why this answer

Integrated gradients approximates Shapley values by integrating gradients along a path. Reducing the number of steps (integral approximation steps) reduces computation, but may reduce quality. A moderate reduction balances cost and quality.

Practice this question →
2
MCQeasy

A data scientist has deployed a model on Vertex AI Endpoints and wants to monitor the model's predictions for any drift over time. Which Vertex AI service should they use?

A.Vertex AI Feature Store
B.Vertex AI Predictions
C.Vertex AI Explainable AI
D.Vertex AI Model Monitoring
AnswerD

Vertex AI Model Monitoring is designed for monitoring drift and skew in deployed models.

Why this answer

Vertex AI Model Monitoring is specifically designed to monitor deployed models for feature drift, feature skew, and prediction drift. It uses statistical methods to compare serving distributions over time or against training data.

Practice this question →
3
Multi-Selectmedium

An ML engineer wants to monitor a deployed model for fairness across different age groups and genders. Which TWO Vertex AI services should they use together to achieve this? (Choose two.)

Select 2 answers
A.Vertex AI Feature Store
B.Vertex AI Explainable AI
C.BigQuery
D.Cloud Monitoring
E.Vertex AI Model Evaluation
AnswersC, E

BigQuery stores the ground truth labels and can be used as the source for sliced evaluation.

Why this answer

Vertex AI Model Evaluation provides sliced evaluation when ground truth is available in BigQuery. Vertex AI Explainable AI can help understand feature importance but is not required for fairness monitoring.

Practice this question →
4
MCQeasy

An MLOps team has deployed a model on Vertex AI Endpoints and wants to monitor for skew between training and serving data distributions. Which Vertex AI service should they use?

A.Vertex AI Explainability
B.Vertex AI Model Monitoring
C.Vertex AI Model Registry
D.Vertex AI Continuous Training
AnswerB

Correct service for monitoring feature skew and drift.

Why this answer

Vertex AI Model Monitoring is specifically designed for monitoring feature skew (training vs serving) and drift (serving over time) on deployed models.

Practice this question →
5
MCQeasy

Which algorithm does Vertex AI Model Monitoring use by default to detect feature drift in a categorical feature?

A.Population Stability Index (PSI)
B.Wasserstein distance
C.Jensen-Shannon divergence
D.Kullback-Leibler divergence
AnswerC

Default algorithm for drift detection on categorical features.

Why this answer

Vertex AI Model Monitoring uses Jensen-Shannon divergence (JS divergence) as the default metric for drift detection on categorical features.

Practice this question →
6
Multi-Selecteasy

An ML engineer wants to monitor the performance of a Vertex AI Endpoint. Which TWO metrics are available in Cloud Monitoring for Vertex AI Endpoints? (Choose 2)

Select 2 answers
A.Model accuracy
B.Error count
C.Feature skew score
D.SHAP values
E.Prediction latency (p50, p95, p99)
AnswersB, E

Correct: Error count is available as a metric.

Why this answer

Cloud Monitoring for Vertex AI Endpoints includes metrics like prediction latency (p50, p95, p99) and error count/rate. CPU/GPU utilization is also available for endpoint machines.

Practice this question →
7
MCQhard

A financial services company deploys a model on Vertex AI Endpoints with GPU acceleration. They notice that the p99 latency for predictions has increased from 200ms to 1.2s over the past week. CPU utilisation is low, but GPU utilisation is high. Which action should they take to reduce latency?

A.Increase the CPU machine type for the endpoint.
B.Switch to a more powerful GPU type (e.g., from T4 to A100).
C.Increase the number of replicas in the endpoint deployment.
D.Reduce the sampling rate for monitoring to free up resources.
AnswerC

More replicas spread the prediction load, reducing per-request latency.

Why this answer

High GPU utilisation with low CPU utilisation suggests the model is compute-bound on GPU. Scaling out by adding more replicas distributes the load, reducing queuing and latency.

Practice this question →
8
MCQmedium

A data scientist has deployed a model with Vertex AI Endpoints and enabled request/response logging to BigQuery. They want to compute a confusion matrix over time to monitor model quality. What should they do?

A.Use Vertex AI Model Monitoring to automatically generate confusion matrices
B.Use Cloud Monitoring to create a confusion matrix dashboard
C.Upload ground truth labels to BigQuery and join with prediction logs, then compute confusion matrix in a scheduled query
D.Enable Vertex AI Explainability to get confusion matrix
AnswerC

Correct: This is the standard approach for model quality monitoring.

Why this answer

To compute a confusion matrix, ground truth labels are needed. The team can upload ground truth labels to BigQuery and join with prediction logs to compare predictions vs actuals.

Practice this question →
9
MCQhard

A company deploys a model on Vertex AI Endpoints and configures Vertex AI Model Monitoring with a sampling rate of 0.1 and monitoring frequency of every hour. They notice that the monitoring alert fires only after several hours of drift. What is the most likely cause?

A.The alerting threshold is set too high; it should be lowered
B.The monitoring frequency is too low; it should be every minute
C.The model endpoint is not receiving enough traffic to generate a statistically significant sample
D.The sampling rate is too low, causing the drift detection algorithm to require more time to accumulate a representative distribution
AnswerD

Correct: low sampling rate delays detection.

Why this answer

A low sampling rate (0.1) means only 10% of predictions are analyzed, which reduces the statistical power to detect drift quickly. Increasing the sampling rate improves detection speed.

Practice this question →
10
MCQmedium

A company wants to track the cost of their Vertex AI prediction endpoint. They use a custom machine type with 1 n1-standard-4 (4 vCPU, 15 GB memory) and 1 NVIDIA T4 GPU. The endpoint is configured for automatic scaling with min=1, max=5 replicas. Which cost monitoring approach should they use?

A.Use Cloud Billing budget alerts and export cost data to BigQuery for analysis.
B.Calculate cost manually based on replica count and GPU hours from endpoint logs.
C.Use Vertex AI Experiments to track cost.
D.Monitor only the CPU utilisation metrics to infer cost.
AnswerA

Correct: Cloud Billing provides accurate cost tracking and alerts.

Why this answer

Vertex AI prediction costs are composed of per-hour per-replica compute and GPU charges plus per-request usage. The best approach is to use Cloud Billing export to BigQuery and query cost data by service and SKU, or use the Vertex AI cost tables in the console.

Practice this question →
11
MCQhard

An ML team has set up automated retraining triggered by Cloud Monitoring alerts. When a feature drift alert fires, a Cloud Function publishes to Pub/Sub, which triggers a Vertex AI Pipeline. However, the retraining pipeline is failing because the training data is not updated. What is the most likely cause?

A.The Cloud Function does not have permission to start the pipeline
B.The Pub/Sub topic is incorrectly configured
C.The training data in the pipeline input is stale or not refreshed
D.The model endpoint is overloaded
AnswerC

Correct: Retraining requires fresh data; if the pipeline uses the same old data, it may fail or not address drift.

Why this answer

Cloud Monitoring alerts can trigger retraining, but the pipeline typically expects the most recent data. If the training data is not refreshed, the pipeline may fail or produce a stale model.

Practice this question →
12
Multi-Selectmedium

An engineer wants to set up request/response logging for a Vertex AI Endpoint to analyze prediction behavior. Which TWO resources must be configured? (Choose 2)

Select 2 answers
A.Vertex AI Model Monitoring enabled
B.A Cloud Pub/Sub topic for streaming logs
C.A Cloud Logging log sink that routes endpoint logs to BigQuery
D.A BigQuery dataset to store the log entries
E.Cloud Functions to transform logs
AnswersC, D

Log sink is required to export logs to BigQuery.

Why this answer

Request/response logging requires a BigQuery table as the sink destination and the log entry must include the prediction input/output.

Practice this question →
13
MCQeasy

An ML engineer needs to monitor the online prediction latency of a Vertex AI Endpoint. Which metrics should they look at in Cloud Monitoring?

A.p50, p95, p99 latency
B.Request count and error rate
C.Skew and drift scores
D.CPU/GPU utilization
AnswerA

Correct: These percentiles are standard for monitoring prediction latency.

Why this answer

Cloud Monitoring provides latency metrics for Vertex AI Endpoints, including p50, p95, and p99 latency, which are key for understanding performance.

Practice this question →
14
MCQmedium

A machine learning engineer wants to monitor the fairness of a credit approval model across demographic subgroups. They have ground truth labels in BigQuery. Which approach should they use to evaluate performance disparities?

A.Use Vertex AI Model Evaluation with sliced evaluation in BigQuery
B.Use Vertex AI Explainability to compute feature attributions per subgroup
C.Use Cloud Monitoring custom metrics to track predictions per subgroup
D.Use Vertex AI Model Monitoring to detect skew in demographic features
AnswerA

Correct: Sliced evaluation computes metrics per subgroup to identify disparities.

Why this answer

Vertex AI Model Evaluation supports sliced evaluation, allowing comparison of metrics (like accuracy, precision) across subgroups defined by features like age, gender, etc.

Practice this question →
15
Multi-Selectmedium

An ML engineer is configuring Vertex AI Model Monitoring for drift detection on a deployed endpoint. Which TWO settings directly affect the frequency and accuracy of drift detection? (Choose 2)

Select 2 answers
A.Model version
B.Explanation method
C.Sampling rate
D.Alerting threshold
E.Monitoring frequency
AnswersC, E

Determines the fraction of predictions used for analysis; a higher rate gives more data for accurate drift detection.

Why this answer

Sampling rate controls what fraction of predictions is analyzed; monitoring frequency controls how often the distribution comparison is performed. Both directly impact detection speed and accuracy.

Practice this question →
16
MCQmedium

A team wants to monitor prediction drift on a Vertex AI Endpoint for a classification model. They have configured Vertex AI Model Monitoring with default settings. Which metric will be used to detect prediction drift?

A.Kullback-Leibler divergence
B.L-infinity distance
C.Population Stability Index (PSI)
D.Jensen-Shannon divergence
AnswerD

Default algorithm for prediction drift.

Why this answer

Vertex AI Model Monitoring uses Jensen-Shannon divergence as the default algorithm for prediction drift detection. It compares the distribution of predicted classes over time against a reference distribution (e.g., training predictions).

Practice this question →
17
Multi-Selectmedium

A company wants to automatically retrain their model when data drift is detected. Which THREE components are needed to implement this pipeline?

Select 3 answers
A.Cloud Function to invoke Vertex AI Pipeline
B.Vertex AI Feature Store
C.Cloud Monitoring alert policy for drift metric
D.Pub/Sub topic
E.Cloud Storage bucket for storing training data
AnswersA, C, D

Executes the retraining pipeline.

Why this answer

The typical flow: Cloud Monitoring alert on drift → Pub/Sub topic → Cloud Function → Vertex AI Pipeline for training. Model Registry stores the new model after training.

Practice this question →
18
MCQhard

A model deployed on a Vertex AI Endpoint uses an image model with XRAI explainability. The team notices that the prediction distributions are shifting over time. They want to monitor prediction drift. However, the explainability feature is not enabled. What must the engineer do to enable monitoring prediction drift?

A.Re-deploy the model with a sampling rate of 100%
B.Configure Vertex AI Model Monitoring to monitor prediction drift
C.Enable Vertex AI Explainability with XRAI on the endpoint deployment
D.Enable request/response logging to BigQuery and build custom drift detection
AnswerB

Correct: Prediction drift monitoring is a built-in feature of Model Monitoring.

Why this answer

Prediction drift monitoring is part of Vertex AI Model Monitoring and does not require explainability to be enabled. It can be configured independently.

Practice this question →
19
MCQmedium

A company has deployed a model to Vertex AI Endpoints and wants to monitor for feature drift using Jensen-Shannon divergence. They have set a threshold of 0.1. After one week, the monitoring job reports a divergence of 0.15 for a feature. What should the engineer do next to diagnose which features are contributing to the drift?

A.Deploy a new model version immediately
B.Use Vertex AI Explainability to compute feature attributions and identify drifted features
C.Check the model's confusion matrix in BigQuery
D.Increase the sampling rate to capture more data
AnswerB

Correct: Explainability provides feature importance, helping to pinpoint which features are driving drift.

Why this answer

To identify which features are drifting, the engineer can use Vertex AI Explainability to compute feature attributions (e.g., SHAP values) and correlate them with drift metrics.

Practice this question →
20
MCQmedium

A company wants to implement a retraining trigger for their ML model. They have set up Cloud Monitoring alerts that fire when drift exceeds a threshold. What should be the target of the alert to automatically start a Vertex AI Pipeline for retraining?

A.Vertex AI Model Registry
B.Cloud Storage bucket
C.Cloud Functions HTTP trigger
D.Pub/Sub topic
AnswerD

Correct: Monitoring alert → Pub/Sub → Cloud Function → Vertex AI Pipeline.

Why this answer

Cloud Monitoring alerts can send notifications to Pub/Sub topics. A Pub/Sub message can then trigger a Cloud Function that starts a Vertex AI Pipeline run.

Practice this question →
21
MCQhard

An organization is deploying a loan approval model and wants to monitor for fairness across demographic subgroups. They have ground truth labels stored in BigQuery. How can they use Vertex AI to evaluate performance disparities between groups?

A.Upload predictions and labels to Vertex AI Model Evaluation and specify slicing columns
B.Query BigQuery and manually compute metrics per group, then visualize in Looker
C.Enable Vertex AI Model Monitoring with fairness detection
D.Use Vertex AI Explainable AI to get feature attributions per subgroup
AnswerA

Sliced evaluation in Vertex AI Model Evaluation can compare metrics across subgroups defined by columns.

Why this answer

Vertex AI Model Evaluation provides sliced evaluation, which computes metrics (e.g., accuracy, precision) per subgroup when slicing columns are specified. This enables detection of performance disparities.

Practice this question →
22
MCQmedium

A company has a model serving predictions on Vertex AI Endpoints and wants to monitor for prediction drift. They enable Vertex AI Model Monitoring but also need to see a confusion matrix over time. How should they set up the confusion matrix monitoring?

A.Use Cloud Monitoring to create a custom dashboard with a confusion matrix chart
B.Export predictions to Cloud Storage and run a Dataflow job to compute confusion matrices
C.Upload ground truth data to BigQuery and use Vertex AI Model Monitoring's model quality monitoring
D.Enable Vertex AI Explainable AI and configure it to output confusion matrices
AnswerC

Model quality monitoring in Vertex AI Model Monitoring uses ground truth from BigQuery to generate confusion matrices.

Why this answer

Vertex AI Model Monitoring can compute confusion matrices when ground truth labels are provided in BigQuery. The service compares predictions against actuals and can output confusion matrices over time windows.

Practice this question →
23
MCQmedium

An organisation wants to monitor fairness of their loan approval model across demographic subgroups. They have predictions stored in BigQuery along with ground truth. Which GCP service can evaluate model performance for each subgroup and identify disparities?

A.Cloud Data Loss Prevention (DLP)
B.Vertex AI Explainable AI
C.Vertex AI Model Evaluation
D.Vertex AI Model Monitoring
AnswerC

Correct: supports sliced evaluation for fairness analysis.

Why this answer

Vertex AI Model Evaluation supports sliced evaluation, where metrics are computed for each subgroup defined by feature values (e.g., gender, race) in BigQuery. This helps identify performance disparities.

Practice this question →
24
MCQhard

An ML engineer is monitoring a model on Vertex AI Endpoint and sees that feature 'age' has a training distribution of (mean=45, std=10) but the serving distribution over the last hour shows (mean=30, std=15). JS divergence is 0.12, but the alert threshold is 0.1. The engineer suspects this is due to a temporary campaign targeting younger users. What should they do first?

A.Use Vertex AI Explainable AI to understand the importance of the 'age' feature for the model.
B.Immediately trigger retraining to avoid model degradation.
C.Increase the sampling rate to get more data before making a decision.
D.Adjust the alert threshold to 0.2 to avoid false positives.
AnswerA

Correct: understand feature importance to decide if drift matters.

Why this answer

Before taking action like retraining, the engineer should investigate the root cause. Using Vertex AI Explainable AI to check feature importance can confirm whether 'age' is a critical feature. If it is not important, the drift may be harmless.

Practice this question →
25
Multi-Selecthard

A team is using Vertex AI Model Monitoring and wants to set up automated retraining when drift is detected. Which THREE services are needed to implement this pipeline? (Choose three.)

Select 3 answers
A.Cloud Scheduler
B.Vertex AI Model Monitoring
C.Cloud Functions
D.Cloud Monitoring
E.Cloud Pub/Sub
AnswersC, D, E

Cloud Functions subscribes to Pub/Sub and invokes the pipeline.

Why this answer

The typical pipeline: Cloud Monitoring alert on drift → Pub/Sub message → Cloud Function or Cloud Run → triggers Vertex AI Pipeline for retraining. Optionally, Cloud Scheduler is not needed as it's event-driven.

Practice this question →
26
MCQhard

An ML team is using Population Stability Index (PSI) to monitor feature drift on a Vertex AI Endpoint. The PSI value for a feature is 0.25, which exceeds the alert threshold of 0.2. The feature has high SHAP importance. The team wants to automatically retrain the model. What is the correct end-to-end setup?

A.Vertex AI Model Monitoring alert → Cloud Scheduler → Vertex AI Training
B.Cloud Functions → Pub/Sub → Cloud Monitoring → Vertex AI Pipeline
C.Cloud Monitoring alert → Pub/Sub → Cloud Functions → Vertex AI Pipeline (with training and deployment steps)
D.Cloud Monitoring alert → Cloud Logging → Cloud Functions → Vertex AI Training
AnswerC

Correct: This is the recommended architecture for automated retraining.

Why this answer

The correct setup involves Cloud Monitoring alert (based on PSI metric) → Pub/Sub → Cloud Function → Vertex AI Pipeline to trigger retraining with updated data.

Practice this question →
27
MCQmedium

A team is monitoring a deployed model and notices that the prediction distribution has changed significantly over the last week. They want to detect which features are contributing most to the drift. Which tool should they use?

A.Vertex AI Explainable AI
B.Vertex AI Feature Store
C.Vertex AI Model Monitoring
D.Vertex AI Pipelines
AnswerA

Explainable AI provides feature attributions (SHAP, integrated gradients) that can help identify which features are drifting.

Why this answer

Vertex AI Explainable AI provides feature attributions (e.g., SHAP values) that can be used to identify which features are most important for predictions. By comparing feature importance over time, they can pinpoint which features are drifting.

Practice this question →
28
MCQmedium

A team is monitoring a model and observes that the error rate (prediction failures) has increased. They have enabled request/response logging on the Vertex AI Endpoint. How can they set up a metric and alert for prediction error rate?

A.Configure Cloud Monitoring to pull error rate from Cloud Endpoints
B.Use Vertex AI Model Monitoring to monitor error rate directly
C.Create a log-based metric in Cloud Logging for error logs and set up an alert in Cloud Monitoring
D.Enable Vertex AI Pipelines to track errors
AnswerC

Log-based metrics are the standard way to derive metrics from logs and alert on them.

Why this answer

Vertex AI Endpoint logs contain information about failed predictions. You can create a log-based metric in Cloud Logging that counts error logs, and then create an alert in Cloud Monitoring based on that metric.

Practice this question →
29
MCQeasy

A company wants to monitor the cost of their Vertex AI prediction endpoint. They are charged per hour per replica and per request for GPU instances. Which approach should they use to track these costs?

A.Set up Cloud Billing budget alerts and export billing data to BigQuery for analysis
B.Use Vertex AI Model Monitoring to track cost metrics
C.Enable Cloud Monitoring dashboards for cost metrics
D.Use Vertex AI Pipelines to track cost per job
AnswerA

Budget alerts and exporting billing data are standard practices for cost monitoring.

Why this answer

Vertex AI prediction costs are tracked via Cloud Billing. Budget alerts can be set up to notify when spending exceeds a threshold. Cost breakdown can be viewed in the Billing reports.

Practice this question →
30
MCQhard

A company has deployed a model for image classification and wants to monitor for feature drift using XRAI attributions. However, they notice that the XRAI attribution maps are too large and are causing high latency in the monitoring pipeline. What is the most effective way to reduce the overhead of explainability monitoring for image models?

A.Disable XRAI and use integrated gradients instead
B.Reduce the sampling rate for the explainability feature
C.Use a smaller image size for the model
D.Increase the number of replicas on the endpoint
AnswerB

Sampling rate controls the fraction of predictions for which explanations are generated, directly reducing overhead.

Why this answer

Vertex AI Explainable AI supports XRAI for image models, but generating XRAI attributions can be computationally expensive. Sampling a subset of predictions reduces the number of explanations generated, lowering latency and cost.

Practice this question →
31
MCQmedium

A data scientist notices that the prediction distribution of a deployed model has changed significantly over the past week. They want to identify which features are contributing most to the drift. Which approach should they use?

A.Use Vertex AI Explainability to get feature importance and identify which features have high importance and significant drift
B.Compute Pearson correlation between each feature's drift score and the model's prediction drift
C.Use Vertex AI Model Monitoring to compare training and serving distributions for each feature
D.Enable request/response logging to BigQuery and manually analyze feature distributions
AnswerA

Combining drift detection with feature importance pinpoints root cause features.

Why this answer

Vertex AI Explainability provides feature attributions (e.g., SHAP values) that can be used to correlate feature drift with model prediction changes.

Practice this question →
32
Multi-Selectmedium

An organization wants to collect ground truth labels for model quality monitoring and store them in BigQuery. They also want to compute and visualize a confusion matrix over time. Which TWO actions should they take? (Choose 2)

Select 2 answers
A.Configure Vertex AI Model Monitoring for prediction drift
B.Use Vertex AI Model Evaluation to run sliced evaluation
C.Enable request/response logging to Cloud Logging
D.Use Vertex AI Model Evaluation to compare predictions with ground truth and generate confusion matrix
E.Upload ground truth data to a BigQuery table
AnswersD, E

Model Evaluation can compute confusion matrices over time from prediction and ground truth tables.

Why this answer

Ground truth labels must be uploaded to BigQuery. Vertex AI Model Evaluation can then be used to compute metrics like confusion matrices.

Practice this question →
33
MCQmedium

An ML team wants to automatically retrain a model when prediction drift is detected on the deployed endpoint. They have Vertex AI Model Monitoring configured to send alerts to Cloud Monitoring. Which minimal set of additional services should they use to trigger a retraining pipeline?

A.Cloud Monitoring + Cloud Functions + Vertex AI Pipeline
B.Cloud Monitoring + Cloud Functions + Vertex AI Endpoint
C.Cloud Logging + Cloud Scheduler + Vertex AI Training
D.Cloud Monitoring + Cloud Pub/Sub + Cloud Functions + Vertex AI Pipeline
AnswerA

Correct chain: alert -> Pub/Sub (implicitly via Cloud Functions trigger) -> Cloud Functions -> Vertex AI Pipeline.

Why this answer

Cloud Monitoring alert policies can be configured to send webhook notifications directly to a Cloud Function's HTTP endpoint, triggering the function without requiring Pub/Sub. The Cloud Function then initiates a Vertex AI Pipeline to retrain the model. This is the minimal set of services needed; adding Pub/Sub (as in option D) would be unnecessary.

Practice this question →
34
MCQeasy

An ML engineer needs to track the costs incurred by Vertex AI prediction endpoints. Which tool should they use to set budget alerts and monitor spending?

A.Vertex AI Model Monitoring
B.Cloud Logging
C.Cloud Monitoring with custom metrics
D.Google Cloud Budgets & Alerts
AnswerD

Correct: Budgets & Alerts allow setting spending thresholds and notifications.

Why this answer

Google Cloud Budgets & Alerts allow setting budget thresholds and sending notifications. Billing reports provide cost breakdowns by service.

Practice this question →
35
Multi-Selectmedium

An ML engineer needs to set up automated retraining triggered by data drift. They have decided to use Cloud Monitoring alerts to detect drift. Which TWO additional services are required to complete the retraining pipeline? (Choose 2)

Select 2 answers
A.Cloud Dataflow
B.Cloud Build
C.Cloud Scheduler
D.Vertex AI Pipeline
E.Cloud Functions
AnswersD, E

Correct: Vertex AI Pipeline orchestrates the training and deployment steps.

Why this answer

The typical architecture uses Cloud Monitoring alert -> Pub/Sub -> Cloud Functions -> Vertex AI Pipeline. Cloud Functions processes the alert and triggers the pipeline. Vertex AI Pipeline orchestrates training and deployment.

Practice this question →
36
MCQeasy

A data scientist has deployed a classification model on a Vertex AI Endpoint and wants to monitor for feature drift in the serving data compared to the training data. Which Vertex AI service should be used?

A.Vertex AI Explainable AI
B.Vertex AI Model Evaluation
C.Vertex AI Continuous Training
D.Vertex AI Model Monitoring
AnswerD

Correct service for monitoring feature drift and skew.

Why this answer

Vertex AI Model Monitoring is specifically designed to monitor deployed models for feature skew, feature drift, and prediction drift. It supports algorithms like Jensen-Shannon divergence and Population Stability Index.

Practice this question →
37
Multi-Selecthard

An ML engineer is troubleshooting why a Vertex AI Endpoint is returning high prediction latency. They have enabled request/response logging and see that some requests take >1 second while most are fast. Which THREE actions should they take to diagnose the issue?

Select 3 answers
A.Check CPU and GPU utilisation metrics on the endpoint.
B.Review the request/response logs in BigQuery to identify if large payloads correlate with high latency.
C.Disable Vertex AI Model Monitoring to reduce overhead.
D.Check the p99 latency metric in Cloud Monitoring.
E.Increase the number of replicas to resolve the issue immediately.
AnswersA, B, D

High utilisation can indicate resource bottleneck.

Why this answer

Check Cloud Monitoring metrics for p99 latency trends, CPU/GPU utilisation to identify bottlenecks, and review request/response logs for patterns (e.g., large payloads causing slow inference).

Practice this question →
38
MCQeasy

A company wants to monitor fairness of a model by evaluating performance metrics across demographic subgroups. They have ground truth labels stored in BigQuery. Which Vertex AI service should they use?

A.Vertex AI Model Monitoring
B.Vertex AI Prediction
C.Vertex AI Explainability
D.Vertex AI Model Evaluation
AnswerD

Model Evaluation's sliced evaluation is designed for fairness assessment.

Why this answer

Vertex AI Model Evaluation supports sliced evaluation, allowing you to compute metrics per subgroup (e.g., by demographic) using data in BigQuery.

Practice this question →
39
MCQeasy

An MLOps engineer needs to collect ground truth labels for a deployed classification model to compare predictions against actuals. Where should the engineer store the ground truth data to enable Vertex AI model quality monitoring?

A.BigQuery
B.Firestore
C.Cloud Spanner
D.Cloud Storage
AnswerA

Vertex AI Model Monitoring uses ground truth labels stored in BigQuery to compute quality metrics.

Why this answer

Vertex AI Model Monitoring expects ground truth data to be uploaded to BigQuery tables, which can then be used to compute confusion matrices and other quality metrics over time.

Practice this question →
40
MCQeasy

An ML team wants to automatically retrain a model when data drift is detected. They have set up a Cloud Monitoring alert on drift. What service should they use to trigger a retraining pipeline in response to the alert?

A.Cloud Functions
B.Cloud Scheduler
C.Vertex AI Feature Store
D.Vertex AI Model Monitoring
AnswerA

Cloud Functions can subscribe to Pub/Sub and trigger Vertex AI Pipelines, enabling automated retraining.

Why this answer

Cloud Monitoring alerts can send notifications to Pub/Sub topics. A Pub/Sub message can then trigger a Cloud Function or Cloud Run service that starts a Vertex AI Pipeline for retraining.

Practice this question →
41
Multi-Selectmedium

A data science team is configuring Vertex AI Model Monitoring for a deployed model. They want to detect both feature skew and feature drift. Which TWO configurations must they set?

Select 2 answers
A.Enable request/response logging.
B.Select the Jensen-Shannon divergence algorithm.
C.Configure the monitoring frequency (e.g., hourly).
D.Set the sampling rate to 1.0 (100%).
E.Specify a training dataset or statistics for baseline.
AnswersC, E

Required to define how often drift is computed.

Why this answer

To detect feature skew, they need to specify a training dataset or statistics as a reference. To detect drift over time, they need to set a monitoring frequency. Sampling rate is optional but recommended.

Practice this question →
42
MCQmedium

A company wants to be alerted when the prediction error rate on their Vertex AI Endpoint exceeds 5% in any 5-minute window. What is the best way to set up this alert?

A.Create a Cloud Monitoring alert on the metric 'prediction/error_count' with a threshold of 5% of total predictions
B.Set up a scheduled Cloud Function to query logs and check error rate
C.Enable BigQuery log sink and create a scheduled query alert
D.Use Vertex AI Model Monitoring to detect prediction drift
AnswerA

Correct: Cloud Monitoring supports alerting on error rate metrics.

Why this answer

Cloud Monitoring can create an alerting policy based on the 'Prediction error count' metric with a rolling 5-minute window and a threshold of 5% (or absolute count).

Practice this question →
43
MCQmedium

A team is monitoring a model on Vertex AI Endpoints and wants to track the p99 latency of online predictions. Which approach should they use to set up latency monitoring and alerting?

A.Enable Vertex AI Model Monitoring and select 'latency' as a metric
B.Enable Vertex AI Explainable AI to output latency statistics
C.Configure Cloud Monitoring to scrape Prometheus metrics from the endpoint
D.Use Cloud Logging to create log-based metrics from prediction logs and set up alerts in Cloud Monitoring
AnswerD

Prediction logs contain latency information; log-based metrics can capture p99 and other percentiles.

Why this answer

Vertex AI Endpoints automatically export request/response logs to Cloud Logging, which can be used to create log-based metrics for latency percentiles. These metrics can then be visualized in Cloud Monitoring dashboards and used for alerting.

Practice this question →
44
Multi-Selecthard

A data science team wants to monitor model quality by comparing predictions against ground truth labels. They have deployed a model on Vertex AI Endpoints and enable request/response logging to BigQuery. Which THREE actions should they take to set up model quality monitoring? (Choose 3)

Select 3 answers
A.Create a scheduled query to compute metrics like accuracy and confusion matrix over time
B.Join prediction logs with ground truth labels on a common key (e.g., request ID)
C.Configure Vertex AI Model Monitoring to detect prediction drift
D.Use Vertex AI Explainability to compute feature attributions
E.Upload ground truth labels to a BigQuery table
AnswersA, B, E

Correct: Scheduled queries automate metric computation.

Why this answer

To monitor model quality, the team needs to upload ground truth labels to BigQuery, join with prediction logs, compute metrics (e.g., accuracy, confusion matrix) over time, and optionally create dashboards.

Practice this question →
45
Multi-Selecteasy

An ML engineer is monitoring a Vertex AI Endpoint and notices a spike in 5xx error rates. Which TWO metrics should they examine to diagnose the issue? (Choose 2)

Select 2 answers
A.Feature drift alert count
B.GPU utilization on the endpoint
C.CPU utilization on the endpoint
D.Vertex AI Model Monitoring skew score
E.Number of predictions per minute
AnswersB, C

GPU exhaustion can lead to prediction failures.

Why this answer

CPU/GPU utilization can indicate resource exhaustion causing errors. Prediction job failures metric directly shows failed predictions.

Practice this question →
46
MCQmedium

An ML engineer has set up Vertex AI Model Monitoring on an endpoint with a sampling rate of 0.1 (10%). They notice that the monitoring job runs hourly but the reported drift metrics seem inconsistent. What is the most likely cause?

A.The sampling rate is too low, leading to insufficient data for reliable drift statistics.
B.Prediction drift monitoring is not enabled; only feature drift is configured.
C.The drift detection algorithm is not suited for this model; try changing from JS divergence to L-infinity distance.
D.The monitoring frequency is too low; it should be set to every 5 minutes.
AnswerA

10% sampling may result in small sample sizes, causing high variance in drift estimates.

Why this answer

A low sampling rate means only a small fraction of requests are logged for monitoring, which can lead to statistical noise and inconsistent drift metrics. Increasing the sampling rate would improve accuracy.

Practice this question →
47
MCQmedium

A team wants to collect ground truth labels for their model deployed on Vertex AI Endpoint to perform model quality monitoring. They have a process that generates actual outcomes within 24 hours of prediction. What is the recommended approach for storing these labels?

A.Upload the ground truth labels to a BigQuery table with a schema that includes prediction timestamp and model version.
B.Use Vertex AI Experiments to log ground truth alongside training runs.
C.Store the ground truth labels in Cloud Storage as CSV files and reference them in the monitoring config.
D.Insert ground truth labels directly into the Vertex AI Endpoint's log sink.
AnswerA

Correct approach; BigQuery is required for ground truth storage.

Why this answer

Vertex AI Model Monitoring for model quality requires ground truth data to be uploaded to BigQuery. Labels are stored in a BigQuery table with the prediction timestamp and model ID for comparison.

Practice this question →
48
Multi-Selecthard

A team uses Vertex AI Pipelines for continuous training triggered by model drift. They want to monitor the pipeline execution cost and optimize resource usage. Which THREE metrics should they track? (Choose 3)

Select 3 answers
A.Pipeline execution duration
B.Number of failed pipeline runs
C.Model accuracy on validation set
D.Total GPU hours consumed per pipeline run
E.Cost per pipeline run in Cloud Billing
AnswersA, D, E

Longer duration increases cost; optimizing duration saves money.

Why this answer

Training cost is influenced by GPU hours, machine type, and training duration. Tracking these helps optimize.

Practice this question →
49
MCQmedium

An engineer is configuring Vertex AI Model Monitoring for a model deployed on an endpoint. They want to monitor feature skew using the training dataset as a baseline. The training dataset is large (10 TB). What is the most efficient way to provide the baseline distribution?

A.Compute the baseline distribution offline and upload a JSON file with feature statistics
B.Upload the entire training dataset as a CSV file to Cloud Storage and reference it in the monitoring config
C.Sample 1% of the training data and use that as baseline
D.Use the BigQuery table containing the training data as the baseline source in the monitoring configuration
AnswerD

Correct: Vertex AI Model Monitoring supports BigQuery tables as baseline sources.

Why this answer

Vertex AI Model Monitoring can automatically compute the baseline distribution from the training dataset stored in BigQuery. This avoids manual computation or exporting large datasets.

Practice this question →
50
MCQmedium

A data scientist notices that the model's prediction latency has increased over the last week. They need to investigate the root cause by examining request and response logs for the Vertex AI Endpoint. What is the recommended way to capture these logs?

A.Enable Vertex AI Model Monitoring with sampling rate 100%
B.Export logs to Cloud Storage using a sink and use gcloud logging read
C.Use Cloud Monitoring custom metrics to capture latency per request
D.Configure request/response logging by specifying a BigQuery destination in the endpoint deployment
AnswerD

Correct: Vertex AI Endpoints can log request/response to BigQuery when enabled in the deployment.

Why this answer

Vertex AI Endpoint can be configured to log request/response data to BigQuery via a log sink. This data can then be analyzed to understand latency issues.

Practice this question →
51
MCQmedium

An engineer wants to configure alerting when the data distribution of a serving feature deviates from the training data distribution. The model is deployed on Vertex AI Endpoints. Which divergence metric should they use to compare the training and serving distributions?

A.Kullback-Leibler divergence
B.Population Stability Index (PSI)
C.Jensen-Shannon divergence
D.Chi-squared test
AnswerC

JS divergence is the recommended metric for detecting distribution skew in Vertex AI Model Monitoring.

Why this answer

Vertex AI Model Monitoring supports Jensen-Shannon divergence for comparing distributions. It is a symmetric and bounded metric suitable for detecting feature skew between training and serving data.

Practice this question →
52
Multi-Selectmedium

An MLOps engineer is setting up monitoring for a deployed model on Vertex AI Endpoints. Which TWO actions are required to enable Vertex AI Model Monitoring for feature skew and drift? (Choose two.)

Select 2 answers
A.Export ground truth labels to Cloud Storage
B.Enable request/response logging on the Vertex AI Endpoint
C.Enable Vertex AI Pipelines to run scheduled monitoring
D.Create a ModelMonitoringJob with a monitoring configuration
E.Deploy the model with an explanation spec
AnswersB, D

Logging captures the serving data needed for monitoring.

Why this answer

To enable model monitoring, you must enable request/response logging on the endpoint (to capture serving data) and create a monitoring job with the desired configuration.

Practice this question →
53
Multi-Selecthard

A company is experiencing high prediction costs on Vertex AI Endpoints. They want to monitor and optimize costs. Which THREE actions should they take? (Choose 3)

Select 3 answers
A.Use Cloud Billing reports to track Vertex AI endpoint costs per hour and per request
B.Use Vertex AI Explainability on every prediction
C.Reduce the number of replicas or use autoscaling to minimize idle resources
D.Set up budget alerts in Google Cloud Billing to notify when costs exceed a threshold
E.Enable Vertex AI Model Monitoring to track prediction latency
AnswersA, C, D

Cloud Billing provides cost breakdowns by service and resource.

Why this answer

Cost monitoring involves tracking per-hour and per-request costs, setting budget alerts, and possibly adjusting scaling to reduce unnecessary compute.

Practice this question →
54
MCQhard

An MLOps engineer is configuring Vertex AI Model Monitoring for a deployed model. They want to monitor for feature skew between training and serving data, but only for a subset of features. The training data has 100 features, and they want to monitor only the top 10 most important features to reduce cost and noise. How can they achieve this?

A.Set the 'monitoring_interval' to a low value so that only frequent features are monitored
B.Train a new model with only the top 10 features and redeploy it
C.Use the 'feature_names' parameter in the ModelMonitoringObjectConfig to specify which features to monitor
D.Set the 'sampling_rate' to 100% and ignore the rest
AnswerC

The feature_names parameter allows you to select a subset of features for monitoring.

Why this answer

Vertex AI Model Monitoring allows you to specify a list of feature names to monitor via the 'feature_names' attribute in the monitoring configuration. This can be set when creating the monitoring job, targeting only the features of interest.

Practice this question →
55
MCQeasy

A company wants to log all prediction requests and responses from a Vertex AI Endpoint to BigQuery for auditing and debugging. How can they achieve this?

A.Export endpoint logs from Cloud Logging to Cloud Storage and then load into BigQuery manually.
B.Use a Cloud Function to intercept predictions and write to BigQuery.
C.Vertex AI endpoints do not support request/response logging.
D.Enable request/response logging on the endpoint and create a BigQuery sink for the log.
AnswerD

Correct: endpoint logging captures data, and a sink routes to BigQuery.

Why this answer

Vertex AI endpoints can be configured to enable request/response logging. The logs can be sent to a BigQuery table via a log sink.

Practice this question →
56
MCQeasy

An ML engineer has deployed a model on Vertex AI Endpoints and wants to detect when the serving data distribution differs from the training data distribution. Which monitoring feature should they enable?

A.Prediction drift monitoring
B.Feature drift monitoring
C.Model quality monitoring
D.Feature skew monitoring
AnswerD

Correct: Feature skew compares training vs serving distributions.

Why this answer

Feature skew monitoring compares the training data distribution (stored in a baseline) with the serving data distribution to detect skew. Feature drift tracks changes over time in serving data only.

Practice this question →
57
MCQhard

A team is using Vertex AI Explainability with a deployed model. They need to generate explanations for image classification predictions. Which explanation method should they configure in the ExplanationSpec?

A.XRAI
B.SHAP with KernelExplainer
C.Sampled Shapley
D.Integrated Gradients
AnswerA

XRAI is the method designed for image models in Vertex AI Explainability.

Why this answer

XRAI (eXplanation with Ranked Area Integrals) is specifically designed for image models to highlight regions that contribute to the prediction.

Practice this question →
58
MCQeasy

An ML engineer needs to monitor the error rate of prediction jobs on a Vertex AI Endpoint. Where can they view the number of failed prediction requests over time?

A.Cloud Monitoring
B.Cloud Console endpoint details page
C.Vertex AI Experiments
D.Cloud Logging
AnswerA

Correct: Cloud Monitoring provides metrics and alerts for endpoint predictions.

Why this answer

Vertex AI Endpoint metrics are integrated with Cloud Monitoring. Specific metrics like 'predictions/failed_count' can be viewed in Cloud Monitoring dashboards.

Practice this question →
59
MCQmedium

An ML engineer wants to monitor the latency of online predictions from a Vertex AI Endpoint. They need to track p50, p95, and p99 latency over time and set up alerts if p99 exceeds 1 second. Which approach should they take?

A.Configure a Cloud Monitoring dashboard using the built-in Vertex AI Endpoint metrics for latency
B.Use Cloud Monitoring custom metrics and publish latency percentiles from the application code
C.Enable request/response logging to BigQuery and calculate latency percentiles using SQL queries
D.Use Vertex AI Model Monitoring to track prediction latency
AnswerA

Built-in metrics include latency percentiles; alerts can be set directly.

Why this answer

Cloud Monitoring can ingest metrics from Vertex AI Endpoints, including prediction latency distributions (p50, p95, p99). Alerts can be configured on these metrics.

Practice this question →

Ready to test yourself?

Try a timed practice session using only Monitoring ML Solutions questions.

Start 20-question session