PDE Operationalizing machine learning models — Timed 20 Questions

Question 1

A company deploys a machine learning model to Vertex AI for real-time predictions. After deployment, they notice that prediction latency spikes during peak traffic hours. Which approach should they take to reduce latency without sacrificing accuracy?

Accepted Answer

Configure auto-scaling with higher min and max instances. Option A is correct because configuring auto-scaling with higher min and max instances ensures that Vertex AI has sufficient pre-warmed replicas to handle traffic spikes without cold-start latency. This approach maintains model accuracy because it does not alter the model architecture or inference logic, only the infrastructure capacity.

Answer

Reduce the number of input features

Answer

Switch from online to batch prediction

Answer

Use a larger machine type for the model

Question 2

A data science team uses Vertex AI Pipelines to automate retraining. They want to ensure that only models with performance above a threshold are deployed. Which component should they add to the pipeline?

Accepted Answer

Vertex AI Model Evaluation. Vertex AI Model Evaluation provides built-in evaluation metrics and threshold-based validation that can be used as a pipeline condition to gate model deployment. By adding a Model Evaluation component, the pipeline can compare model performance against a predefined threshold and only proceed to deploy if the metrics (e.g., AUC, precision, recall) meet or exceed the required value.

Answer

Vertex AI Feature Store

Answer

Cloud Build trigger

Answer

Cloud Monitoring alert

Question 3

A company trains a custom model using TensorFlow and wants to deploy it to Vertex AI for low-latency predictions. The model is large (2 GB). Which deployment option should they choose?

Accepted Answer

Deploy to Vertex AI Endpoint with a custom container. Option C is correct because deploying a large (2 GB) model to Vertex AI Endpoint with a custom container allows you to package the model, its dependencies, and a serving framework (e.g., TensorFlow Serving) into a Docker image. This approach supports low-latency predictions by keeping the model loaded in memory across requests, and it can scale to handle real-time inference traffic, unlike batch or serverless options that have cold-start or size limitations.

Answer

Use Vertex AI Batch Prediction job

Answer

Deploy as a Cloud Function

Answer

Deploy to Cloud Run with minimum instances

Question 4

A company uses Vertex AI to serve a model. They notice that some predictions are incorrect due to data drift. What is the best way to detect and retrain the model automatically?

Accepted Answer

Use Vertex AI Model Monitoring with alerts and retraining pipeline. Option D is correct because Vertex AI Model Monitoring is specifically designed to detect data drift and feature skew in production models. It can be configured to send alerts and trigger an automated retraining pipeline via Cloud Functions or Vertex AI Pipelines, enabling continuous model improvement without manual intervention. This directly addresses the need for automatic detection and retraining in response to data drift.

Answer

Store predictions in BigQuery and run scheduled queries

Answer

Create a Cloud Monitoring dashboard

Answer

Set up Cloud Logging metrics to monitor predictions

Question 5

A financial services company needs to explain predictions from a complex ensemble model for regulatory compliance. Which Vertex AI service should they use?

Accepted Answer

Vertex AI Explainable AI. Vertex AI Explainable AI is the correct service because it provides feature attributions and other explainability techniques (e.g., Shapley value approximations, integrated gradients) that help interpret predictions from complex ensemble models. This is essential for regulatory compliance, where the company must demonstrate how input features influence each prediction, ensuring transparency and auditability.

Answer

Vertex AI Vizier

Answer

Vertex AI Feature Store

Answer

Vertex AI Prediction

Question 6

A team wants to retrain a model weekly using new data stored in BigQuery. They want to minimize manual effort. Which approach should they use?

Accepted Answer

Create a Vertex AI Pipeline scheduled via Cloud Scheduler. Vertex AI Pipelines allow you to define a repeatable, automated ML workflow that can be triggered on a schedule via Cloud Scheduler. This minimizes manual effort by handling data extraction from BigQuery, model retraining, and deployment without human intervention, while also providing versioning and monitoring capabilities.

Answer

Use Cloud Scheduler to trigger a Cloud Function that retrains

Answer

Retrain manually in a notebook each week

Answer

Use Cloud Composer to orchestrate retraining

Question 7

A company deploys a model to Vertex AI Endpoint. They want to run a canary deployment to test a new model version with 10% of traffic. How should they configure this?

Accepted Answer

Deploy the new model to the same endpoint and set traffic split. Option C is correct because Vertex AI Endpoints natively support traffic splitting between model versions deployed to the same endpoint. By deploying the new model version to the same endpoint and setting a traffic split of 10% to the new version and 90% to the current version, the company can perform a canary deployment without changing the application code or infrastructure.

Answer

Deploy to a new endpoint and update the application to call both

Answer

Use Cloud Load Balancing to route traffic

Answer

Deploy to Cloud Run and use gradual rollout

Question 8

A data scientist uses Vertex AI Workbench notebooks for model development. They want to share the environment with team members while maintaining version control. Which approach should they use?

Accepted Answer

Use a user-managed notebook instance with multiple users. A user-managed notebook instance with multiple users is the correct approach because Vertex AI Workbench supports collaboration by allowing multiple users to access the same instance via IAM permissions, while the underlying Git integration enables version control. This setup provides a shared, persistent environment where team members can work on the same codebase without duplicating work, and changes can be tracked through Git repositories.

Answer

Use Cloud Shell and clone the repo

Answer

Share the notebook via Cloud Storage

Answer

Store notebooks in Cloud Source Repositories

Question 9

A company wants to monitor the performance of a deployed model in production. Which metric indicates that the model's predictions are degrading?

Accepted Answer

Increase in prediction error rate. An increase in prediction error rate directly indicates that the model's outputs are deviating from the expected or ground-truth values, signaling degradation in predictive performance. This metric captures the core concept of model drift, where the statistical properties of the input data or the relationship between features and labels change over time, leading to less accurate predictions. In production ML monitoring, tracking error rate (e.g., classification accuracy, RMSE) is the primary method to detect when a model needs retraining or updating.

Answer

Increase in prediction latency

Answer

Decrease in throughput

Answer

Increase in number of requests

Question 10

A team uses Vertex AI AutoML Tables to train a model. They need to deploy the model for real-time predictions with high availability. Which deployment configuration should they use?

Accepted Answer

Deploy to a Vertex AI Endpoint with multiple replicas and auto-scaling. For real-time predictions with high availability, you need a deployment that can handle traffic spikes and failover. Deploying to a Vertex AI Endpoint with multiple replicas and auto-scaling ensures that the model is served from multiple instances, providing redundancy and the ability to scale up or down based on demand. This configuration meets the high-availability requirement by distributing load and automatically recovering from instance failures.

Answer

Export as a Cloud Function

Answer

Deploy to a Vertex AI Endpoint with 1 replica

Answer

Use a Vertex AI Batch Prediction job

Question 11

A company uses Vertex AI to serve a model that requires GPU for inference. They want to minimize cost while handling variable traffic. Which strategy should they use?

Accepted Answer

Use a Vertex AI Endpoint with GPU and configure auto-scaling to zero when idle. Option B is correct because Vertex AI Endpoints support GPU-accelerated inference with autoscaling, including the ability to scale down to zero replicas when there is no traffic. This minimizes cost by only incurring GPU charges during active inference, while still handling variable traffic through dynamic scaling.

Answer

Deploy the model to Cloud Functions with GPU

Answer

Use Vertex AI Batch Prediction with GPU

Answer

Use a Vertex AI Endpoint with GPU with a fixed number of replicas

Question 12

Which TWO steps are required to deploy a custom scikit-learn model to Vertex AI for online predictions?

Accepted Answer

Save the model using joblib or pickle. Option C is correct because scikit-learn models must be serialized using joblib or pickle to be saved as a model artifact that can be uploaded to Vertex AI. Vertex AI's pre-built prediction containers for scikit-learn expect the model file to be in this format (typically model.joblib or model.pkl) to serve online predictions.

Answer

Write a custom prediction routine

Answer

Containerize the model using Docker

Answer

Create a Vertex AI Endpoint manually

Question 13

Which THREE factors should be considered when designing a Vertex AI Pipeline for continuous training?

Accepted Answer

Cost of training and infrastructure. Cost of training and infrastructure (A) is correct because Vertex AI Pipelines incur compute costs for each pipeline run, including training, data processing, and orchestration. Continuous training amplifies these costs, so you must consider budget constraints, resource optimization (e.g., using preemptible VMs), and cost monitoring to avoid unexpected bills.

Answer

Debugging tools like Cloud Debugger

Answer

Number of model versions to keep

Question 14

Which TWO actions can help reduce prediction latency for a Vertex AI endpoint?

Accepted Answer

Optimize the model architecture to reduce size. Optimizing the model architecture to reduce size directly decreases the computational load during inference, which lowers prediction latency. Smaller models require fewer floating-point operations (FLOPs) per prediction, enabling faster response times on Vertex AI endpoints.

Answer

Increase the number of features

Answer

Use a larger machine type with more vCPUs

Answer

Set min replicas to 0 to save cost

Question 15

Which THREE metrics should be monitored for a deployed machine learning model in production?

Accepted Answer

Prediction error rate. Prediction error rate (Option B) is a direct measure of model accuracy in production, reflecting how often the model's predictions deviate from actual outcomes. Monitoring this metric is essential for detecting model degradation, data quality issues, or concept drift that can silently reduce model performance over time.

Answer

Number of replicas

Answer

Training time

Question 16

A company has a production machine learning model deployed on Vertex AI Endpoint that predicts customer churn. The model is retrained weekly using a Vertex AI Pipeline that pulls new data from BigQuery. Recently, the model's accuracy has been declining. The data science team suspects data drift but is unsure. They have enabled Vertex AI Model Monitoring but have not set up any alerts. The team wants to diagnose and address the issue quickly. The pipeline runs successfully, and no errors are reported. The model endpoint is serving predictions with average latency of 200ms. What should the team do first?

Accepted Answer

Review Vertex AI Model Monitoring drift reports and set up alerts for significant drift. Option D is correct because the team has already enabled Vertex AI Model Monitoring, which automatically tracks feature distributions and prediction statistics over time. The first diagnostic step should be to review the drift reports generated by Model Monitoring to confirm whether data drift is occurring, and then set up alerts so the team is proactively notified of significant drift in the future. This directly addresses the suspected root cause without unnecessary operational changes.

Answer

Immediately trigger a retraining pipeline with more recent data

Answer

Increase the number of replicas to reduce latency

Answer

Examine Cloud Logging for prediction errors

Question 17

A retail company uses a Vertex AI endpoint to serve product recommendations. The model is a TensorFlow model deployed with a custom container. Recently, users have reported that recommendations are stale. The model is retrained daily using Vertex AI Pipelines. The pipeline completes successfully, but the endpoint continues to serve the old model. The team checks the pipeline logs and sees that the new model is uploaded to the Vertex AI Model Registry. The endpoint has traffic split set to 100% for the old model. The team needs to update the endpoint to serve the new model version. What should they do?

Accepted Answer

Update the endpoint to deploy the new model version from the registry and adjust traffic split. Option D is correct because the pipeline successfully uploaded the new model to the Vertex AI Model Registry, but the endpoint still has its traffic split configured to 100% for the old model. To serve the new model, the team must explicitly update the endpoint to deploy the new model version from the registry and adjust the traffic split to route 100% of traffic to it. This is a standard operational step in Vertex AI: uploading a model does not automatically update the endpoint's deployment or traffic allocation.

Answer

Check the pipeline for errors in the deployment step

Answer

Re-upload the model with a different version ID

Answer

Redeploy the same model to the endpoint

Question 18

A company has deployed a machine learning model on Vertex AI Prediction that serves real-time predictions for a customer-facing application. The model was trained using a custom container and is hosted on a single endpoint with a minimum number of nodes. Recently, the team noticed that during peak traffic, prediction latency increases significantly and some requests time out. The endpoint is configured with a baseline traffic split of 100% on the current model version. Which action should the team take to reduce latency and improve reliability?

Accepted Answer

Configure horizontal autoscaling with a higher maximum number of nodes and set a CPU utilization target.. Option C is correct because configuring horizontal autoscaling with a higher maximum number of nodes and a CPU utilization target allows Vertex AI Prediction to automatically add more nodes during peak traffic, distributing the inference load and reducing latency. This directly addresses the root cause—insufficient compute resources under high demand—without requiring architectural changes or sacrificing availability.

Answer

Reduce the minimum number of nodes to zero to allow scale-to-zero during low traffic.

Answer

Place a Google Cloud Load Balancer in front of the Vertex AI endpoint to distribute requests across multiple endpoints.

Answer

Implement A/B testing by splitting traffic between two model versions to distribute load.

Question 19

A data science team is operationalizing a batch prediction job using Vertex AI Batch Prediction. The model uses a custom container that requires a specific GPU for inference. The job processes a large dataset stored in Cloud Storage. The team wants to minimize cost while ensuring the job completes within a 2-hour window. Which configuration should they choose?

Accepted Answer

Use a custom training job with a GPU worker pool and run the inference as a custom job.. Option A is correct because Vertex AI Batch Prediction does not support custom containers with GPU accelerators; it only supports CPUs for batch prediction jobs. To run GPU-accelerated inference on a large dataset, the team must use a custom training job (which supports GPU worker pools) and run inference as a custom job. This approach allows them to leverage GPU hardware for the 2-hour window while minimizing cost by using preemptible VMs or choosing the smallest GPU instance that meets throughput requirements.

Answer

Use a custom machine type with a GPU accelerator in the batch prediction request.

Answer

Use a high-memory machine type (e.g., n1-highmem-32) without GPU to reduce cost.

Answer

Configure a Vertex AI endpoint with GPU and submit batch requests to the endpoint.

Question 20

A company is deploying a machine learning model for fraud detection. The model is trained using TensorFlow and will be served on Vertex AI Prediction. The team wants to implement model monitoring to detect prediction drift. Which TWO actions should they take? (Choose 2)

Accepted Answer

Configure Vertex AI Model Monitoring to compare online prediction inputs against training data statistics.. Option A is correct because Vertex AI Model Monitoring can be configured to compare online prediction inputs against training data statistics to detect skew, which is a form of drift. This is a standard capability of Vertex AI Model Monitoring, where you specify a baseline dataset (typically training data) and the service automatically computes statistics on incoming prediction requests to identify distribution shifts.

Answer

Collect ground truth labels for all predictions to measure accuracy drift.

Answer

Set up a separate Cloud Monitoring alerting policy to watch for prediction errors.

Answer

Enable automatic model retraining in Vertex AI Model Monitoring when drift is detected.