PDE Operationalizing machine learning models Practice Test 2 — 15 Questions

Question 1

A company has a production machine learning model deployed on Vertex AI Endpoint that predicts customer churn. The model is retrained weekly using a Vertex AI Pipeline that pulls new data from BigQuery. Recently, the model's accuracy has been declining. The data science team suspects data drift but is unsure. They have enabled Vertex AI Model Monitoring but have not set up any alerts. The team wants to diagnose and address the issue quickly. The pipeline runs successfully, and no errors are reported. The model endpoint is serving predictions with average latency of 200ms. What should the team do first?

Accepted Answer

Review Vertex AI Model Monitoring drift reports and set up alerts for significant drift. Option D is correct because the team has already enabled Vertex AI Model Monitoring, which automatically tracks feature distributions and prediction statistics over time. The first diagnostic step should be to review the drift reports generated by Model Monitoring to confirm whether data drift is occurring, and then set up alerts so the team is proactively notified of significant drift in the future. This directly addresses the suspected root cause without unnecessary operational changes.

Answer

Immediately trigger a retraining pipeline with more recent data

Answer

Increase the number of replicas to reduce latency

Answer

Examine Cloud Logging for prediction errors

Question 2

A retail company uses a Vertex AI endpoint to serve product recommendations. The model is a TensorFlow model deployed with a custom container. Recently, users have reported that recommendations are stale. The model is retrained daily using Vertex AI Pipelines. The pipeline completes successfully, but the endpoint continues to serve the old model. The team checks the pipeline logs and sees that the new model is uploaded to the Vertex AI Model Registry. The endpoint has traffic split set to 100% for the old model. The team needs to update the endpoint to serve the new model version. What should they do?

Accepted Answer

Update the endpoint to deploy the new model version from the registry and adjust traffic split. Option D is correct because the pipeline successfully uploaded the new model to the Vertex AI Model Registry, but the endpoint still has its traffic split configured to 100% for the old model. To serve the new model, the team must explicitly update the endpoint to deploy the new model version from the registry and adjust the traffic split to route 100% of traffic to it. This is a standard operational step in Vertex AI: uploading a model does not automatically update the endpoint's deployment or traffic allocation.

Answer

Check the pipeline for errors in the deployment step

Answer

Re-upload the model with a different version ID

Answer

Redeploy the same model to the endpoint

Question 3

A company has deployed a machine learning model on Vertex AI Prediction that serves real-time predictions for a customer-facing application. The model was trained using a custom container and is hosted on a single endpoint with a minimum number of nodes. Recently, the team noticed that during peak traffic, prediction latency increases significantly and some requests time out. The endpoint is configured with a baseline traffic split of 100% on the current model version. Which action should the team take to reduce latency and improve reliability?

Accepted Answer

Configure horizontal autoscaling with a higher maximum number of nodes and set a CPU utilization target.. Option C is correct because configuring horizontal autoscaling with a higher maximum number of nodes and a CPU utilization target allows Vertex AI Prediction to automatically add more nodes during peak traffic, distributing the inference load and reducing latency. This directly addresses the root cause—insufficient compute resources under high demand—without requiring architectural changes or sacrificing availability.

Answer

Reduce the minimum number of nodes to zero to allow scale-to-zero during low traffic.

Answer

Place a Google Cloud Load Balancer in front of the Vertex AI endpoint to distribute requests across multiple endpoints.

Answer

Implement A/B testing by splitting traffic between two model versions to distribute load.

Question 4

A data science team is operationalizing a batch prediction job using Vertex AI Batch Prediction. The model uses a custom container that requires a specific GPU for inference. The job processes a large dataset stored in Cloud Storage. The team wants to minimize cost while ensuring the job completes within a 2-hour window. Which configuration should they choose?

Accepted Answer

Use a custom training job with a GPU worker pool and run the inference as a custom job.. Option A is correct because Vertex AI Batch Prediction does not support custom containers with GPU accelerators; it only supports CPUs for batch prediction jobs. To run GPU-accelerated inference on a large dataset, the team must use a custom training job (which supports GPU worker pools) and run inference as a custom job. This approach allows them to leverage GPU hardware for the 2-hour window while minimizing cost by using preemptible VMs or choosing the smallest GPU instance that meets throughput requirements.

Answer

Use a custom machine type with a GPU accelerator in the batch prediction request.

Answer

Use a high-memory machine type (e.g., n1-highmem-32) without GPU to reduce cost.

Answer

Configure a Vertex AI endpoint with GPU and submit batch requests to the endpoint.

Question 5

A company is deploying a machine learning model for fraud detection. The model is trained using TensorFlow and will be served on Vertex AI Prediction. The team wants to implement model monitoring to detect prediction drift. Which TWO actions should they take? (Choose 2)

Accepted Answer

Configure Vertex AI Model Monitoring to compare online prediction inputs against training data statistics.. Option A is correct because Vertex AI Model Monitoring can be configured to compare online prediction inputs against training data statistics to detect skew, which is a form of drift. This is a standard capability of Vertex AI Model Monitoring, where you specify a baseline dataset (typically training data) and the service automatically computes statistics on incoming prediction requests to identify distribution shifts.

Answer

Collect ground truth labels for all predictions to measure accuracy drift.

Answer

Set up a separate Cloud Monitoring alerting policy to watch for prediction errors.

Answer

Enable automatic model retraining in Vertex AI Model Monitoring when drift is detected.

Question 6

A data scientist deploys a new version of a fraud detection model (model2) alongside the existing model (model1) on the same Vertex AI endpoint with a 70/30 traffic split. After 24 hours, the team notices that model2's predictions are significantly different from model1's, and the fraud detection rate has increased. What is the most likely explanation for the change in predictions?

Accepted Answer

Model2 uses a different model artifact (fraud_detection_v2) that produces different predictions.. Option D is correct because the most straightforward explanation for a significant change in predictions and an increased fraud detection rate is that model2 uses a different model artifact (fraud_detection_v2) that was designed to produce different outputs. In Vertex AI, deploying a new model version with a traffic split means both models receive the same input data, but each model artifact independently processes it. If model2's predictions differ substantially, it indicates the model artifact itself has been updated or replaced, not that there is a data or routing issue.

Answer

Model2 was trained on data that leaked future information, causing unrealistic results.

Answer

Model2 is receiving corrupted input data due to a bug in the traffic routing.

Answer

The traffic split is misconfigured and sending all traffic to model2.

Question 7

You are a machine learning engineer at a FinTech company. Your team has developed a credit risk model using XGBoost and deployed it on Vertex AI Prediction using a custom container. The model is used for real-time credit decisions, and the endpoint is configured with a single machine type (n1-standard-4) and min_replica_count = 2, max_replica_count = 10. Recently, the team observed that during a promotional campaign, the endpoint's prediction latency increased from 200ms to over 2 seconds, and some requests resulted in 503 errors. You check the Cloud Monitoring metrics and see that CPU utilization reached 100% on the existing replicas, but the number of replicas never scaled beyond the initial 2. The deployment uses a custom container that runs a TensorFlow Serving-like model server. The container image is stored in Artifact Registry. The Vertex AI endpoint is configured with a traffic split of 100% to this model version. What is the most likely cause of the scaling failure, and what step should you take to resolve it?

Accepted Answer

Ensure the custom container exposes the correct metrics for CPU utilization so that Vertex AI autoscaling can trigger.. Option C is correct because Vertex AI's autoscaling relies on the custom container exposing standard metrics (e.g., CPU utilization via the /metrics endpoint in a Prometheus format or through the Vertex AI custom metric adapter). If the container does not expose these metrics, the autoscaler cannot detect high CPU usage and will not trigger scaling beyond the initial replicas, leading to latency spikes and 503 errors under load.

Answer

Increase min_replica_count to 5 to handle the baseline load.

Answer

Change the endpoint configuration to use gRPC instead of HTTP to reduce latency.

Answer

Set the max_replica_count to a higher value like 20.

Question 8

You have deployed a TensorFlow model on Vertex AI Endpoints with autoscaling. The model receives high traffic during peak hours, but you notice that inference latency increases significantly during cold starts. Which strategy would best minimize cold-start latency without incurring unnecessary cost?

Accepted Answer

Set minNodes to a value that handles baseline traffic, and use traffic splitting to gradually shift traffic to new replicas. Setting minNodes to a value that handles baseline traffic ensures that a minimum number of replicas are always warm, eliminating cold starts for baseline requests. Traffic splitting gradually shifts new traffic to newly created replicas, allowing them to warm up before receiving full load, which minimizes latency spikes without over-provisioning resources.

Answer

Set minNodes to 0 and enable node auto-scaling

Answer

Increase maxNodes to allow more replicas during peak, and rely on Kubernetes Horizontal Pod Autoscaler

Answer

Use Cloud Functions with Cloud Run for the model inference to leverage serverless cold-start mitigation

Question 9

Your team is using Vertex AI Pipelines to orchestrate a model retraining workflow. The pipeline includes a data validation step, a training step, and a model evaluation step. You want to ensure that if the evaluation step fails due to low model performance, the pipeline stops and does not deploy the model. Which approach should you use?

Accepted Answer

Use a Conditional in the pipeline to check evaluation metrics and only run the deployment step if metrics pass thresholds. Option C is correct because Vertex AI Pipelines supports conditional execution via the `Condition` component, which allows you to evaluate model performance metrics (e.g., accuracy, RMSE) and gate subsequent steps. By placing the deployment step inside a conditional branch that only executes when evaluation metrics meet predefined thresholds, the pipeline automatically stops and avoids deploying a poor-performing model. This approach aligns with MLOps best practices for automated gating in production pipelines.

Answer

Run the evaluation step after deployment and roll back if performance is low

Answer

Configure the evaluation step to retry up to 3 times on failure

Answer

Create a separate pipeline for deployment and trigger it manually after review

Question 10

You are using AI Platform Prediction (now Vertex AI) for online predictions. You notice that some requests are failing with a 503 status code. Which is the most likely cause?

Accepted Answer

The model is experiencing high traffic and the underlying nodes are still scaling up. A 503 status code in Vertex AI (formerly AI Platform Prediction) indicates that the prediction service is temporarily unavailable, most commonly due to autoscaling latency. When a model receives a sudden spike in traffic, the underlying nodes (compute instances) may still be provisioning and initializing, causing requests to be rejected until the new nodes are ready to serve. This is a transient condition that resolves once scaling completes.

Answer

The input data format does not match the model's expected schema

Answer

The project has exceeded its prediction requests quota

Answer

The service account used for prediction does not have the required permissions

Question 11

A retail company uses a machine learning model to predict inventory demand. The model is retrained weekly using Vertex AI Pipelines. Recently, the model's accuracy has degraded because the data distribution has shifted. Which action should you take to monitor and detect this drift automatically?

Accepted Answer

Enable Vertex AI Model Monitoring for the endpoint and configure alerting on feature drift. Vertex AI Model Monitoring is purpose-built to automatically detect feature drift and prediction drift on deployed endpoints. By enabling it and configuring alerting on feature drift, you can proactively identify when the distribution of incoming features deviates from the training data, which directly addresses the root cause of accuracy degradation without manual intervention.

Answer

Set up alerts for when the model's mean absolute error exceeds a threshold on the evaluation dataset

Answer

Enable Cloud Logging for the prediction endpoint and search for error logs

Answer

Schedule a job to compare the distribution of incoming features with the training data using Cloud Dataflow

Question 12

You are responsible for deploying a PyTorch model for real-time inference. The model requires GPU acceleration. You want to minimize infrastructure management overhead. Which serving option should you choose?

Accepted Answer

Deploy the model on Vertex AI Endpoints using a custom container with GPU support. Vertex AI Endpoints with a custom container and GPU support is the correct choice because it is purpose-built for serving ML models at scale, fully managed, and supports GPU acceleration for low-latency inference. It minimizes infrastructure overhead by handling auto-scaling, health checks, and model versioning, unlike the other options that lack GPU support or are designed for training rather than serving.

Answer

Deploy the model as a Cloud Function with a GPU backend

Answer

Use Cloud Run with GPU enabled

Answer

Use AI Platform Training to host the model as a prediction service

Question 13

A data science team has built a model using scikit-learn. They want to operationalize it on Google Cloud without rewriting the code. Which approach should they take?

Accepted Answer

Package the model in a custom container and deploy to Vertex AI Endpoints. Option C is correct because Vertex AI Endpoints support custom containers, allowing you to package your scikit-learn model with its dependencies (e.g., a Flask or FastAPI inference server) and deploy it without rewriting any code. This approach directly meets the requirement to operationalize the existing model on Google Cloud without modification.

Answer

Export the model as a PMML file and use BigQuery ML

Answer

Use AI Platform Training to host the model directly

Answer

Convert the scikit-learn model to TensorFlow SavedModel format

Question 14

You have a batch prediction job on Vertex AI that processes millions of records. The job is failing with an out-of-memory error. What is the best way to resolve this?

Accepted Answer

Use a machine type with more memory for the batch prediction job. Option D is correct because a batch prediction job on Vertex AI runs on a single machine (or a cluster of machines) and an out-of-memory (OOM) error indicates that the model or data processing exceeds the available RAM of the chosen machine type. Increasing the machine's memory directly addresses the root cause by providing more heap space for loading the model and processing large batches of predictions, without altering the job's parallelism or data partitioning.

Answer

Increase the minNodes and maxNodes for the batch prediction job

Answer

Split the input data into smaller files and run multiple batch prediction jobs

Answer

Enable autoscaling on the batch prediction job

Question 15

Your MLOps pipeline uses Vertex AI Pipelines. You want to ensure that model training uses a consistent environment with specific Python package versions. Which approach best achieves this?

Accepted Answer

Build a custom container image with all dependencies and use it in the training step. Option D is correct because building a custom container image with all dependencies ensures a fully deterministic and reproducible environment for model training. Vertex AI Pipelines executes each step as a container, so by pre-installing specific Python package versions into a custom image, you eliminate any risk of version drift or network issues during package installation at runtime. This approach aligns with MLOps best practices for environment consistency and is the most reliable method when exact package versions are critical.

Answer

Include a requirements.txt file in the pipeline step and let Vertex AI install them

Answer

Use a pre-built deep learning container from Deep Learning Containers and install packages at runtime

Answer

Specify the Python version and package versions in the training job configuration