PDE Operationalizing machine learning models Practice Test 11 — 15 Questions

Question 1

Your company uses Vertex AI Pipelines to automate the ML lifecycle. The pipeline includes training, evaluation, and deployment steps. You want to ensure that if a pipeline run fails due to a transient error (e.g., resource quota shortage), it automatically retries before marking the run as failed. What is the best way to implement this?

Accepted Answer

In the pipeline component code, implement retry logic using exponential backoff for specific exceptions.. Vertex AI Pipelines does not have built-in retry logic for failed steps. You can wrap each step's logic to catch transient errors and retry, or use a retry mechanism in the container itself. Kubeflow Pipelines' retry policy can be specified. Modifying pipeline code is the most direct way.

Answer

Configure Vertex AI Pipelines to automatically restart failed runs.

Answer

Set a high timeout value for the pipeline so that transient errors resolve before timeout.

Answer

Use Cloud Tasks to schedule pipeline runs and retry upon failure.

Question 2

You are designing a system to serve predictions from a large language model (LLM) with a latency SLO of 500ms. The model does not fit on a single GPU and requires model parallelism. You are considering using Vertex AI Endpoints with a custom container. What additional setup is required to achieve the latency target?

Accepted Answer

Use a machine type with multiple GPUs and configure the container to use tensor parallelism.. Model parallelism across multiple GPUs on a single machine can be handled by the container using libraries like TensorFlow Distribution Strategies. Sharding across endpoints would incur network latency. Using TPUs is an alternative but not necessarily required. The key is to configure multi-GPU in the machine type.

Answer

Compile the model using TensorFlow XLA to optimize for single GPU execution.

Answer

Deploy the model across multiple endpoints and use a load balancer to send requests to different parts of the model.

Answer

Use Vertex AI Prediction as a service for LLMs, which automatically handles hardware selection.

Question 3

Which TWO configurations are required to enable online prediction for a model deployed on Vertex AI Endpoints?

Accepted Answer

The endpoint must be configured with a machine type (e.g., n1-standard-2).. Option B is correct because Vertex AI Endpoints require a machine type to be specified when deploying a model. The machine type determines the compute resources (CPU/memory) allocated to the serving container, which is essential for handling prediction requests. Without a machine type, the endpoint cannot provision the underlying infrastructure to serve online predictions.

Answer

A feature store must be attached to the endpoint.

Answer

The model must be trained on Vertex AI.

Answer

Autoscaling must be enabled.

Question 4

Which THREE metrics should be monitored to detect model drift in a production ML system?

Accepted Answer

Prediction distribution (prediction drift).. Prediction drift (distribution of predictions), feature drift (distribution of input features), and model performance metrics (e.g., accuracy) are key indicators. Infrastructure metrics (CPU usage) and training loss are not directly drift indicators.

Answer

Training loss convergence.

Answer

CPU utilization of the serving nodes.

Question 5

Which THREE steps are essential for implementing a continuous training pipeline with Vertex AI?

Accepted Answer

If the new model passes evaluation, deploy it to a production endpoint.. A continuous training pipeline involves automated retraining, evaluation, and deployment when new data or model improvements occur. Manual approval is optional, not essential. One-time manual deployment is not continuous. The three essential steps are: trigger on new data, train, and evaluate/promote.

Answer

Manually approve each new model version before deployment.

Answer

Deploy the original model once and set it to auto-update.

Question 6

Your company runs a real-time recommendation system for a popular e-commerce website using a machine learning model deployed on Vertex AI Endpoints. The model takes user features and product catalog data as input and returns top-10 product recommendations. The system uses a feature store to serve user embeddings and product embeddings. Recently, the recommender team retrained the model with a new algorithm and deployed it as a new version. Since the deployment, the latency for recommendation requests has increased from 100ms to 500ms on average, exceeding the 200ms SLO. The model accuracy is acceptable, and there are no errors. The endpoint uses an n1-standard-8 machine with a single GPU. The new model is larger but still fits on the GPU. You investigate and find that the GPU utilization remains low (<20%), but CPU utilization is high (90%). What should you do to reduce latency while maintaining accuracy?

Accepted Answer

Increase the number of replicas (nodes) to parallelize the CPU-bound preprocessing work.. Option C is correct because the high CPU utilization (90%) with low GPU utilization (<20%) indicates that the bottleneck is CPU-bound preprocessing, not GPU inference. Increasing the number of replicas (nodes) distributes the CPU preprocessing load across multiple instances, reducing per-request latency without affecting model accuracy. This directly addresses the root cause while keeping the existing GPU resources.

Answer

Upgrade the machine type to one with more GPU memory (e.g., n1-standard-8 with a larger GPU) to reduce model inference time.

Answer

Change the batch size in the model serving code to process multiple requests together, improving GPU utilization.

Answer

Offload preprocessing to a dedicated Cloud Run service that runs asynchronously and returns precomputed feature vectors.

Question 7

You are a data engineer at a financial services company that uses Vertex AI to train and deploy models for credit risk assessment. The company has strict governance requirements: every model version must be approved by the risk committee before going to production. The approval process can take several days. Currently, the team trains a new model weekly and manually deploys it to a staging endpoint for review, then manually promotes to production after approval. This process is error-prone and slow. You want to automate the pipeline: training should trigger automatically when new data arrives, the model should be automatically deployed to a staging endpoint for review, and after manual approval, it should be promoted to production. Additionally, you need to ensure that if a model in staging performs poorly (e.g., low accuracy), it should not be promoted even if approved. What should you do?

Accepted Answer

Create a Vertex AI Pipeline that: (1) Triggers on new data, (2) Trains model, (3) Evaluates and stores metrics in the model registry, (4) Deploys to staging endpoint as a new model version. Then use a manual approval step (e.g., via Cloud Build approval or external system) to trigger a second pipeline that checks the stored metrics and, if acceptable, deploys to production endpoint.. The best approach uses Vertex AI Pipelines to automatically train and deploy to a staging endpoint. After manual approval, a separate pipeline step checks model performance metrics (which were stored during training/evaluation) and if they meet a threshold, promotes to production. This enforces governance and automation.

Answer

Use Vertex AI Experiments to track model versions, then manually deploy from the Experiments UI.

Answer

Use Cloud Scheduler to run training weekly, then use Cloud Functions to deploy to staging, and after manual approval, use another Cloud Function to check performance and deploy to production.

Answer

Train models on Vertex AI Workbench and use a CI/CD tool like Cloud Build to deploy to staging. Use a Cloud Build approval step to promote to production after manual check.

Question 8

A company deploys a scikit-learn model on Vertex AI for online predictions. The model is packaged in a custom container with all dependencies. Users report high latency (over 5 seconds) for predictions. The model size is 2 GB. What is the most likely cause of the high latency?

Accepted Answer

Using a custom container with a large unoptimized model. Option C is correct because a 2 GB model loaded into a custom container without optimization (e.g., quantization, pruning, or ONNX conversion) will cause significant cold-start latency and per-request loading overhead. Vertex AI online predictions require the model to be loaded into memory for each request or container instance; a large, unoptimized model increases both loading time and inference time, easily exceeding 5 seconds.

Answer

Using online predictions instead of batch prediction

Answer

Not enabling GPU acceleration

Answer

Using a small machine type (e.g., n1-standard-2)

Question 9

A data scientist trains a TensorFlow model using Vertex AI Training and wants to deploy it for online prediction. Which Vertex AI resource should the data scientist use to create an endpoint for serving predictions?

Accepted Answer

Vertex AI Endpoint. Option A is correct because Vertex AI Endpoint is the resource for serving online predictions. Option B (Model Registry) stores models but does not serve. Option C (Batch Prediction Job) is for batch predictions. Option D (Feature Store) is for managing features.

Answer

Vertex AI Batch Prediction Job

Answer

Vertex AI Feature Store

Answer

Vertex AI Model Registry

Question 10

A company has a production model deployed on Vertex AI that shows declining accuracy over time. The model uses features from a BigQuery feature store. The data science team suspects data drift. What is the most efficient way to monitor and detect drift for this model?

Accepted Answer

Enable Vertex AI Model Monitoring on the endpoint to automatically detect skew and drift. Option B is correct because Vertex AI Model Monitoring can automatically monitor prediction input data for drift and send alerts. Option A (manual comparison) is not efficient. Option C (Cloud Monitoring dashboards) can show metrics but not automatically detect drift. Option D (retraining pipeline) is reactive, not proactive monitoring.

Answer

Periodically export training data and production data to CSV and compare distributions manually

Answer

Create a scheduled retraining pipeline that runs weekly

Answer

Set up Cloud Monitoring dashboards to track prediction request volumes and error rates

Question 11

A team uses Vertex AI Pipelines to automate retraining of a model every month. The pipeline includes data preprocessing, training, and deployment steps. After a recent update, the pipeline fails intermittently with a timeout error during the deployment step. What is the most likely cause?

Accepted Answer

The trained model size has increased due to more data, causing the deployment step to time out. Option D is correct because a model with increased size due to training on more data can cause the deployment step to time out. Option A (BigQuery quotas) would affect preprocessing, not deployment. Option B (insufficient service account permissions) would cause persistent errors, not intermittent. Option C (pipeline step order) would cause consistent failure.

Answer

The service account used by the pipeline lacks permissions to deploy the model

Answer

The pipeline is configured to run steps in parallel, leading to resource contention

Answer

BigQuery query quotas are being exceeded during data preprocessing

Question 12

A financial services company deploys a fraud detection model on Vertex AI using a custom prediction container that runs a PyTorch model. The model requires GPU acceleration. The deployment succeeds but predictions return an error: 'CUDA error: out of memory'. What should the team do to resolve this issue?

Accepted Answer

Increase the GPU machine type to one with more memory (e.g., from NVIDIA T4 to A100). Option A is correct because the GPU memory is insufficient; using a machine with more GPU memory or optimizing the model is the solution. Option B (enabling model monitoring) does not fix memory. Option C (adding more CPUs) does not address GPU memory. Option D (using CPU-only) would defeat the purpose of GPU acceleration.

Answer

Change the container to use a CPU-only image to avoid CUDA errors

Answer

Enable Vertex AI Model Monitoring to automatically scale the endpoint

Answer

Add CPU replicas to distribute the inferencing load

Question 13

A company uses Vertex AI Feature Store for serving features to both training and prediction. The team notices that predictions made shortly after training use different feature values, causing a training-serving skew. What is the most effective way to prevent this skew?

Accepted Answer

Configure the Feature Store to use point-in-time lookup using the training timestamp. Option C is correct because using a feature timestamp to serve the exact feature point-in-time that was used during training ensures consistency. Option A (retraining more often) does not prevent skew. Option B (batch prediction) still uses current features. Option D (identical compute resources) does not affect feature values.

Answer

Retrain the model more frequently to adapt to the new feature distributions

Answer

Use batch prediction instead of online prediction to ensure consistent features

Answer

Ensure that the training and prediction environments use identical compute resources

Question 14

A company wants to version its ML models and track lineage from training data to deployed model. Which Google Cloud service should they use?

Accepted Answer

Vertex AI ML Metadata. Option B is correct because Vertex AI ML Metadata manages lineage and artifacts. Option A (Cloud Storage) is for storage only. Option C (Artifact Registry) is for container images, not ML models. Option D (Data Catalog) is for data discovery.

Answer

Cloud Storage with object versioning

Answer

Data Catalog

Answer

Artifact Registry

Question 15

A data science team wants to deploy a model that requires a custom container with specific NVIDIA CUDA version. They build the image and push to Artifact Registry. When deploying to Vertex AI, the model fails to load with an error: 'Failed to start container: invalid ELF header'. What is the most likely cause?

Accepted Answer

The container image was built for a different CPU architecture (e.g., ARM64) than the Vertex AI machine (x86_64). Option A is correct because the image was built for the wrong architecture (e.g., building on an ARM Mac for a x86 deployment). Option B (CUDA version mismatch) would cause a different error. Option C (container permissions) would cause a permission denied error. Option D (model file format) would cause loading errors but not container startup failure.

Answer

The model file (saved as .pkl) is corrupted

Answer

The CUDA version in the container is incompatible with the GPU on the machine

Answer

The container does not have the necessary permissions to access the model file in Cloud Storage