PMLE Serving and Scaling Models — Timed 15 Questions

Question 1

A team is building a batch prediction pipeline that processes raw data from Cloud Storage, performs complex preprocessing, and then runs predictions using a large model. The preprocessing step is compute-intensive and the prediction step is I/O-bound. Which TWO Google Cloud services should they combine to optimize cost and performance? (Choose 2)

15 minute time limit — choose an answer to begin.

Accepted Answer

Dataflow for preprocessing and writing results to Cloud Storage. Dataflow is ideal for the compute-intensive preprocessing step because it can horizontally scale across many workers to handle complex transformations in parallel, and it writes results directly to Cloud Storage, which serves as the input source for Vertex AI Batch Prediction. Vertex AI Batch Prediction is optimized for I/O-bound inference workloads: it reads batches of data from Cloud Storage, runs predictions using the large model, and writes results back to Cloud Storage, all without requiring a persistent serving endpoint, which minimizes cost for offline predictions.

Answer

Cloud Functions to preprocess data row by row

Answer

Cloud Run to serve the preprocessed data as an API

Answer

Vertex AI Batch Prediction with BigQuery source

Question 2

A data scientist wants to deploy a trained TensorFlow model to Vertex AI for online predictions. They need to serve predictions with low latency and want to leverage GPU acceleration. Which machine type should they select when creating the Vertex AI endpoint?

Accepted Answer

n1-standard-4 with 1 NVIDIA Tesla T4. The n1-standard-4 machine type supports attaching GPUs such as the NVIDIA Tesla T4, which provides GPU acceleration for low-latency online predictions. Vertex AI endpoints require a machine type that allows GPU attachment, and the n1-series is one of the few families that supports GPUs, while the T4 offers a good balance of cost and performance for inference workloads.

Answer

n1-standard-4

Answer

e2-standard-4

Answer

n1-highmem-8

Question 3

You are deploying a new version of a model to a Vertex AI endpoint that already has a champion model serving 100% of traffic. You want to gradually shift traffic to the new version while monitoring for errors. Which approach should you use?

Accepted Answer

Deploy the challenger to the same endpoint with initial traffic split, e.g., champion 90%, challenger 10%, and gradually adjust.. Vertex AI endpoints support traffic splitting between model versions deployed to the same endpoint. By deploying the challenger to the same endpoint and setting an initial split (e.g., champion 90%, challenger 10%), you can gradually shift traffic while monitoring for errors. This approach uses the endpoint's built-in traffic management, avoiding the complexity and latency of external load balancers.

Answer

Use Cloud Load Balancing with weighted backend services pointing to different endpoints.

Answer

Delete the champion model and redeploy with the challenger as the new version.

Answer

Create a new endpoint for the challenger and use a load balancer to split traffic.

Question 4

A company is using Vertex AI Prediction with a custom container that performs preprocessing before inference. The preprocessing step is CPU-intensive and the inference step uses a GPU. They want to minimize prediction latency while optimizing cost. Which architecture should they use?

Accepted Answer

Use a single GPU machine (e.g., n1-standard-4 with T4) and perform both preprocessing and inference on the same instance.. Using a CPU-only node for preprocessing and then sending the preprocessed data to a GPU node for inference separates concerns and allows independent scaling, but adds network latency. The best approach is to use a single machine with both CPU and GPU to avoid network round-trip, and to adjust the machine type to have enough CPU resources.

Answer

Use Cloud Run for preprocessing and send HTTP requests to a GPU-backed Vertex AI endpoint for inference.

Answer

Use two separate Vertex AI endpoints: one CPU-based for preprocessing, one GPU-based for inference, and chain them with Cloud Tasks.

Answer

Use Dataflow for preprocessing and then invoke the model, but Dataflow is not designed for real-time prediction.

Question 5

You need to serve a large embedding model for similarity search with low latency. The model was trained to generate 256-dimensional embeddings. You plan to use Vertex AI Vector Search. Which index type should you choose to balance accuracy and performance for a dataset with 10 million vectors?

Accepted Answer

Approximate nearest neighbor (ANN) index using ScaNN. Vertex AI Vector Search uses ScaNN (Scalable Nearest Neighbors) as its underlying ANN algorithm, which is specifically designed for high-dimensional embeddings (like 256-d) and large-scale datasets (10M vectors). ScaNN balances accuracy and performance by employing anisotropic quantization and tree-based partitioning, making it the optimal choice for low-latency similarity search without requiring exhaustive comparison.

Answer

Tree-based index

Answer

Brute-force index

Answer

Hash-based index

Question 6

A machine learning engineer needs to run batch predictions on 50 TB of data stored in BigQuery using a Vertex AI model. The model is a custom container. What is the most efficient way to set up the batch prediction job?

Accepted Answer

Create a Vertex AI batch prediction job with BigQuery source and BigQuery destination.. Vertex AI batch prediction supports BigQuery as both input and output source, which is the most direct approach. Dataflow preprocessing is optional only if needed.

Answer

Use Dataflow to process the data and call the model via Vertex AI online prediction.

Answer

Export BigQuery data to CSV in GCS, then create a batch prediction job with GCS source.

Answer

Create a Cloud Function to iterate over BigQuery rows and call the endpoint.

Question 7

You have a Vertex AI endpoint with min_replica_count=2 and max_replica_count=10. You notice that during a traffic spike, the endpoint does not scale up quickly enough, causing increased latency. What should you do to improve autoscaling responsiveness?

Accepted Answer

Reduce the target CPU utilization percentage from default to a lower value.. Reducing the target CPU utilization percentage (e.g., from the default 60% to a lower value like 40%) causes the autoscaler to trigger scale-up actions sooner, as the threshold for adding replicas is reached at a lower CPU load. This improves responsiveness during traffic spikes by initiating scaling earlier, reducing latency. The endpoint's min_replica_count=2 and max_replica_count=10 remain unchanged, so the scaling range is preserved.

Answer

Increase max_replica_count to 20.

Answer

Disable autoscaling and manually manage replicas.

Answer

Increase min_replica_count to 10.

Question 8

You are deploying a PyTorch model on Vertex AI using a custom container with NVIDIA Triton Inference Server. The model is a large transformer that requires GPU. You want to optimize GPU utilization and reduce memory footprint. Which technique should you apply?

Accepted Answer

Apply model quantization using TensorRT.. Model quantization using TensorRT reduces the precision of model weights (e.g., from FP32 to FP16 or INT8), which directly decreases GPU memory usage and can improve throughput by enabling faster arithmetic operations on compatible NVIDIA GPUs. This technique is specifically designed to optimize GPU utilization and memory footprint for large transformer models deployed with Triton Inference Server.

Answer

Enable dynamic batching in Triton.

Answer

Use CPU-only instances to avoid GPU memory issues.

Answer

Increase the number of GPU replicas.

Question 9

A company wants to cache predictions for identical requests to reduce latency and cost. They use Vertex AI Prediction with a custom container. Which GCP service should they use to implement prediction caching?

Accepted Answer

Cloud Memorystore for Redis. Cloud Memorystore (Redis) is ideal for caching because it provides low-latency key-value storage. The prediction request can be hashed to create a key.

Answer

Cloud Bigtable

Answer

Cloud Storage

Answer

Cloud Firestore

Question 10

You have a Vertex AI endpoint that serves a model for real-time predictions. You want to update the model to a new version with zero downtime. Which approach should you take?

Accepted Answer

Deploy the new model version to the same endpoint and then set traffic to 100% for the new version.. Vertex AI endpoints support canary deployments by allowing you to deploy a new model version to the same endpoint and then gradually shift traffic to it using the `traffic_split` parameter. Setting traffic to 100% for the new version after deployment ensures zero downtime, as the endpoint remains active and serves requests from the old version until the switch is complete.

Answer

Delete the endpoint and recreate it with the new model.

Answer

Use Cloud Load Balancing to switch traffic between two endpoints.

Answer

Create a new endpoint and update the client application to point to the new endpoint.

Question 11

You are using Vertex AI Vector Search with an approximate nearest neighbor index. You need to update the index with new data every hour. The updates must be available for queries immediately. Which update method should you use?

Accepted Answer

Streaming updates using the streaming API.. Vertex AI Vector Search supports streaming updates via its streaming API, which allows you to insert, update, or delete vectors in real time. This ensures that new data is immediately available for approximate nearest neighbor (ANN) queries without requiring index recreation or redeployment, meeting the requirement for hourly updates with instant query availability.

Answer

Recreate the index every hour using a scheduled job.

Answer

Batch update by creating a new index and deploying it.

Answer

Use a brute-force index that supports real-time updates.

Question 12

An ML team wants to deploy multiple models (e.g., a recommender and a classifier) behind a single Vertex AI endpoint. The models have different resource requirements: the recommender needs GPU, the classifier needs high memory. How should they configure the endpoint?

Accepted Answer

Deploy both models to the same endpoint with different machine types per deployed model.. Vertex AI allows deploying multiple models on the same endpoint, each with its own machine type and resources. Traffic splitting routes requests to the correct model.

Answer

Use Cloud Run for one model and Vertex AI for the other.

Answer

Use a single machine type that meets the highest requirements.

Answer

Create separate endpoints for each model.

Question 13

You need to run a batch prediction job on Vertex AI using a model that requires custom preprocessing using a Python script. The preprocessing must be applied before inference. Which approach should you use?

Accepted Answer

Use Dataflow to preprocess the data and write the results to BigQuery or GCS, then launch a batch prediction job on the preprocessed data.. Dataflow (Apache Beam) is the recommended serverless service for distributed, scalable preprocessing of large datasets on Google Cloud. It can read raw data from GCS, apply custom Python preprocessing logic, and write the preprocessed results to GCS or BigQuery. The batch prediction job then reads the preprocessed data directly, avoiding the need to embed preprocessing in the prediction container or handle data on a single VM.

Answer

Use Cloud Functions to preprocess each record individually.

Answer

Preprocess the data on a single VM and then upload to GCS.

Answer

Include the preprocessing logic in the custom container used for batch prediction.

Question 14

A company is deploying a model on Vertex AI for online predictions with strict latency SLOs. The model requires GPU acceleration. Which TWO configurations should they consider to meet the SLOs while optimizing cost?

Accepted Answer

Set min_replica_count to handle base traffic and max_replica_count to handle spikes. Enabling autoscaling with min_replica_count to handle base load and max_replica_count for spikes, and using scale-to-zero for non-production (but for production, scale-to-zero may not meet SLOs due to cold starts; however, the question says 'optimizing cost', so scale-to-zero is not appropriate for low latency. The correct answers are: set appropriate min and max replicas, and use GPU-enabled machine types. The other options are either irrelevant or counterproductive.

Answer

Use n1-highmem-32 machine types without GPU

Answer

Enable autoscaling with min_replica_count=0 and max_replica_count=10

Answer

Disable autoscaling and set a fixed number of replicas equal to peak load

Question 15

You are designing a batch prediction pipeline using Vertex AI. The input data is 100 TB of images stored in Cloud Storage. The model is a custom TensorFlow model that expects TFRecord format. The pipeline must be cost-effective and run within a time window of 2 hours. Which THREE steps should you include?

Accepted Answer

Create a Vertex AI batch prediction job with input from GCS (TFRecord files).. Vertex AI batch prediction jobs natively accept TFRecord files stored in Cloud Storage as input, which aligns with the requirement for a custom TensorFlow model. This approach is cost-effective and can complete within 2 hours by leveraging Vertex AI's managed infrastructure, avoiding the need to spin up and manage compute resources manually.

Answer

Store batch prediction results in BigQuery.

Answer

Use Cloud Functions to convert images to TFRecord.