hardMultiple ChoiceObjective-mapped
Reduce Vertex AI Endpoint Latency — CPU Preprocessing Bottleneck
Your company runs a real-time recommendation system for a popular e-commerce website using a machine learning model deployed on Vertex AI Endpoints. The model takes user features and product catalog data as input and returns top-10 product recommendations. The system uses a feature store to serve user embeddings and product embeddings. Recently, the recommender team retrained the model with a new algorithm and deployed it as a new version. Since the deployment, the latency for recommendation requests has increased from 100ms to 500ms on average, exceeding the 200ms SLO. The model accuracy is acceptable, and there are no errors. The endpoint uses an n1-standard-8 machine with a single GPU. The new model is larger but still fits on the GPU. You investigate and find that the GPU utilization remains low (<20%), but CPU utilization is high (90%). What should you do to reduce latency while maintaining accuracy?
Quick Answer
The answer is to increase the number of replicas (nodes) to parallelize the CPU-bound preprocessing work. This is correct because the high CPU utilization at 90% with low GPU utilization below 20% clearly indicates a CPU preprocessing bottleneck, not a GPU inference limitation; the new model’s larger size has shifted the workload to CPU-intensive feature transformations, so distributing requests across multiple replicas reduces per-node CPU pressure and lowers latency without altering model accuracy. On the Google Professional Data Engineer exam, this scenario tests your ability to diagnose scaling bottlenecks by reading utilization metrics—a common trap is to assume a larger model always needs a bigger GPU, but the real fix is horizontal scaling for CPU-bound preprocessing. Remember the memory tip: when GPU is idle but CPU is maxed, think “preprocessing is the bottleneck—scale out, not up.”
⚠ Common exam trap
Google Cloud often tests the misconception that GPU utilization must be increased to reduce latency, but the trap here is that the bottleneck is CPU-bound preprocessing, not GPU inference, so scaling replicas (horizontal scaling) is the correct fix, not GPU upgrades or batching.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Increase the number of replicas (nodes) to parallelize the CPU-bound preprocessing work.
The high CPU utilization (90%) with low GPU utilization (<20%) indicates that the bottleneck is CPU-bound preprocessing, not GPU inference. Increasing the number of replicas (nodes) distributes the CPU preprocessing load across multiple instances, reducing per-request latency without affecting model accuracy. This directly addresses the root cause while keeping the existing GPU resources.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Upgrade the machine type to one with more GPU memory (e.g., n1-standard-8 with a larger GPU) to reduce model inference time.
Why it's wrong here
GPU memory is not the issue; GPU utilization is low, so inference is fast. More GPU memory won't help CPU bottleneck.
- ✗
Change the batch size in the model serving code to process multiple requests together, improving GPU utilization.
Why it's wrong here
Batching increases latency for individual requests as they wait for a batch to fill. It would not reduce CPU bottleneck and could hurt latency SLO.
- ✓
Increase the number of replicas (nodes) to parallelize the CPU-bound preprocessing work.
Why this is correct
Adding more nodes will distribute the preprocessing load across multiple CPUs, reducing the overall latency per request if the load balancer dispatches requests efficiently. However, this increases cost.
- ✗
Offload preprocessing to a dedicated Cloud Run service that runs asynchronously and returns precomputed feature vectors.
Why it's wrong here
Asynchronous preprocessing would not help if the model needs features inline; synchronous processing is required. Offloading could add network latency.
Go deeper
Related to this question
About these practice questions
One of 890 original PDE practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on PDE
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. You deployed a model on Vertex AI Endpoints using a custom container. The model serves predictions but the latency is higher than expected. You suspect the container is not making full use of the CPU resources. What should you do to reduce latency?
medium- ✓ A.Modify the container to use multi-threading or increase the number of workers in the prediction server (e.g., Gunicorn workers).
- B.Enable response caching on the endpoint.
- C.Change the machine type to a GPU-accelerated machine.
- D.Increase the number of nodes by adjusting autoscaling limits.
Why A: High latency in a CPU-based custom container often stems from underutilizing available CPU cores. By increasing the number of workers (e.g., Gunicorn workers) or enabling multi-threading, you allow the prediction server to handle multiple requests concurrently, reducing queue time and improving throughput. This directly addresses the symptom of the container not making full use of CPU resources.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.