Question 337 of 990
hardMultiple ChoiceObjective-mapped
GPU Acceleration to Reduce Inference Latency on Vertex AI — Ensemble Models
You are a machine learning engineer at a financial technology company. You have deployed a complex ensemble model consisting of three sub-models (XGBoost, TensorFlow, and PyTorch) for real-time fraud detection. The model is served on Vertex AI online prediction with a custom container that orchestrates the three models sequentially. The endpoint currently uses n1-highmem-8 machines with no accelerators. You are experiencing high latency (avg 500ms) during peak trading hours (9:30 AM - 4:00 PM EST), exceeding the 200ms SLA. The container is CPU-bound, and memory usage is around 60%. The model weights total 500 MB. You have already tried increasing the batch size per request from 1 to 4, which reduced latency slightly but not enough. The traffic pattern is very spiky, with sudden bursts of up to 1000 requests per second. Your goal is to meet the latency SLA without significantly increasing cost. Which action should you take?
Quick Answer
The answer is to add a NVIDIA T4 GPU accelerator to the existing machine type. This directly addresses the GPU acceleration inference latency Vertex AI bottleneck because the TensorFlow and PyTorch sub-models in the ensemble are deep learning models that benefit massively from parallelized GPU compute, while the XGBoost component runs efficiently on the existing CPU. On the Google Professional Machine Learning Engineer exam, this scenario tests your understanding of when to use accelerators versus scaling vertically—a common trap is to increase CPU cores, but that only marginally helps when the bottleneck is matrix operations in neural networks. The key insight is that GPU acceleration reduces inference latency on Vertex AI without the cost explosion of adding many high-CPU machines, especially for spiky traffic where a single GPU-equipped node can handle bursts. Memory tip: think “GPU for deep nets, CPU for trees” to avoid over-provisioning the wrong resource.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Add a NVIDIA T4 GPU accelerator to the existing machine type.
Adding a GPU accelerator (e.g., NVIDIA T4) to the instances can significantly speed up the TensorFlow and PyTorch components, which are deep learning models. The XGBoost part runs on CPU but the overall latency bottleneck is likely the deep learning models. GPU will accelerate inference of those models, reducing total latency. Increasing CPUs will help only marginally as the main bottleneck is compute. Reducing min replicas may increase cold start and queue. Switching to batch prediction changes the model from real-time to batch, which does not meet the latency requirement.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Add a NVIDIA T4 GPU accelerator to the existing machine type.
Why this is correct
GPU accelerates the deep learning parts, reducing total latency.
- ✗
Reduce the min_replica_count to 0 to allow scaling down aggressively and add more replicas during spikes.
Why it's wrong here
min=0 increases cold start latency, not reducing overall latency.
- ✗
Increase the machine type to n1-highmem-16 with more vCPUs.
Why it's wrong here
More CPU may help but the deep learning models will still be slow on CPU.
- ✗
Switch the model to Vertex AI batch prediction and run predictions every hour.
Why it's wrong here
Batch prediction does not meet real-time 200ms SLA.
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on PMLE
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company deploys a custom TensorFlow model to Vertex AI Endpoint for online predictions. After deployment, prediction latency is consistently high (over 500ms) even under low traffic. The model is CPU-only and the default machine type (n1-standard-2) is used. Which action will most likely reduce prediction latency?
medium- A.Increase the max_replica_count to 10 to allow more parallel requests.
- ✓ B.Change the machine type to n1-highcpu-16 with a GPU accelerator.
- C.Set min_replica_count to 3 to ensure always-on capacity.
- D.Increase the batch size in the prediction request.
Why B: Changing the machine type to n1-highcpu-16 with a GPU accelerator provides significantly more compute resources for the custom TensorFlow model. The n1-highcpu-16 offers 16 vCPUs (vs. 2 in n1-standard-2), which reduces CPU-bound inference time, and adding a GPU accelerates matrix operations common in TensorFlow models, directly reducing latency per request. Option A is wrong because increasing max_replica_count allows more parallel requests but does not improve the processing time of a single request. Option C is wrong because setting min_replica_count ensures always-on capacity to avoid cold starts, but does not reduce steady-state latency. Option D is wrong because increasing batch size in the prediction request increases throughput by processing multiple inputs together, but does not reduce latency for a single prediction—it may actually increase the time to return a result for a given request.
Last reviewed: Jun 24, 2026
This PMLE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PMLE exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.