The answer is to add a NVIDIA T4 GPU accelerator to the existing machine type. This directly addresses the GPU acceleration inference latency Vertex AI bottleneck because the TensorFlow and PyTorch sub-models in the ensemble are deep learning models that benefit massively from parallelized GPU compute, while the XGBoost component runs efficiently on the existing CPU. On the Google Professional Machine Learning Engineer exam, this scenario tests your understanding of when to use accelerators versus scaling vertically—a common trap is to increase CPU cores, but that only marginally helps when the bottleneck is matrix operations in neural networks. The key insight is that GPU acceleration reduces inference latency on Vertex AI without the cost explosion of adding many high-CPU machines, especially for spiky traffic where a single GPU-equipped node can handle bursts. Memory tip: think “GPU for deep nets, CPU for trees” to avoid over-provisioning the wrong resource.
PMLE Serving and scaling models Practice Question
This PMLE practice question tests your understanding of serving and scaling models. Read the scenario carefully and evaluate each option against the stated constraints before committing to an answer. After answering, compare your reasoning against the explanation and wrong-answer breakdown below. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.
You are a machine learning engineer at a financial technology company. You have deployed a complex ensemble model consisting of three sub-models (XGBoost, TensorFlow, and PyTorch) for real-time fraud detection. The model is served on Vertex AI online prediction with a custom container that orchestrates the three models sequentially. The endpoint currently uses n1-highmem-8 machines with no accelerators. You are experiencing high latency (avg 500ms) during peak trading hours (9:30 AM - 4:00 PM EST), exceeding the 200ms SLA. The container is CPU-bound, and memory usage is around 60%. The model weights total 500 MB. You have already tried increasing the batch size per request from 1 to 4, which reduced latency slightly but not enough. The traffic pattern is very spiky, with sudden bursts of up to 1000 requests per second. Your goal is to meet the latency SLA without significantly increasing cost. Which action should you take?
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Add a NVIDIA T4 GPU accelerator to the existing machine type.
Adding a GPU accelerator (e.g., NVIDIA T4) to the instances can significantly speed up the TensorFlow and PyTorch components, which are deep learning models. The XGBoost part runs on CPU but the overall latency bottleneck is likely the deep learning models. GPU will accelerate inference of those models, reducing total latency. Increasing CPUs will help only marginally as the main bottleneck is compute. Reducing min replicas may increase cold start and queue. Switching to batch prediction changes the model from real-time to batch, which does not meet the latency requirement.
Key principle: NAT direction and interface roles matter as much as the IP address mapping. Inside/outside designation controls which traffic is translated.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
✓
Add a NVIDIA T4 GPU accelerator to the existing machine type.
Why this is correct
GPU accelerates the deep learning parts, reducing total latency.
Related concept
Static NAT maps one inside address to one outside address.
✗
Reduce the min_replica_count to 0 to allow scaling down aggressively and add more replicas during spikes.
Why it's wrong here
min=0 increases cold start latency, not reducing overall latency.
✗
Increase the machine type to n1-highmem-16 with more vCPUs.
Why it's wrong here
More CPU may help but the deep learning models will still be slow on CPU.
✗
Switch the model to Vertex AI batch prediction and run predictions every hour.
Why it's wrong here
Batch prediction does not meet real-time 200ms SLA.
Common exam traps
Common exam trap: NAT rules depend on direction and matching traffic
NAT is not only about the public address. The inside/outside interface roles and the ACL or rule that matches traffic are just as important.
Detailed technical explanation
How to think about this question
NAT questions usually test address translation, overload/PAT behaviour, static mappings and whether the right traffic is being translated. Read the interface direction and address terms carefully.
KKey Concepts to Remember
Static NAT maps one inside address to one outside address.
PAT allows many inside hosts to share one public address using ports.
Inside local and inside global describe the private and translated addresses.
NAT ACLs identify traffic for translation, not always security filtering.
TExam Day Tips
→Identify inside and outside interfaces first.
→Check whether the scenario needs static NAT, dynamic NAT or PAT.
→Do not confuse NAT matching ACLs with normal packet-filtering intent.
Key takeaway
NAT direction and interface roles matter as much as the IP address mapping. Inside/outside designation controls which traffic is translated.
Real-world example
How this comes up in practice
A startup's cloud architect reviews their monthly bill and notices costs are higher than expected for a long-running batch job. Switching from on-demand instances to Reserved Instances — or using Spot/Preemptible VMs — can reduce compute costs by up to 72 %. Questions like this test whether you understand the tradeoffs between commitment, flexibility, and cost across cloud pricing models.
What to study next
Got this wrong? Here's your next step.
Review the four NAT address types (inside local, inside global, outside local, outside global), PAT port overload, and static vs dynamic NAT use cases. Then practise related PMLE NAT questions on configuration and troubleshooting.
Serving and scaling models — This question tests Serving and scaling models — Static NAT maps one inside address to one outside address..
What is the correct answer to this question?
The correct answer is: Add a NVIDIA T4 GPU accelerator to the existing machine type. — Adding a GPU accelerator (e.g., NVIDIA T4) to the instances can significantly speed up the TensorFlow and PyTorch components, which are deep learning models. The XGBoost part runs on CPU but the overall latency bottleneck is likely the deep learning models. GPU will accelerate inference of those models, reducing total latency. Increasing CPUs will help only marginally as the main bottleneck is compute. Reducing min replicas may increase cold start and queue. Switching to batch prediction changes the model from real-time to batch, which does not meet the latency requirement.
What should I do if I get this PMLE question wrong?
Review the four NAT address types (inside local, inside global, outside local, outside global), PAT port overload, and static vs dynamic NAT use cases. Then practise related PMLE NAT questions on configuration and troubleshooting.
What is the key concept behind this question?
Static NAT maps one inside address to one outside address.
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company deploys a custom TensorFlow model to Vertex AI Endpoint for online predictions. After deployment, prediction latency is consistently high (over 500ms) even under low traffic. The model is CPU-only and the default machine type (n1-standard-2) is used. Which action will most likely reduce prediction latency?
medium
A.Increase the max_replica_count to 10 to allow more parallel requests.
✓ B.Change the machine type to n1-highcpu-16 with a GPU accelerator.
C.Set min_replica_count to 3 to ensure always-on capacity.
D.Increase the batch size in the prediction request.
Why B: Option A is correct because using a machine type with more CPUs or adding a GPU accelerator can reduce inference time for compute-intensive models. Option B is wrong because increasing max replicas does not improve single-request latency. Option C is wrong because batch size affects throughput, not latency per request. Option D is wrong because increasing min replicas reduces cold start but not steady-state latency.
Last reviewed: Jun 24, 2026
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
This PMLE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PMLE exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.