hardMultiple Choice
PDE Practice Question: A company has a model that requires GPU for…
A company has a model that requires GPU for inference and has strict latency requirements. They deployed on Vertex AI Endpoint with autoscaling but observe cold start latency when scaling up. What is the best solution?
⚠ Common exam trap
Many exam-takers confuse inference optimization techniques (like TensorRT or larger GPUs) with infrastructure-level scaling configurations, failing to recognize that cold start is a provisioning delay, not a compute performance issue.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Set a higher min_replica_count to keep instances warm
Setting a higher min_replica_count ensures that a baseline number of GPU instances are always running and ready to serve inference requests, eliminating cold start latency because new instances do not need to be provisioned and loaded from scratch when traffic spikes. This directly addresses the autoscaling-induced cold start issue by maintaining a warm pool of replicas.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Set a higher min_replica_count to keep instances warm
Why this is correct
Setting a higher min_replica_count keeps a baseline of GPU-backed instances running continuously, so scaling events add capacity without provisioning new nodes. This eliminates the cold start latency that violates the strict latency requirement, since requests never wait for instance initialisation.
- ✗
Pre-compile the model with TensorRT
Why it's wrong here
TensorRT reduces per-inference latency through kernel fusion and precision calibration, but it does not eliminate the endpoint's container image pull and model load during scale-up, which is what causes the cold start. It is tempting because it genuinely speeds up GPU inference, and would be right when steady-state latency, not scaling delay, is the bottleneck.
- ✗
Use a larger GPU instance
Why it's wrong here
A larger GPU instance increases per-replica capacity but does not remove cold start, since new replicas still initialise the model and runtime. It would be correct if the bottleneck were throughput per replica rather than scale-up latency.
- ✗
Switch to batch prediction
Why it's wrong here
Batch prediction processes asynchronous jobs and returns results after completion, so it cannot serve low-latency online inference. It suits large offline scoring workloads. Batch prediction would be correct if the requirement were periodic bulk scoring rather than real-time responses.
Go deeper
Related to this question
About these practice questions
One of 747 original PDE practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.