hardMultiple ChoiceObjective-mapped
PDE Practice Question: A company has a model that requires GPU for…
A company has a model that requires GPU for inference and has strict latency requirements. They deployed on Vertex AI Endpoint with autoscaling but observe cold start latency when scaling up. What is the best solution?
⚠ Common exam trap
Many exam-takers confuse inference optimization techniques (like TensorRT or larger GPUs) with infrastructure-level scaling configurations, failing to recognize that cold start is a provisioning delay, not a compute performance issue.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Set a higher min_replica_count to keep instances warm
Setting a higher min_replica_count ensures that a baseline number of GPU instances are always running and ready to serve inference requests, eliminating cold start latency because new instances do not need to be provisioned and loaded from scratch when traffic spikes. This directly addresses the autoscaling-induced cold start issue by maintaining a warm pool of replicas.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Set a higher min_replica_count to keep instances warm
Why this is correct
Keeping a minimum number of instances online avoids cold starts when traffic spikes.
- ✗
Pre-compile the model with TensorRT
Why it's wrong here
TensorRT optimizes inference but does not address cold start latency during scaling.
- ✗
Use a larger GPU instance
Why it's wrong here
Larger GPU does not reduce the time to initialize new instances.
- ✗
Switch to batch prediction
Why it's wrong here
Batch prediction is not for real-time online predictions.
Go deeper
Related to this question
About these practice questions
One of 890 original PDE practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.