Google Cloud's Generative AI Offerings →hardMultiple Select

Deploying Custom Models for Low Latency on Vertex AI

Which THREE considerations are critical when deploying a generative AI model using Vertex AI Endpoints for a latency-sensitive application? (Choose THREE.)

⚠ Common exam trap

Google Cloud often tests the distinction between configuration choices that affect latency (GPU type, autoscaling, model size) versus operational or lifecycle management choices (version count, manual instance count) that do not directly impact per-request response time.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Model size and architecture

Model size and architecture directly impact inference latency because larger models with more parameters require more computation per request. For latency-sensitive applications, choosing a smaller or distilled model (e.g., Gemma 2B vs. 27B) or using quantization can reduce response times. Vertex AI Endpoints serve the model as-is, so the model's inherent computational cost is the primary driver of per-request latency.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Model size and architecture
Why this is correct
Larger models introduce higher latency.
✗
Number of model versions
Why it's wrong here
Model versioning does not directly affect latency.
✓
GPU type and number
Why this is correct
GPU selection impacts inference speed.
✓
Autoscaling configuration
Why this is correct
Proper autoscaling ensures low latency under varying load.
✗
Number of model instances
Why it's wrong here
While important, autoscaling handles instance count dynamically.

Go deeper

Deploying Custom Models for Low Latency on Vertex AI

Why each option matters

Option-by-option breakdown

Related to this question