Techniques to Improve Generative AI Model Output →mediumMultiple ChoiceObjective-mapped

Generative AI Leader Practice Question: Techniques to Improve Generative AI Model Output

Exhibit

Refer to the exhibit.

```
# Vertex AI Endpoint configuration
{
  "model": "gemini-1.5-pro",
  "endpoint": "projects/my-project/locations/us-central1/endpoints/123456789",
  "deployedModel": {
    "modelVersion": "1",
    "minReplicaCount": 1,
    "maxReplicaCount": 5,
    "autoscalingMetricSpecs": [
      {
        "metricName": "custom.googleapis.com|genai|request_count",
        "target": 100
      }
    ]
  }
}
```

Refer to the exhibit. The endpoint is experiencing high latency during traffic spikes. The team wants to improve response time by reducing queueing. Which change to the configuration would be most effective?

⚠ Common exam trap

Google Cloud often tests the misconception that lowering the autoscaling target metric (Option C) is the primary fix for high latency, when in fact the maxReplicaCount ceiling is the bottleneck that must be raised to allow sufficient capacity during spikes.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Increase maxReplicaCount to 10

Increasing maxReplicaCount to 10 allows the autoscaler to provision more replicas during traffic spikes, distributing the incoming requests across additional endpoints. This directly reduces queueing at each replica because the load is spread over more instances, lowering per-instance latency. The change targets the root cause—insufficient capacity to handle peak load—rather than adjusting thresholds or model versions.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Decrease minReplicaCount to 0
Why it's wrong here
Fewer minimum replicas can increase cold starts.
✗
Change the model version to '2'
Why it's wrong here
Model version doesn't affect latency directly.
✗
Decrease the target value in autoscaling metric to 50
Why it's wrong here
Lower target triggers scaling earlier but may not reduce queueing if max replicas are insufficient.
✓
Increase maxReplicaCount to 10
Why this is correct
More replicas handle higher load.

Go deeper

Generative AI Leader Practice Question: Techniques to Improve Generative AI Model Output

Exhibit

Why each option matters

Option-by-option breakdown

Related to this question