Business Strategies for Generative AI Solutions →hardMultiple ChoiceObjective-mapped

Generative AI Leader Practice Question: Business Strategies for Generative AI Solutions

Exhibit

Refer to the exhibit.
```json
{
  "deployment": {
    "machineType": "n1-highmem-16",
    "minReplicaCount": 1,
    "maxReplicaCount": 5,
    "accelerator": {
      "acceleratorType": "NVIDIA_TESLA_T4",
      "acceleratorCount": 1
    },
    "trafficSplit": {"default": 100}
  }
}
```

A company deployed a large language model on Vertex AI using the configuration shown in the exhibit. During peak usage, users report high latency. Which change is most likely to improve latency?

⚠ Common exam trap

The Generative AI Leader exam often tests the misconception that upgrading hardware (GPU memory or type) is the primary fix for latency, when in fact scaling out replicas is the more direct solution for handling concurrent request load.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Increase minReplicaCount to 3.

Increasing minReplicaCount to 3 ensures that at least three instances of the model are always running and ready to serve requests. This reduces cold-start latency and distributes the load across multiple replicas, directly addressing high latency during peak usage by providing more concurrent serving capacity.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Remove the accelerator to simplify deployment.
Why it's wrong here
Removing the GPU would make inference much slower.
✓
Increase minReplicaCount to 3.
Why this is correct
More replicas ready at all times reduces cold-start and scaling latency.
✗
Switch to a GPU with more memory, such as NVIDIA_TESLA_A100.
Why it's wrong here
GPU memory may not be the bottleneck; the T4 is sufficient for many models.
✗
Change machineType to n1-standard-4 to reduce cost.
Why it's wrong here
A less powerful machine will likely increase latency.

Go deeper

Generative AI Leader Practice Question: Business Strategies for Generative AI Solutions

Exhibit

Why each option matters

Option-by-option breakdown

Related to this question