Business Strategies for Generative AI Solutions →mediumMultiple ChoiceObjective-mapped

Generative AI Leader Practice Question: Business Strategies for Generative AI Solutions

Exhibit

displayName: customer-support-endpoint
dedicatedEndpoint: false
machineType: n1-standard-4
minReplicaCount: 2
maxReplicaCount: 10
trafficSplit:
  - modelId: support-v1
    percentage: 100

Refer to the exhibit. A sudden surge of traffic reaches 15,000 requests per second, but the endpoint can only handle 1,000 req/s per replica. What will happen to new requests?

⚠ Common exam trap

Watch out — candidates often assume autoscaling can handle any traffic surge indefinitely, ignoring the hard limit of maxReplicaCount, and thus incorrectly choose Option A or D, failing to recognize that HTTP 429 is the standard throttling mechanism when capacity is exhausted.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

They will receive HTTP 429 (Too Many Requests) errors.

When a surge of 15,000 requests per second hits an endpoint configured with a maxReplicaCount (e.g., 10 replicas at 1,000 req/s each = 10,000 req/s capacity), any excess requests beyond that capacity are rejected with an HTTP 429 (Too Many Requests) status code. This is standard behavior in autoscaling systems: once the replica count reaches its maximum limit, the service cannot scale further, and new requests are throttled to prevent overload.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
They will be processed, and replicas will exceed maxReplicaCount.
Why it's wrong here
maxReplicaCount is a hard limit; replicas cannot exceed it.
✗
They will be redirected to a different model.
Why it's wrong here
No automatic redirection; traffic is only split as defined in trafficSplit.
✓
They will receive HTTP 429 (Too Many Requests) errors.
Why this is correct
Once max replicas are reached, new requests get a 429 status code.
✗
They will be queued until capacity becomes available.
Why it's wrong here
Vertex AI does not automatically queue requests; they are rejected when capacity is exceeded.

Go deeper

Generative AI Leader Practice Question: Business Strategies for Generative AI Solutions

Exhibit

Why each option matters

Option-by-option breakdown

Related to this question