AIF-C01 Applications of Foundation Models Practice Question

A company runs a chatbot using a large language model on Amazon Bedrock. They notice high latency during peak hours. Which action would be MOST effective to reduce latency without degrading response quality?

⚠ Common exam trap

AWS often tests the misconception that reducing model size or output length is the primary way to reduce latency, but the real bottleneck in peak-hour scenarios is often infrastructure contention, which Provisioned Throughput resolves without sacrificing quality.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Use Provisioned Throughput for model inference

Provisioned Throughput on Amazon Bedrock reserves dedicated capacity for model inference, ensuring consistent low latency even during peak hours. This eliminates the variability caused by resource contention in the on-demand tier, directly addressing high latency without altering model size or output quality.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Increase the number of concurrent invocations
Why it's wrong here
More concurrent calls could overload the model, increasing latency.
✗
Switch to a smaller model
Why it's wrong here
Switching to a smaller model may reduce quality; it's not guaranteed to be the best approach.
✗
Decrease the maxTokens parameter
Why it's wrong here
Reducing maxTokens may truncate responses, affecting quality.
✓
Use Provisioned Throughput for model inference
Why this is correct
Provisioned Throughput ensures reserved capacity, reducing latency variability.

Go deeper

AIF-C01 Applications of Foundation Models Practice Question

Why each option matters

Option-by-option breakdown

Related to this question