AIF-C01 Fundamentals of Generative AI Practice Question

A company is using Amazon SageMaker JumpStart to deploy a pre-trained text generation model. After deployment, the model produces slow inference responses. Which action is most likely to improve inference latency?

⚠ Common exam trap

The AIF-C01 exam often tests the misconception that model optimization techniques like quantization always improve latency without trade-offs, but the most direct and reliable method for reducing inference latency is upgrading to a more powerful instance type with higher GPU memory.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Deploy the model on a more powerful instance type with higher GPU memory.

Deploying the model on a more powerful instance type with higher GPU memory directly addresses the computational bottleneck causing slow inference. A larger GPU provides more CUDA cores and memory bandwidth, enabling faster matrix operations and reducing the time per forward pass for the pre-trained text generation model.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Quantize the model weights to FP16 or INT8.
Why it's wrong here
Quantization can reduce latency but may also reduce accuracy. It is not always the most straightforward fix.
✓
Deploy the model on a more powerful instance type with higher GPU memory.
Why this is correct
More compute resources reduce inference time per request.
✗
Fine-tune the model on a smaller dataset.
Why it's wrong here
Fine-tuning does not affect inference speed.
✗
Increase the batch size for inference requests.
Why it's wrong here
Larger batch sizes improve throughput but may increase latency for individual requests.

Go deeper

AIF-C01 Fundamentals of Generative AI Practice Question

Why each option matters

Option-by-option breakdown

Related to this question