A company is deploying a GenAI system that generates product descriptions. During A/B testing, the new system shows a 20% increase in click-through rate (CTR) but a 15% increase in average cost per query due to the model size. The team wants to optimize cost without sacrificing the CTR gain. Which THREE actions should they take? (Choose three.)
Batching reduces the number of API calls and can lower cost.
Why this answer
Batching similar requests reduces the per-request overhead by combining multiple inference calls into a single batch, which amortizes the fixed costs (e.g., model loading, token processing) across more outputs. This directly lowers the average cost per query while preserving the model architecture and CTR gains, as the model's output quality remains unchanged.
Exam trap
Google often tests the misconception that adding more few-shot examples always improves output quality, but in reality, it increases token costs and can degrade performance due to context window limits or irrelevant examples.