A developer needs to reduce costs for a Bedrock application that processes high volumes of similar queries. The queries are repetitive and the model is invoked many times with the same prompt. Which cost optimization technique is MOST suitable?
Caching avoids recomputation for identical prompts, reducing latency and cost.
Why this answer
Model caching (prompt caching) stores responses to common prompts, reducing the need to recompute. Batch inference is for asynchronous processing, not real-time. Right-sizing the model helps but does not leverage query repetition.
Fine-tuning is for adapting the model, not cost savings on repetitive queries.