A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?
Trap 1: Use a larger foundation model with a longer context window and…
Pasting all documents into every prompt is expensive, hits context limits for large document sets, and does not scale as the document library grows.
Trap 2: Fine-tune a base LLM on the policy documents monthly
Fine-tuning is expensive and time-consuming; monthly cycles are impractical and fine-tuned knowledge becomes stale immediately after cutoff.
Trap 3: Train a custom model from scratch on the policy documents each month
Training from scratch requires massive compute resources and weeks of time — entirely disproportionate for monthly document updates.
- A
Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store
RAG retrieves relevant document chunks at query time, ensuring the chatbot always answers from the latest uploaded documents without any model retraining.
- B
Use a larger foundation model with a longer context window and paste all documents into each prompt
Why wrong: Pasting all documents into every prompt is expensive, hits context limits for large document sets, and does not scale as the document library grows.
- C
Fine-tune a base LLM on the policy documents monthly
Why wrong: Fine-tuning is expensive and time-consuming; monthly cycles are impractical and fine-tuned knowledge becomes stale immediately after cutoff.
- D
Train a custom model from scratch on the policy documents each month
Why wrong: Training from scratch requires massive compute resources and weeks of time — entirely disproportionate for monthly document updates.