CCAR-F Context and Reliability Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Fine-tune Claude on the entire policy corpus to embed the knowledge directly into the model weights.

  Why it's wrong here

  Fine-tuning embeds knowledge into the model but is costly, slow to update, and does not provide traceability to source documents. When policies change, you must re-fine-tune, which is impractical for frequently updated content. It also increases the risk of hallucinations because the model may not have seen the latest updates. RAG is better suited for dynamic knowledge bases.

• ✗

  Cache all previous answers and serve them for repeated questions to reduce API calls.

  Why it's wrong here

  Caching can improve performance and reduce cost, but it risks serving stale answers when policies are updated. It also does not ensure that answers are traceable to source documents unless the cache includes citations. In a frequently updated knowledge base, caching without invalidation can lead to inaccuracies. It is not a reliability or auditability best practice.

• ✓

  Instruct Claude to include the source document title and section in its answer.

  Why this is correct

  Requiring Claude to cite the source document and section enhances auditability by allowing users to verify the answer against the original policy. This practice also encourages the model to rely on the provided context rather than internal knowledge. It is a simple yet effective way to improve traceability and trust in the system, especially when combined with RAG.

• ✗

  Set a low temperature to make answers more deterministic and consistent.

  Why it's wrong here

  Lowering temperature reduces randomness but does not ensure accuracy or traceability. A deterministic wrong answer is still wrong. While it can be useful for consistency, it does not address the need to ground answers in current source documents or provide citations. It is not a primary practice for reliability in this scenario.

• ✓

  Use retrieval-augmented generation (RAG) to fetch relevant policy excerpts and include them in the prompt.

  Why this is correct

  RAG ensures that Claude's answers are grounded in the most current policy documents by retrieving relevant excerpts at query time. This reduces hallucinations and allows the model to cite sources. It also keeps the knowledge base external and updatable without retraining the model. Including retrieved excerpts in the prompt gives Claude the context needed to generate accurate, traceable responses.