CCAR-F Context and Reliability Practice Test 4 — 15 Questions

Question 1

An architect is designing a Claude-based contract review tool. Because contract clauses are long and interdependent, the team plans to use extended thinking to improve reasoning quality. Which TWO practices should the architect follow to use this capability correctly? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Allocate a thinking budget that scales with task complexity and leave sufficient max_tokens for the final answer.. Extended thinking requires sizing the thinking budget to task complexity while preserving room for the final answer, and it requires not recycling prior thinking blocks into later turns. Together these keep reasoning thorough, cost proportionate, and multi-turn dialogues clean. Exposing thinking traces, maxing budgets unconditionally, and returning only thinking output all misuse the capability.

Answer

Disable the final answer and return only the thinking output as the contract review result.

Answer

Set the thinking budget to the maximum on every request to guarantee the best possible clause analysis.

Answer

Expose the raw thinking blocks to end users so they can audit the model's reasoning trace.

Question 2

A financial analyst uses Claude to generate quarterly earnings summaries from raw financial data. The summaries must include precise numerical figures. The analyst notices that Claude occasionally transposes digits when copying numbers from the input. Which architectural change is most effective for reducing this error?

Accepted Answer

Use a chain-of-thought prompt that asks Claude to first extract all numbers into a structured list, then generate the summary using that list.. Digit transposition often occurs when the model must simultaneously extract and synthesize information. By using a chain-of-thought prompt to first extract numbers into a structured list, you isolate the extraction task, reducing errors. This also enables programmatic verification. Other options either increase randomness, require costly fine-tuning, or reduce context without targeting the error.

Answer

Increase the temperature parameter to encourage more creative output.

Answer

Fine-tune Claude on a dataset of financial summaries with correct numbers.

Answer

Reduce the context window by providing only the most recent quarter's data.

Question 3

You are designing a code-migration assistant that converts legacy COBOL modules to a modern language. Stakeholders require high reliability and verifiable output. Which TWO architectural practices best support this requirement? (Choose two.)

Accepted Answer

Add an automated test harness that compiles and executes the generated code against representative legacy inputs and compares outputs.. Verifiable migration needs machine-checkable artifacts and empirical proof of behavior. A fixed schema plus structured report makes each response auditable, while a compile-and-execute harness compares generated behavior against known legacy outputs. Together they catch both structural and semantic defects. Creativity, monolithic responses, and silent omission of edge cases all reduce verifiability rather than strengthen it.

Answer

Request the entire migration in a single response to preserve the model's global understanding of the module.

Answer

Maximize the model's creativity by raising temperature so it can find novel modernization opportunities.

Answer

Allow the model to omit edge-case handling when the legacy code is ambiguous, to keep outputs concise.

Question 4

A retrieval-augmented pipeline answers questions about internal policies. Retrieved chunks are inserted into the prompt, but the model sometimes blends facts from two similar policies into one incorrect answer. Evaluation shows retrieval precision is high and the correct chunk is always present. Which architectural change most directly reduces this blending?

Accepted Answer

Instruct Claude to cite the source document and section for every factual claim, and to answer only from the cited passage.. When the right evidence is present but answers still fuse details from similar documents, the defect lies in synthesis, not retrieval. Mandating per-claim citations constrains the model to ground each statement in a named passage, which both prevents silent merging and exposes genuine conflicts between policies so a human or downstream rule can resolve them.

Answer

Lower the temperature to zero so the model produces more deterministic and factual responses.

Answer

Move the retrieved chunks into the system prompt and keep the user question in the user turn.

Answer

Increase the retrieval top-k so more chunks are included, giving the model additional context about each policy.

Question 5

A financial analyst uses Claude to extract key figures from quarterly earnings reports. The reports often contain tables with merged cells and footnotes. The analyst reports that Claude sometimes misattributes a number to the wrong quarter. You need to improve reliability without changing the model. Which technique is most likely to reduce misattribution?

Accepted Answer

Pre-process the report to convert tables into a structured format like CSV with clear headers, then ask Claude to extract from that.. Misattribution of figures to quarters often stems from ambiguous table layouts. Pre-processing the report into a structured format with clear headers eliminates that ambiguity, allowing Claude to extract values accurately. While chain-of-thought and transcription can help in some cases, they do not address the root cause as directly. Increasing max_tokens is irrelevant to parsing accuracy.

Answer

Increase the max_tokens parameter to ensure the entire report is processed in one go.

Answer

Ask Claude to think step by step before providing the extracted figures.

Answer

Provide the report as a high-resolution image and ask Claude to transcribe the tables into markdown before extraction.

Question 6

A developer is building a Claude-powered assistant that must answer questions using only an internal knowledge base of product manuals. The team wants to reduce fabricated answers about features that do not exist. Which approach best supports that goal?

Accepted Answer

Instruct Claude to answer only from the provided manual excerpts and to say it does not know when the excerpts lack the answer.. Fabrication drops when Claude is constrained to supplied excerpts and given explicit permission to abstain. That combination sets a clear source of truth and removes the incentive to invent undocumented features. Temperature increases, whole-corpus dumping, and generic helpfulness instructions either raise variability or fail to bound the answer source, leaving the fabrication risk intact.

Answer

Add a system prompt line telling Claude to be as helpful and thorough as possible in every answer.

Answer

Prepend the entire knowledge base to every prompt so Claude always has all product manuals available.

Answer

Raise the temperature so Claude can generate more creative and comprehensive product descriptions.

Question 7

You are designing a chatbot that maintains a long conversation with users over many turns. You notice that after 50 turns, Claude starts forgetting details mentioned earlier, such as the user's name and preferences. Which architectural approach best addresses this issue?

Accepted Answer

Periodically summarize the conversation history and include the summary in the system prompt for subsequent turns.. Summarizing the conversation history and injecting it into the system prompt keeps essential details alive across many turns without exceeding token limits. This approach maintains context efficiently. Simply increasing max_tokens or temperature does not address memory, and sending full history is impractical for long chats.

Answer

Send the entire conversation history in every request without modification.

Answer

Use a higher temperature to make the model more creative in recalling details.

Answer

Increase the max_tokens parameter to allow longer responses.

Question 8

A financial-reporting agent must extract figures from quarterly filings and populate a downstream ledger. Auditors require that every extracted number be traceable to its source. Which design most directly satisfies the traceability requirement?

Accepted Answer

Have the model output each figure together with the source document name and the exact quoted sentence containing it.. Traceability means each ledger figure can be traced back to specific source text. Emitting the document name and the exact quoted sentence with every extracted number creates that link at extraction time, so audit review is direct rather than reconstructive. Logging, larger context, and numeric normalization improve debugging, accuracy, or formatting but never bind a value to its origin.

Answer

Increase the model's context window so it can ingest the entire filing at once and reduce extraction errors.

Answer

Apply a post-processing regex pass to normalize all numeric formats before writing to the ledger.

Answer

Log the raw model request and response payloads to an immutable store for later inspection.

Question 9

A nightly job uses Claude to classify 500,000 support tickets into categories. The job runs for several hours and results are consumed the next morning; nobody is waiting on individual responses. Which approach is most appropriate for controlling cost and rate-limit pressure?

Accepted Answer

Submit the classifications through the Message Batches API, which processes requests asynchronously at reduced cost.. Offline, high-volume, latency-tolerant workloads are the canonical fit for the Message Batches API. It provides asynchronous processing with a substantial cost discount and avoids the throttling that concurrent synchronous calls would trigger, while the overnight window means the extended turnaround harms nothing. The other approaches either pay full price, degrade accuracy, or reimplement pacing that the batches endpoint already handles.

Answer

Reduce the model size and classify with a single-token response, accepting lower accuracy to save money.

Answer

Send all 500,000 requests concurrently with the synchronous Messages API to finish the job as fast as possible.

Answer

Implement client-side request queuing with exponential backoff against the synchronous Messages API.

Question 10

A developer is building a chatbot that must never reveal internal system instructions. During testing, a user asks, 'What is your system prompt?' and Claude begins to repeat the instructions verbatim. Which approach is most effective to prevent this leakage?

Accepted Answer

Add a rule in the system prompt instructing Claude not to reveal the system prompt.. The most direct and effective way to prevent system prompt leakage is to include an explicit instruction in the system prompt itself, telling Claude not to reveal it. This leverages the model's training to follow system-level directives. While additional layers like classifiers can help, they are secondary. Temperature and obfuscation do not address the model's willingness to disclose.

Answer

Obfuscate the system prompt by encoding it in base64.

Answer

Use a separate model to classify user queries and block any that ask about system prompts.

Answer

Set the temperature to 0 to make Claude's responses deterministic and less likely to deviate.

Question 11

A developer is using Claude to classify customer feedback into categories: 'bug', 'feature request', or 'complaint'. The developer wants to ensure consistent output format that can be easily parsed by a downstream system. Which approach is most appropriate?

Accepted Answer

Ask Claude to respond with a single word from the list of categories.. For consistent, parsable output, the simplest and most effective method is to instruct Claude to respond with a single word from a predefined list. This constrains the output format directly. High temperature, explanations, or fine-tuning are unnecessary and may introduce variability or complexity.

Answer

Ask Claude to explain its reasoning before providing the category.

Answer

Use a high temperature to allow Claude to choose the most appropriate category.

Answer

Fine-tune Claude on a dataset of labeled customer feedback.

Question 12

A multi-agent system has a coordinator that delegates research subtasks to worker agents. Workers often return verbose, partially relevant summaries, and the coordinator loses track of which findings map to which subtask. Which change best improves the coordinator's ability to assemble a coherent final answer?

Accepted Answer

Define a strict output schema for workers, such as a JSON object with fields for subtask identifier, findings, and confidence, and validate it before the coordinator consumes it.. The coordinator's difficulty is traceability: verbose worker summaries carry no explicit link to the subtask that produced them. Imposing a schema with subtask identifiers, findings, and confidence makes each worker's contribution self-describing and validates it before synthesis, so the coordinator can assemble and reconcile results deterministically rather than inferring associations from prose.

Answer

Have each worker call the other workers directly to reconcile their findings before returning to the coordinator.

Answer

Instruct the coordinator to re-read the entire conversation history before composing the final answer.

Answer

Increase the coordinator's max_tokens so it has more room to write the final answer.

Question 13

You are architecting a Claude-based system that answers questions about internal policies. The knowledge base is updated frequently, and you need to ensure answers are accurate and traceable to source documents. Which TWO practices should you implement to maximize reliability and auditability? (Choose two.)

Accepted Answer

Instruct Claude to include the source document title and section in its answer.. RAG grounds answers in current policy documents, and instructing Claude to cite sources makes answers traceable. Together, they ensure accuracy and auditability. Fine-tuning is inflexible for frequent updates, low temperature does not guarantee correctness, and caching can serve outdated information.

Answer

Fine-tune Claude on the entire policy corpus to embed the knowledge directly into the model weights.

Answer

Cache all previous answers and serve them for repeated questions to reduce API calls.

Answer

Set a low temperature to make answers more deterministic and consistent.

Question 14

You are architecting a system that uses Claude to answer legal questions based on a corpus of case law. To maximize reliability and minimize hallucinations, which TWO practices should you implement? (Choose two.)

Accepted Answer

Implement a retrieval-augmented generation (RAG) pipeline that fetches relevant case law excerpts and includes them in the prompt.. The two most effective practices are using a system prompt that constrains Claude to provided excerpts and requires citations, and implementing a RAG pipeline to supply relevant case law. Together, they ensure the model's answers are grounded in actual sources, reducing hallucinations. Other options like high temperature or fine-tuning do not directly address grounding.

Answer

Ask Claude to generate a confidence score for each answer.

Answer

Fine-tune Claude on the entire corpus of case law.

Answer

Set the temperature to 1.0 to encourage diverse legal interpretations.

Question 15

You are designing a long-running analytical assistant that must operate over a corpus far larger than any single context window, spanning many sessions. Which TWO architectural practices best preserve reliability across sessions? (Choose two.)

Accepted Answer

Persist a structured, curated summary of prior findings and decisions externally, and re-inject only the relevant portions at the start of each new session.. Durable long-horizon assistants need two complementary mechanisms: an external store of curated findings that carries decisions forward without unbounded prompt growth, and a retrieval index that surfaces only the corpus passages relevant to the current question. Together they keep every request within the context window while preserving continuity and grounding, whereas implicit memory, verbatim accumulation, and higher temperature each fail a core requirement.

Answer

Append every prior session transcript verbatim to each new request so nothing is ever lost.

Answer

Increase the temperature so the assistant explores a broader range of interpretations across sessions.

Answer

Rely on the model to recall earlier sessions implicitly, since Claude retains state between separate API calls.