CCAR-F Practice Test 16 — 15 Questions

Question 1

You are building an agent that uses tool calls to interact with an external API. The agent is failing to select the correct tool. What is the most effective way to improve the agent's tool selection performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Improve the 'description' field for each tool to clearly state when it should be used.. For tool selection, the quality of the tool's name and description is paramount. Claude relies on these descriptions to understand the semantic intent of the tool. If the descriptions are vague or lack clear context, the model will struggle to map user requests to the appropriate tool. Improving the documentation within the tool definition is the single most effective architectural lever for improving tool-use accuracy.

Answer

Increase the system prompt length to include more examples of tool calls.

Answer

Combine multiple tools into one large tool with optional parameters.

Answer

Set the temperature to 1.0 to allow the model to explore different tool options.

Question 2

A developer notices that Claude occasionally provides different answers to the same technical question during automated testing. Which parameter should be adjusted to maximize the reliability and reproducibility of these responses?

Accepted Answer

Temperature. Reliability in LLM deployments often hinges on the trade-off between creativity and consistency. By lowering the temperature, architects ensure that Claude selects the most probable tokens, which reduces the likelihood of hallucinations in factual or technical contexts. This setting is a foundational lever for stabilizing application behavior across thousands of independent user sessions in production.

Answer

Top-p (Nucleus Sampling)

Answer

Max Tokens

Answer

Presence Penalty

Question 3

Which strategy most effectively handles long documents that exceed the model's immediate focus in a single prompt?

Accepted Answer

Implement a Map-Reduce strategy to process document chunks independently.. The 'Map-Reduce' pattern is the standard architectural approach for handling large datasets. You break the document into smaller chunks, process them independently to generate intermediate results, and then perform a final 'aggregation' pass to synthesize those results. This ensures that no individual prompt exceeds the optimal processing capacity, maintaining high quality throughout the pipeline while overcoming the limitations of single-prompt context handling for massive amounts of data.

Answer

Use a single prompt and ask the model to process the entire document at once.

Answer

Increase the 'max_tokens' parameter to its maximum limit.

Answer

Switch to a different model family that supports infinite input.

Question 4

You are processing user input that might contain malicious injection attempts. Which architectural layer is best suited to handle this?

Accepted Answer

The application-level code before the request is sent to the LLM.. You should never rely on the LLM as the sole security boundary. A multi-layered approach is required: first, an application-level input filter to sanitize data; second, a well-constructed system prompt that defines strict boundaries; and third, an output validator to ensure the model hasn't been successfully 'jailbroken.' Placing the primary security logic outside the model's inference loop is the only way to guarantee a reliable defense-in-depth posture.

Answer

The LLM's system prompt.

Answer

The model's fine-tuning training data.

Answer

A secondary LLM to monitor the first LLM.

Question 5

You are building an application that extracts data from messy, unstructured text. Which THREE practices help maximize the accuracy and consistency of the extraction?

Accepted Answer

Providing few-shot examples that show both clean and noisy input scenarios.. To achieve high accuracy in extraction, you need to provide the model with a clearly defined schema, few-shot examples of inputs and outputs, and clear negative constraints. These three pillars guide the model on how to handle noise, what the expected structure is, and what to avoid, ensuring the output is consistently usable for automated systems without manual intervention or excessive cleanup.

Answer

Using a very high temperature setting to encourage creative extraction interpretations.

Answer

Requesting the model to summarize the text before performing the extraction.

Question 6

An enterprise architect is designing a customer support bot that needs to reference a 50,000-token internal manual for every query. To ensure high reliability and low latency while managing costs, which Claude feature should be prioritized for this specific workload?

Accepted Answer

Prompt Caching (ephemeral). Prompt caching is a cornerstone of building reliable, low-latency applications with Claude 3.5. It allows architects to store frequently used context, such as system instructions or massive reference libraries, directly on Anthropic's servers. This strategy not only optimizes financial expenditure but also ensures consistent response times, which is vital for maintaining user trust and operational reliability in production-grade AI systems.

Answer

Recursive Retrieval-Augmented Generation (RAG)

Answer

Temperature scaling to 1.0

Answer

Logit bias adjustments

Question 7

Refer to the exhibit. The developer wants to ensure the summary is concise and follows a specific structure. What is the most effective way to improve this prompt?

Accepted Answer

Use a system prompt to define the task, format, and style constraints.. Providing a system prompt to establish the 'Persona' and 'Task Constraint' is superior to relying on the user message. By separating instructions from content, you provide a stable context for the model. This is standard architectural practice for Anthropic models, ensuring that the model maintains its directive despite varying document lengths, thus improving the consistency and quality of the generated summarization output over time.

Answer

Include the instructions directly inside the user content field.

Answer

Increase the temperature setting to 1.0 to ensure more creative summaries.

Answer

Repeat the instruction three times within the user content to enforce it.

Question 8

Refer to the exhibit. The model output is inconsistently professional. What is the best architectural improvement?

Accepted Answer

Use a few-shot prompt with an example of the desired professional output.. By providing a few-shot example that demonstrates a 'before' and 'after' transformation, you give the model a concrete target for 'professionalism.' Professionalism is subjective; examples provide the standard by which the model should calibrate its transformations. This approach is superior to abstract instructions, as it aligns the model's style with your specific enterprise requirements effectively.

Answer

Add a definition of 'professional' in the system prompt.

Answer

Set the top_p parameter to 0.1.

Answer

Change the role to 'Professional Writer' in the system prompt.

Question 9

You are building a system that must extract entities from messy transcripts. The model often misses entities at the end of the text. What is the most likely cause and architectural fix?

Accepted Answer

Place the instructions before the transcript and repeat them after the transcript.. The model is likely experiencing 'recency bias' or 'middle-of-the-prompt' degradation, where focus shifts away from the end. By moving instructions to the end of the prompt or using 'sandwich prompting'—placing key instructions at both the start and end—you reinforce the task. This architectural adjustment ensures the model maintains focus on the entire context window, which is vital for long-context entity extraction tasks in production environments.

Answer

The context window is too small; increase it by reducing the prompt complexity.

Answer

Switch to a smaller model to increase processing speed and focus.

Answer

Set the top_p parameter to 0.0 to force deterministic entity extraction.

Question 10

Which TWO of the following are primary benefits of using the Model Context Protocol (MCP) over a proprietary REST-based tool integration?

Accepted Answer

Standardization allows for easier reuse of tools across different MCP-compliant clients.. MCP provides a standardized communication layer that decouples data sources from the model, enabling interoperability across different client applications. By leveraging a common protocol, developers avoid building custom wrappers for every new model integration. These benefits simplify maintenance, promote ecosystem-wide compatibility, and ensure that tool definitions are portable and consistent, which is essential for scaling complex AI-agent architectures across diverse deployment environments.

Answer

It forces all communication to use binary protocols for maximum performance.

Answer

It eliminates the need for any authentication or security headers in tool calls.

Answer

It automatically converts all SQL databases into text files for the model.

Question 11

An enterprise application uses Claude to summarize complex legal documents. The system occasionally hallucinates specific clause numbers when the context window contains multiple similar contracts. Which strategy most effectively improves factual reliability for specific data extraction?

Accepted Answer

Use a retrieval-augmented generation (RAG) architecture to inject only relevant text segments into the context window.. Grounding the model with high-precision retrieval augmented generation (RAG) is the gold standard for reducing hallucinations in document processing. By forcing the model to operate strictly within the provided retrieved context, the probability of hallucinating external data decreases significantly. This approach is essential for architectural reliability, as it bounds the model's creative potential, ensuring that legal summaries remain tethered to the provided document artifacts rather than probabilistic memory.

Answer

Increase the system prompt's temperature setting to 1.0 to ensure more creative clause identification.

Answer

Implement a few-shot prompting strategy using examples of common legal clause structures.

Answer

Enable verbose chain-of-thought prompting to force the model to explain its reasoning process.

Question 12

You notice that your prompt, which works perfectly with Claude 3.5 Sonnet, fails when you switch to Claude 3 Haiku. What is the most likely reason?

Accepted Answer

Haiku requires more granular instructions and more few-shot examples.. Smaller models like Haiku have less 'instruction-following capacity' and may require more explicit, simplified prompting compared to larger models like Sonnet. When switching to smaller models, you must often reduce task complexity, provide more examples, and simplify formatting instructions. The smaller model's reduced parameter count necessitates clearer, less ambiguous prompts to achieve the same level of structural adherence and task accuracy expected from larger, more capable models.

Answer

The context window is smaller on Haiku.

Answer

Haiku has a higher temperature default, causing more errors.

Answer

The API endpoint has changed for the smaller model.

Question 13

You are designing a system to process customer support tickets. You want to ensure Claude provides accurate summaries and extracts the priority level. Which TWO strategies should you implement to improve output reliability?

Accepted Answer

Provide clear, concise system instructions that define the role and the goal of the task.. Combining clear, task-oriented system instructions with structured output formats provides the best results. The system prompt sets the boundary for behavior, while tool use or structured formatting ensures the output is predictable. Together, these two strategies minimize hallucinations and ensure that the extracted information can be easily mapped to backend database fields, which is vital for automated ticket routing and resolution workflows.

Answer

Use a long system prompt that covers every possible edge case for ticket classification.

Answer

Always set the maximum token limit to 4096 to ensure the model has enough space to explain its reasoning.

Answer

Enable high temperature settings to ensure the model explores all possible priority levels.

Question 14

You are building an RAG system. Which THREE factors most significantly impact the reliability of the retrieved information?

Accepted Answer

Chunking strategy (size and overlap).. RAG reliability is entirely dependent on the quality of the ingested data and the effectiveness of the retrieval mechanism. If the source material is poor, the model's output will be poor regardless of prompt engineering. By focusing on chunking strategy, embedding quality, and retrieval relevance, you ensure that the context provided to the model is accurate, complete, and highly targeted for the user's intent.

Answer

The model's temperature setting.

Answer

The language of the user query.

Question 15

Why are XML tags specifically recommended for structuring prompts in Claude, as opposed to other formats like JSON or simple bullet points, when managing large context?

Accepted Answer

Claude was specifically trained to recognize XML as structural delimiters.. Claude was specifically trained on a large amount of data containing XML-style tags, making it highly proficient at recognizing them as structural markers. While Claude can understand JSON, XML is more robust for wrapping large, multi-line blocks of text without needing to worry about escaping characters, which improves the reliability of the prompt's structure.

Answer

XML tags use fewer tokens than JSON or Markdown syntax.

Answer

JSON is not supported by the Claude 3.5 Sonnet context window.

Answer

XML tags automatically encrypt the data sent to the API.