CCAO-F Hard Questions — 25 Difficult Questions

Question 1

A developer is optimizing a RAG (Retrieval-Augmented Generation) pipeline using Claude 3.5 Sonnet. Which TWO techniques will most significantly improve the model's ability to extract accurate information from a 50,000-token context window?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Wrapping each retrieved document in distinct XML tags.. Handling large contexts requires structural clarity and logical transparency. XML tags help the model distinguish between different documents, while Chain of Thought (CoT) prompting forces the model to deliberate on the retrieved information before formulating a final answer. Together, these techniques reduce hallucinations and improve the precision of information extraction in dense context environments.

Answer

Setting top_k to a value of 1 to minimize output variety.

Answer

Converting all text to uppercase to improve character recognition.

Answer

Reducing the temperature to exactly 0.5 for balanced reasoning.

Question 2

A marketing team wants to use Claude to generate personalized political advertisements for a local election, targeting specific demographics with tailored messaging about voting records. According to Anthropic's 'Safety and Responsible Use' policies, how should this use case be handled?

Accepted Answer

It is prohibited because it involves personalized political campaigning and election influence.. Anthropic's policies are particularly strict regarding political campaigning and election integrity. Generating personalized political advertisements or targeting demographics to influence voting behavior is generally prohibited or highly restricted. This is to prevent the use of AI in spreading misinformation, manipulating public opinion, or interfering with the democratic process through large-scale automated messaging.

Answer

It is permitted if the team provides a disclaimer that the content was AI-generated.

Answer

It is allowed as long as the voting records used in the prompts are public information.

Answer

It is permitted only for local elections but prohibited for national or federal elections.

Question 3

Which TWO of the following are valid ways to handle long-running conversations within the Messages API to stay within context window limits?

Accepted Answer

Summarize previous chat history and append it as a single 'user' message.. Managing the context window is vital for long-term state retention. Truncating the oldest messages or summarizing previous interactions are standard practices to ensure the most relevant information is always included in the prompt. These techniques prevent the context from exceeding the model's window, which would otherwise result in an API error and a failure to generate a valid response for the user.

Answer

Increase the system prompt size to include the entire conversation archive.

Answer

Increase the 'max_tokens' value to accommodate the total conversation length.

Answer

Restart the conversation by clearing the history after every five messages.

Question 4

When designing a prompt for Claude, why is it recommended to place the most important instructions at the very beginning or the very end of the prompt?

Accepted Answer

To improve the model's attention to instructions.. Models sometimes suffer from 'lost in the middle' phenomena, where information buried in the center of a long context is less likely to be prioritized. By placing key instructions at the start or end, you leverage the model's tendency to focus on the initial 'pre-fill' context and the final instructions, ensuring that the model adheres strictly to your defined constraints and goals.

Answer

To reduce the token count of the prompt.

Answer

To make the prompt easier to read for humans.

Answer

To allow the model to cache the instructions for future calls.

Question 5

A red team is probing a Claude-powered tutoring application that helps high-school students with chemistry homework. During testing, a user submits a prompt asking Claude to role-play as a teacher who will provide step-by-step instructions for synthesizing a hazardous compound at home. The application currently passes raw user input directly to the model. Which response best reflects responsible handling of this scenario?

Accepted Answer

The team should add application-level controls such as input screening for hazardous-intent patterns, system-prompt scope limits for the tutoring context, and logging so that attempts are visible and reviewable.. A tutoring application serving minors needs defense in depth rather than a single reliance on model behavior. Application-level input screening, scoped system prompts, and privacy-preserving logging create detection and accountability around hazardous-intent attempts. Removing logging sacrifices visibility, and increasing creative latitude would weaken rather than strengthen the safety posture of the deployment.

Answer

The application should rely on Claude's built-in safety behavior to refuse, and the team should not add any additional controls since that would duplicate the model's protection.

Answer

The team should disable logging entirely to protect student privacy, and rely on periodic manual spot checks of random sessions instead.

Answer

The team should raise the model's temperature and encourage more creative role-play responses so students remain engaged with the tutoring experience.

Question 6

An engineer is streaming Claude's output to a web client and notices that the first visible token sometimes appears after a noticeable delay on long prompts. They want to reduce perceived latency without changing the model or the prompt content. Which change is most likely to help?

Accepted Answer

Enable streaming in the request so tokens are delivered incrementally as Claude generates them.. Streaming changes when tokens are delivered, not how they are generated, so the client renders the first token as soon as it is produced. This is the standard way to improve perceived responsiveness for long prompts. The other options either affect output length, model cost, or sampling behavior, none of which shorten the wait before the first token.

Answer

Switch to a larger Claude model so the response is generated more accurately.

Answer

Reduce the temperature to 0 so Claude decides on the first token faster.

Answer

Increase the max_tokens value so Claude has more room to generate the full answer.

Question 7

An analytics team asks Claude to extract structured fields from messy invoice text. They provide three input/output examples inside tags, then the real invoice inside tags. Accuracy is high on invoices that resemble the examples but drops sharply on unusual layouts. Which adjustment most directly improves generalization to the unusual layouts?

Accepted Answer

Add more examples that cover diverse invoice layouts, including edge cases, inside the examples block.. Few-shot learning generalizes in proportion to how representative the demonstrations are. When Claude performs well on inputs that resemble the examples and poorly on inputs that do not, the examples are too homogeneous. Expanding the demonstration set to include diverse and edge-case layouts teaches the task itself instead of a single pattern, which is the most direct fix for the observed failure on unusual invoices.

Answer

Raise the temperature so Claude explores more varied interpretations of each invoice.

Answer

Move the examples block after the invoice so Claude reads the real input first.

Answer

Shorten each example to only the input and the final JSON, removing any intermediate reasoning text.

Question 8

A legal team asks Claude to summarize contracts and cite the exact clause supporting each summary point. Claude produces accurate summaries but cites clauses that do not exist. The contracts are supplied inside <contract> tags in the same message. Which change most directly reduces the fabricated citations?

Accepted Answer

Instruct Claude to quote the exact sentence from the contract for each point and to say 'not found' when no sentence supports it.. Fabricated citations occur when a model is asked to attribute claims without being forced to ground them in source text. Requiring a verbatim quote for each point, plus an explicit 'not found' option when no sentence supports a claim, removes the incentive to invent plausible clauses. The other options either layer metadata over unverified citations, rely on examples alone, or enlarge input without changing the grounding requirement.

Answer

Increase the contract length limit so Claude has more space to search for the cited clauses.

Answer

Ask Claude to rank each citation by confidence so reviewers can skip the low-confidence ones.

Answer

Add a few-shot example showing a correct summary with a correct clause citation.

Question 9

An application uses Claude to summarize user-generated medical feedback. To comply with privacy requirements, you must ensure no Personally Identifiable Information (PII) is sent to the model. Which is the most appropriate workflow?

Accepted Answer

Processing the text through a dedicated PII anonymization layer before calling the API.. Data sanitization should occur prior to hitting the API. By using a PII-scrubbing service (like Presidio or a custom regex layer) before the payload reaches Anthropic, you enforce data residency and privacy controls at the edge. This approach is essential for responsible AI because it ensures PII never leaves your secure environment, preventing potential leaks and maintaining strict compliance with global privacy standards like GDPR or HIPAA.

Answer

Prompting the model to ignore and redact any PII found in the input.

Answer

Using a specific system prompt that warns the model to treat all input as private.

Answer

Requesting that users manually redact their own PII before submitting feedback.

Question 10

A company is developing an application that generates long-form creative content. They notice that as the output length increases, the model's speed seems to fluctuate. Which technical factor most significantly impacts the latency of the 'Time to Last Token' (TTLT) in this scenario?

Accepted Answer

The total number of output tokens generated.. The Time to Last Token (TTLT) is primarily a function of the total number of tokens generated by the model. Each token is generated sequentially, meaning the model must perform a full inference pass for every single word or sub-word it produces. Therefore, longer outputs naturally take more time, as the total latency scales linearly with the generation length. (70 words)

Answer

The number of concurrent users in the API queue.

Answer

The use of a system prompt instead of a user prompt.

Answer

The presence of images in the input context.

Question 11

Which TWO of the following statements accurately describe the characteristics of the Claude 3.5 Sonnet model's context window and performance?

Accepted Answer

Claude 3.5 Sonnet offers a 200k token context window capacity.. Claude 3.5 Sonnet features a 200k token context window, which allows for the ingestion of large codebases or documents. Furthermore, the model is optimized for high throughput and low latency, making it ideal for tasks requiring rapid processing. Understanding these technical trade-offs is essential for architecting scalable applications that need to balance the need for deep contextual understanding with the speed requirements of real-time user interfaces.

Answer

Claude 3.5 Sonnet supports an infinite context window for streaming data.

Answer

The model's performance decreases linearly as the context window is filled.

Answer

Context window usage is irrelevant to API response latency.

Question 12

A developer wants to utilize 'Tool Use' (function calling) with Claude. Which TWO steps are necessary to properly define a tool in the API request?

Accepted Answer

Provide a 'description' field explaining what the tool does.. To use tools, developers must provide a 'tools' array in the request. Each tool needs a 'name', a 'description' (which helps Claude understand when to use it), and an 'input_schema'. The 'input_schema' must be a valid JSON Schema object, defining the parameters the tool expects. This structured approach allows Claude to generate valid arguments that the developer's code can then execute.

Answer

Include the tool's source code in the 'implementation' field.

Answer

Set the 'role' of the message to 'function'.

Answer

Upload a CSV file containing example tool outputs.

Question 13

Refer to the exhibit. This request uses a technique to force Claude to output valid JSON. What is the technical name for this technique, and what is its primary benefit?

Accepted Answer

Response Prefilling; it eliminates conversational filler and ensures correct formatting.. The technique shown is 'prefilling the assistant response.' By starting the assistant's turn with the beginning of a JSON object, the developer forces Claude to continue the pattern. This is the most reliable way to ensure the output starts exactly with the required characters for programmatic parsing, bypassing any conversational preamble.

Answer

Few-shotting; it provides a single example of the JSON format to follow.

Answer

System Role Prompting; it defines the model's persona as a JSON generator.

Answer

XML Delimitation; it uses the curly braces as tags to separate the data.

Question 14

Refer to the exhibit. The developer notices the model often hallucinates data not present in the <data> tags. Which adjustment is most likely to mitigate this behavior?

Accepted Answer

Add an explicit instruction in the system prompt to only use information contained within the tags.. Adding a strong negative constraint to the system prompt, such as 'If the information is not present in the provided , state that you do not have sufficient information,' directly addresses hallucination. This forces the model to prioritize factual grounding over generative completion. In production systems, explicit constraints about the source material are essential for maintaining the integrity of data processing workflows and preventing the propagation of false information.

Answer

Reduce the temperature to 0.0 to make the model's output more deterministic.

Answer

Increase the max_tokens to ensure the model has enough room to explain its reasoning.

Answer

Change the model to an older version that is less prone to creative generation.

Question 15

Refer to the exhibit. An engineer wants to use Prompt Caching to optimize this request. What is the correct way to modify the request body to enable this?

Accepted Answer

Nest a 'cache_control' object within the content block of the message.. Prompt caching requires the inclusion of a 'cache_control' block within the message content. This tells the API to store the processed state of that specific chunk. This is critical for high-latency or high-cost prompts because it allows for faster processing of subsequent requests that share the same cached prefix, significantly reducing latency and cost for repetitive tasks like data analysis.

Answer

Add a 'cache_control' field to the root level of the JSON body.

Answer

Rename the 'system' field to 'system_cached'.

Answer

Enable 'caching=true' in the request headers.

Question 16

An HR department is using Claude to screen resumes for a high-volume engineering role. They are concerned that the model might inadvertently favor candidates from certain universities over others, reflecting historical biases in the training data. Which concept in AI safety does this concern directly address?

Accepted Answer

Algorithmic Bias and Fairness.. Algorithmic bias and fairness are critical components of responsible AI use. In hiring, bias can manifest as a preference for certain demographics or backgrounds that were overrepresented in the training data. Addressing this involves careful testing, monitoring, and potentially using debiasing techniques to ensure the model makes equitable decisions based on merit rather than historical prejudice.

Answer

Model Hallucination.

Answer

Prompt Injection.

Answer

Data Sovereignty.

Question 17

Refer to the exhibit. What is the specific effect of the 'tool_choice' parameter as configured in this request?

Accepted Answer

It forces Claude to use the 'get_weather' tool specifically.. The 'tool_choice' parameter allows developers to control Claude's tool-calling behavior. By setting it to type 'tool' and specifying a 'name', the developer is forcing Claude to use that specific tool, even if it thinks it could answer without it. This is useful for specialized workflows where a specific function must always be executed as the next step in the application logic.

Answer

It allows Claude to choose between using the tool or answering normally.

Answer

It prevents Claude from using any tools during this request.

Answer

It enables Claude to use multiple tools in a single response.

Question 18

An application is using Claude to generate structured data. Which approach is most reliable for ensuring the model strictly adheres to the requested JSON schema?

Accepted Answer

Using the system prompt to define the schema and providing an example.. Combining a clear system prompt with a well-defined output example and a 'stop_sequence' is the most robust way to enforce structure. Using the system prompt to explicitly state the desired format and providing a template ensures the model understands the constraints, while the stop sequence prevents it from adding conversational filler, which is the most common cause of parsing errors in automated pipelines.

Answer

Only relying on the model to 'understand' JSON from its training data.

Answer

Increasing the temperature to 1.0 to give the model more flexibility.

Answer

Asking the model to repeat the JSON schema back to you before generating data.

Question 19

You are designing a high-throughput application that uses Claude. Which TWO of the following strategies are best practices for managing rate limits and ensuring reliability?

Accepted Answer

Implement exponential backoff with jitter when encountering 429 errors.. Managing production-grade AI applications requires proactive handling of concurrency limits and transient network errors. Using exponential backoff ensures that your application gracefully handles rate limiting (429 errors) without overwhelming the API, while implementing asynchronous processing or queues helps manage spikes in load, preventing your application from failing during peak usage periods and improving overall system resilience.

Answer

Increase the max_tokens parameter to the maximum limit to minimize requests.

Answer

Hard-code a fixed 10-second sleep timer between every API call.

Answer

Use the streaming API to reduce the time to first byte.

Question 20

Which THREE of the following are valid practices when handling streaming responses from the Claude API?

Accepted Answer

Use the 'content_block_delta' event to append text chunks in real-time.. Streaming is critical for reducing perceived latency. Proper implementation involves handling Server-Sent Events (SSE), listening for specific events like 'content_block_delta', and gracefully managing connection interruptions. These practices ensure the UI updates fluidly as the model generates text, providing a better user experience while simultaneously allowing the application to process partial data chunks immediately, which is essential for responsive, interactive AI-powered software interfaces.

Answer

Buffer the entire stream and display it only after it finishes.

Answer

Ignore 'message_stop' events, as they contain no usable content.