Refer to the Exhibit Practice Questions

Catch the error and retry the request immediately in a loop.

Why it fails: Immediate retries can lead to a thundering herd problem, where the application continuously hammers the API, worsening the rate limiting issue. This approach ignores the need for backoff, likely resulting in repeated failures and potential service degradation for other parts of the application or other API users.

Increase the timeout duration for all API calls in the application.

Why it fails: Increasing timeouts does not address the fundamental issue of rate limiting. If the API is rejecting requests due to volume, waiting longer for a response will not result in a successful request. It only delays the error handling process and may lead to resource exhaustion in the client application.

Switch to a synchronous architecture to serialize all API requests.

Why it fails: Serializing requests eliminates concurrency but significantly decreases throughput and user experience. It is not a scalable solution for high-load scenarios. A robust design should support high concurrency through intelligent request management and backoff strategies rather than artificially constraining the system to sequential processing, which limits growth potential.

Immediately terminate the agent and alert the user

Why it fails: Terminating the session on a transient error is a poor user experience and leads to wasted computational work. Since 'overloaded_error' is usually temporary, the system should attempt to recover automatically rather than forcing the user to restart the entire multi-step process from the beginning, losing all current progress.

Restart the entire agentic loop from the first message

Why it fails: Restarting from scratch is inefficient and costly, as it repeats all previous tool calls and reasoning steps. This significantly increases latency and token usage. The system should instead retry the specific failed request, maintaining the existing conversation history and state to continue exactly where the error occurred.

Reduce the temperature of the next request to 0

Why it fails: The error shown is a service-level error related to API capacity, not a model generation error. Changing the temperature has no effect on the server's ability to process the request and will not resolve the 'overloaded' status. It is an irrelevant parameter change for this specific technical failure.

Tell them the system is broken and that we need to stop the development immediately.

Why it fails: Reporting the system as 'broken' is inaccurate and causes unnecessary alarm. The logs show that the safety filters are functioning correctly by rejecting inappropriate output. The architect should explain that this is a desired safety mechanism, not a system failure, and discuss how to refine the model's behavior if needed.

Keep the logs hidden to avoid making the project look like it has problems.

Why it fails: Hiding system logs is a dishonest practice that creates a false sense of security. If a major incident occurs, the lack of transparency will be discovered, severely damaging the architect's credibility and the project's future. Transparency is required for effective stakeholder management and risk oversight in any enterprise environment.

Ask the product owner to manually approve all output that the model generates.

Why it fails: Manually approving every output is not a scalable solution and would bottleneck the entire business process. The goal of implementing AI is automation; the architect should focus on improving the system instructions or safety filters, not shifting the operational burden onto the product owner, which is both inefficient and impractical.

Increase the timeout duration of the API calls indefinitely.

Why it fails: Indefinite timeouts are dangerous and can cause thread exhaustion in the application, leading to cascading failures. It does not address the 429 error and will only make the system less responsive. Effective error handling must be active and intelligent, not simply passive waiting for longer periods.

Bypass the API gateway and send requests directly to the model's backend IP.

Why it fails: Bypassing the API gateway is a violation of security protocols and operational governance. It does not prevent rate limiting, as the model provider's service will still enforce these limits at the infrastructure level. This approach is unstable, unauthorized, and provides no solution to the rate-limiting problem.

Disable all retries to prevent the application from crashing.

Why it fails: Disabling retries means that every transient error results in a failed user request, leading to a poor user experience. While it prevents cascading failure, it is not an effective solution to the underlying need for high availability. Intelligent retries are essential for robust LLM application design.

Man-in-the-middle attack; the system prompt encrypts the secret code name.

Why it fails: A man-in-the-middle attack involves intercepting communication between two parties, which is a network security issue. The system prompt is not an encryption tool; it is a set of behavioral instructions that guide the model's text generation logic to ensure policy compliance.

Denial of Service (DoS); the system prompt limits the tokens used to hide the secret.

Why it fails: A Denial of Service attack aims to make a system unavailable by overwhelming it with requests. The system prompt's role in this exhibit is about data protection and behavior control, not resource management or preventing the service from responding to legitimate traffic.

Data poisoning; the system prompt cleans the training data in real-time.

Why it fails: Data poisoning occurs during the training phase when malicious data is introduced into the model's dataset. The system prompt is used during inference, meaning it affects how the model uses its existing knowledge to respond to a specific request, rather than altering the model's underlying training.

The input parameters do not match the JSON schema

Why it fails: If the input parameters were invalid according to the defined schema, the error would typically occur during the validation phase before the tool is even executed. However, the exhibit shows the model successfully generating the call, suggesting the problem lies in the subsequent step of the orchestration loop, not the schema.

The tool_use ID is missing the required prefix

Why it fails: The provided ID 'tool_102' is a valid string representation within the content block. While Anthropic IDs often follow specific formats like 'toolu_', the API itself accepts various string identifiers. A prefix mismatch would not stop the orchestrator from attempting a response; it would more likely result in an API validation error.

The assistant content block lacks a stop_reason

Why it fails: The 'stop_reason' is a property of the API response metadata, not a field that resides within the content array of the assistant's message. The absence of metadata in the JSON snippet provided does not indicate an architectural failure, as the exhibit focuses on the structure of the message content itself.

It significantly reduces the total cost of the API call by limiting output length.

Why it fails: While stop sequences limit the length of a specific response, they are not primarily cost-saving tools. Their main purpose in governance is to enforce structural boundaries and prevent the model from assuming the user's identity, which is a critical security and behavioral constraint for multi-turn interactions.

It improves the creativity of the model by forcing it to summarize more frequently.

Why it fails: Stop sequences have no direct impact on model creativity or summarization capability. Their influence is strictly on the termination of the output generation process, not on the qualitative content of the model's response. This is a common misconception about the role of stop tokens in LLM engineering.

It allows the model to handle more complex logic by processing it in smaller chunks.

Why it fails: Stop sequences do not break up or simplify logical processing. They simply tell the API to stop generating once a specific string is reached. This is a configuration control for behavioral stability, not a mechanism for enhancing the underlying logical reasoning capabilities of the model itself.

The external Python-based regex filter applied to the API output.

Why it fails: While some organizations use regex filters to catch keywords, these are reactive and easily bypassed by creative phrasing. The refusal described in the scenario happens within the model itself during the inference process, rather than being a post-processing step that scrubs text after it has already been generated.

The 'max_tokens' parameter being set to a value low enough to truncate the response.

Why it fails: The 'max_tokens' parameter simply limits the length of the response; it does not influence the content's safety or the model's decision to refuse a prompt. Truncation would result in a cut-off sentence, not a coherent refusal message based on ethical or safety guidelines.

A hardcoded list of forbidden words in the Anthropic Messages API gateway.

Why it fails: Anthropic does not rely on simple word-matching at the gateway level to ensure safety, as this approach is brittle and context-unaware. Instead, the model's safety is deeply integrated through training, allowing it to understand the intent of the prompt and refuse based on the underlying harm.

Increase the temperature setting to 1.0.

Why it fails: Higher temperature values increase randomness and creativity, which actually makes the model more susceptible to following adversarial instructions. Increasing temperature does not provide any defense against prompt injection; in fact, it may make the model's output even more erratic and difficult to predict during an attack.

Reduce max_tokens to 10.

Why it fails: Reducing the token limit truncates the model's response but does not prevent the initial injection attempt from succeeding. If the first few tokens contain sensitive information, the attack still succeeds. Controlling output length is a cost-management tool, not a valid security control for preventing adversarial prompt injection.

Add a user authentication layer to the API.

Why it fails: While authentication is necessary for identifying who is making the request, it does not prevent a malicious or compromised user from attempting a prompt injection attack. Once authenticated, the user still poses a risk through the content they submit. Application-level security must complement identity management for total safety.

The model version string

Why it fails: The model version string specifies which version of Claude to invoke but does not contain any logic or instructions for safety. While newer models have improved safety features, the version string itself is not a governance mechanism but rather a resource identifier for the API request.

The metadata object fields

Why it fails: Metadata tags are useful for tracking and auditing purposes but do not directly influence the model's output or provide a layer of safety against malicious inputs. While metadata is essential for usage monitoring and billing, it lacks the functional capacity to enforce behavioral constraints or prevent security breaches.

The user role in the messages array

Why it fails: The 'user' role identifies the source of the input but does not impose constraints on what the user can ask or how the model should respond. It is a structural requirement of the API rather than a governance tool for enforcing safety policies or managing organizational risk.

The input text exceeds the model's total context window size.

Why it fails: If the input exceeded the context window, the API would return a context length error before processing. The issue described is a truncation of the output, which happens when the generation limit set by 'max_tokens' is reached during the streaming or completion process for the response.

The system prompt is too short to handle long-form summarization.

Why it fails: System prompts define behavior but do not control the token output limits. A short system prompt may affect the quality of the summary, but it does not cause premature truncation of the generated output. Truncation is exclusively a function of the 'max_tokens' configuration parameter and the generation process.

The model version being used does not support large outputs.

Why it fails: Anthropic models support large output token counts, provided the 'max_tokens' parameter is configured correctly. If the model were incapable of long outputs, it would be a fundamental limitation of the model, not a configuration issue. The current symptom points directly to an insufficient token budget allocation for the request.

Tell them the service is unusable and wait for Anthropic to fix it.

Why it fails: Telling stakeholders the service is unusable is inaccurate and shows a lack of architectural ownership. Rate limits are a standard part of API usage that should be handled by robust application logic, such as exponential backoff. This answer deflects responsibility rather than explaining how the system is currently managing the issue.

Ignore the issue until the stakeholder asks about it again.

Why it fails: Ignoring a known, recurring error is a failure to manage the application lifecycle. Proactive communication is essential for maintaining trust. By the time a stakeholder asks, the issue could have already negatively impacted users. Being transparent from the start is much better than waiting for the stakeholder to discover the issue.

Blame the engineering team for poor code quality.

Why it fails: Assigning blame within the team is toxic and unprofessional. The architect represents the entire team and should focus on solutions. Even if the code needs improvement, the focus should be on the technical path to resolution, not on creating internal conflict. This behavior destroys team morale and professional relationships.

Discard the task and return a failure message to the user.

Why it fails: Discarding the task is a poor user experience. Rate limits are often transient. A well-designed agent should have the capability to wait and retry automatically before giving up, which allows it to handle temporary spikes in traffic without manual user intervention or task failure.

Immediately retry the request without any delay.

Why it fails: Immediate retries will almost certainly fail again and potentially exacerbate the rate limit issue. This behavior constitutes a 'retry storm' that can lead to permanent blacklisting of your API key. You must always wait for the specified time or use exponential backoff to allow the system to stabilize.

Switch to a backup API key and continue immediately.

Why it fails: Using backup keys to bypass rate limits can lead to account suspension. Most providers track rate limits by organization or underlying billing entity, not just by individual key. You should architect for resiliency through backoff, rather than attempting to circumvent the limits set by the API provider.

Change the model to a smaller, less capable version.

Why it fails: Reducing model capacity does not prevent the model from attempting to fulfill the request. Smaller models can still be instructed to generate malicious content if they are not constrained by proper safety guardrails. Architectural safety relies on filtering and monitoring, not on decreasing the quality or capabilities of the LLM.

Require the user to log in with MFA.

Why it fails: Multi-factor authentication (MFA) ensures only authorized users can access the system, but it does not prevent an authenticated user from making a malicious request. A malicious insider or a compromised account can still attempt to exploit the model, necessitating proactive content moderation regardless of the authentication method.

Increase the frequency of system prompt updates.

Why it fails: Updating system prompts is reactive and hard to scale for every possible malicious request. It is not a substitute for a structured guardrail system designed to identify and block prohibited content. Governance models require robust, automated filters to ensure consistent safety across the entire application ecosystem.

The temperature setting is too high for professional administrative tasks.

Why it fails: While a temperature of 0.7 is relatively high, it is not a primary governance concern compared to the explicit disabling of safety filters. Temperature impacts creativity and consistency, whereas safety filter configuration directly pertains to the legal and ethical liability of the organization using the model.

The max_tokens limit is too low, restricting the capability of the system.

Why it fails: A max_tokens limit of 1024 is a performance configuration choice, not a risk management issue. While it may restrict the depth of the model's response, it does not create an inherent security or governance vulnerability, unlike the intentional removal of safety guardrails for harmful content.

The system prompt is too generic, leading to poor model performance.

Why it fails: A generic system prompt might affect the quality of the assistant's helpfulness, but it does not represent a governance or safety risk. The configuration of safety settings is a critical governance failure, whereas a poor system prompt is merely an optimization issue for the application.