Select Two (Multi-Select) Questions

Automatic rotation of all API keys every 24 hours.

Why it fails: While frequent rotation is a security best practice, the Anthropic Console does not natively provide an automated 24-hour rotation service for all keys. This would typically need to be implemented by the customer using external secrets management tools like AWS Secrets Manager or HashiCorp Vault.

Real-time packet inspection of all API traffic.

Why it fails: Packet inspection is a network-level security measure that is not a feature of the Anthropic Console. This type of monitoring would occur at the corporate firewall or proxy level. The Console focuses on identity and resource management rather than inspecting the contents of the HTTPS traffic.

Hardware Security Module (HSM) storage for all model weights.

Why it fails: Model weights are part of Anthropic's core intellectual property and infrastructure, managed entirely by Anthropic. Customers do not have access to these weights and cannot configure HSM storage for them through the Console; this is outside the scope of customer-facing governance features.

Enable the model to have full administrative access to the file system.

Why it fails: Granting administrative access is a major security vulnerability. An agent should operate within a heavily sandboxed environment with the minimum set of permissions required. Full access would allow a prompt-injection attack to potentially compromise the entire infrastructure and lead to catastrophic data leaks or system destruction.

Embed all user data directly into the system prompt for faster access.

Why it fails: Putting sensitive user data in the system prompt increases the risk of exposure and makes it impossible to manage permissions effectively. It also bloats the context window unnecessarily. Data should be fetched dynamically with appropriate security controls, rather than hard-coded into the persistent system prompt.

Physical security of the data centers housing the TPU/GPU clusters.

Why it fails: The physical security of the hardware and the environment where the models are trained and hosted is the responsibility of Anthropic and its cloud infrastructure partners. Customers have no control over data center access or hardware maintenance, as these are managed at the platform and infrastructure layers.

Fine-tuning the base model's Constitutional AI principles.

Why it fails: Constitutional AI principles are baked into the model's core during the training and RLHF phases by Anthropic researchers. Customers cannot modify these foundational safety guardrails; instead, they must use system prompts and external moderation tools to layer their own specific business rules on top of the model.

Hardcode system prompts to ensure the model behavior cannot change over time.

Why it fails: Hardcoding prompts is a maintenance nightmare. It prevents updating instructions to improve quality or safety and makes version control impossible. Over time, as your application needs evolve, being unable to update prompts will force you into difficult refactoring sessions, which is the opposite of productive architecture.

Use the largest available context window for every single request to maximize intelligence.

Why it fails: Using the maximum context window for every request is wasteful and inefficient. It increases costs significantly and can introduce unnecessary latency. Architects should advise using the smallest context window necessary for the task, which improves efficiency and keeps costs predictable, demonstrating a mature approach to resource management.

Promise stakeholders that the model will have a 100% accuracy rate for all inputs.

Why it fails: Promising 100% accuracy is a dangerous and dishonest approach that will inevitably lead to a loss of trust when the model eventually makes an error. It is critical to manage expectations by clearly communicating the probabilistic nature of LLMs and the importance of human-in-the-loop validation processes.

Avoid mentioning technical constraints to keep the stakeholder focused on the positive benefits.

Why it fails: Omitting technical constraints is a deceptive practice that undermines long-term credibility. Stakeholders need a balanced view to make informed risk decisions. Hiding limitations will only lead to greater distrust once those limitations eventually manifest during operations, potentially damaging the architect's reputation and the project's viability.

Allow the model to finalize decisions without human review.

Why it fails: Automated decision-making without human oversight is contrary to responsible AI practices in high-stakes contexts. Models can experience drift or hallucinations, which could lead to significant errors. Requiring a human-in-the-loop provides a necessary accountability layer and ensures that critical decisions receive proper ethical and logical scrutiny before being finalized.

Only use the most advanced model available for all tasks.

Why it fails: Using the most complex model is not a substitute for safety governance. Choosing the right model should depend on task requirements, latency, and cost. Over-reliance on model complexity does not address the need for structural safety controls like red-teaming, rigorous evaluation, and human-in-the-loop processes, which are independent of model size.

Disable all system logs to preserve user privacy.

Why it fails: Disabling logs prevents essential debugging and incident response. Responsible AI governance requires auditability to understand how decisions were made and to investigate potential safety breaches. Privacy can be maintained through data masking and access controls rather than by eliminating the ability to audit system behavior and performance metrics.

The total number of parameters in the model.

Why it fails: The parameter count is an internal implementation detail and is not directly exposed or useful for architectural decision-making. Architects should focus on functional capabilities, performance metrics, and cost, rather than the underlying model architecture, which is subject to change by the provider.

Whether the model is open-source or proprietary.

Why it fails: The licensing model (open-source vs. proprietary) is a secondary concern compared to the performance, cost, and reliability of the model provider. For most enterprises, the operational support, security guarantees, and consistent performance of a managed proprietary model are more important than the open-source status of the weights.

Hardcode all prompt versions directly into the application source code.

Why it fails: Hardcoding prompts creates tight coupling, forcing a full application deployment for every minor prompt tweak. This practice severely limits developer productivity and increases the risk of deployment-related errors. It prevents independent scaling and testing of prompt logic, making the codebase significantly harder to maintain and audit over time.

Combine prompts and business logic into monolithic classes for better encapsulation.

Why it fails: Monolithic structures lead to high cognitive load and make testing individual components difficult. When prompt logic is tangled with application code, it becomes impossible to verify prompt performance in isolation. This design pattern discourages modularity and hinders the ability of teams to iterate quickly without introducing unintended side effects.

Avoid using variables in prompts to ensure absolute predictability.

Why it fails: Static prompts lack the flexibility required for modern LLM applications. Variable injection is a standard requirement for dynamic content generation. Avoiding variables limits the utility of the system and forces developers to write duplicate prompts for minor variations, which creates significant maintenance overhead and redundancy across the repository.

Enable verbose debug logging of every prompt and response for later forensic review.

Why it fails: Verbose logging is valuable for investigation but is detective, not preventive. It records what a compromised agent did without restricting what it can do, so refunds and record edits would still occur at full scale. Forensic evidence helps after an incident, but the risk committee asked for controls that limit the damage an agent can cause while it is misbehaving.

Schedule quarterly reviews of the agent's conversation transcripts to identify emerging misuse patterns.

Why it fails: Periodic transcript review supports trend analysis and program improvement, but it operates on a quarterly cadence and cannot contain an active incident. Between reviews, a compromised agent could issue unlimited refunds and alter orders. As a governance rhythm it complements preventive controls, yet it does not reduce the maximum impact of a single malicious or erroneous action.

Publish an acceptable use policy that prohibits employees from manipulating the agent.

Why it fails: A policy communicates expectations and supports disciplinary action, but it does not technically constrain the agent's capabilities. A malicious external user or injected instruction is unaffected by an internal policy document. Since the goal is limiting blast radius from a compromised agent, a non-enforceable policy statement cannot substitute for scoped permissions or approval gates.

Route every request through a gateway that logs the full request and response payloads with the model identifier attached.

Why it fails: Payload logging is valuable for incident forensics and quality review, but logs are observational evidence, not a control over model behavior. A log showing that responses changed overnight does not prevent the change, attribute it to a decision-maker, or provide a rollback path. Regulators asking for attributable, reversible change management need the pinning and evaluation record instead.

Enable prompt caching on all production requests so that previously validated prompts are reused verbatim.

Why it fails: Prompt caching reduces latency and cost by reusing prefix computations, but it does not version model weights or record behavioral change. Cached prompts still execute against whatever model is currently aliased, so a silent upgrade would change outputs while the cache layer reports nothing. It is a performance feature, not an auditability control.

Store the system prompt in a version-controlled repository and require pull-request review before it is merged.

Why it fails: Version-controlling the system prompt governs one input to the system, and review discipline is genuinely useful. However, the model itself can change beneath an unchanged prompt when an alias such as claude-3-5-sonnet-latest silently advances. This practice therefore cannot demonstrate that model behavior changes are tracked or reversible, which is the committee's stated obligation.

A verbatim copy of the vendor's model card and system prompt.

Why it fails: Model cards and prompts are useful supporting evidence, but they are inputs to risk analysis, not register entries. Pasting them verbatim produces documentation without an owner, rating, or review date, so the entry cannot be tracked, escalated, or closed, and it will not satisfy the governance lead's requirement for actionable residual-risk records.

A list of every employee who has ever accessed the assistant.

Why it fails: Access rosters belong to identity and access management records, not to a risk register line item. Attaching exhaustive user lists to each risk bloats the register with data that changes constantly and provides no statement of exposure, ownership, or treatment, so it does not help the network manage the discharge-summary risk.

A projected cost saving attributed to deploying the assistant.

Why it fails: Expected financial benefit is a business-case input, not a risk attribute. Mixing savings estimates into risk entries conflates upside with downside and can bias ratings toward understating harm. The register needs a residual exposure statement with an owner and review cycle, which a savings figure does not supply.

A list of all stakeholders who attended the project kickoff meeting.

Why it fails: Kickoff attendance records are historical project artifacts, not evidence of operational readiness. Stakeholders at kickoff may have changed roles, and attendance implies no ongoing responsibility. The steering committee needs current, forward-looking commitments about who operates the service and under what terms, so a historical attendee list adds no assurance about post-launch support, monitoring, or accountability.

A complete archive of every prompt version ever tested during development.

Why it fails: Historical prompt archives are useful for traceability and audits, but they do not demonstrate operational ownership. Keeping every experimental version also creates confusion about which prompt is authoritative in production. The readiness question is who will operate and maintain the service, not what was tried during development. An archive can support a runbook, but it cannot substitute for documented ownership and support commitments.

A slide deck summarizing the assistant's key features for the marketing team.

Why it fails: A marketing overview communicates capabilities to external or internal audiences but says nothing about operational accountability. It does not identify who fixes incidents, how quality is monitored, or what service levels apply. Presenting it as readiness evidence would leave the steering committee without confirmation that support and maintenance are staffed, which is the core concern of a go-live handover review.

Increasing the frequency of tool calls

Why it fails: Simply calling more tools does not improve reasoning; in fact, it can lead to more opportunities for error. If the underlying logic is flawed, more tool calls will just result in more incorrect data being processed, potentially leading the agent further away from the correct solution.

Reducing the temperature to exactly 0.0

Why it fails: While temperature 0 makes the model deterministic, it does not improve the quality of its reasoning. A model can be perfectly deterministic and still consistently wrong. Determinism helps with reproducibility but doesn't fix the logical gaps or the lack of self-correction that the architect is trying to solve.

Using the 'tool_choice' parameter to force tool use

Why it fails: Forcing a tool choice ensures the model picks a tool, but it doesn't ensure it picks the *right* tool or understands the results correctly. This technique is for controlling output format, not for improving the cognitive depth or the self-correction capabilities of the agent's reasoning process.

The ability to ignore rules during testing.

Why it fails: Ignoring rules during testing undermines the integrity of the safety process. If you don't test under the same constraints that will be applied in production, the testing results are invalid. Safety policies must be enforced consistently throughout the entire lifecycle, including development, testing, and production.

Complete removal of human oversight.

Why it fails: Removing human oversight is the opposite of good safety governance. AI systems, even those with guardrails, require human supervision to verify outcomes, investigate incidents, and update policies. Human-in-the-loop processes are the ultimate safety mechanism for high-stakes decisions and for validating the system's overall performance and safety.

Store API keys as plain text in the CLI configuration file for ease of developer access.

Why it fails: Storing API keys in plain text creates a critical security vulnerability, exposing the organization to unauthorized access. Credentials should always be managed through secure environment variables or secret management services like AWS Secrets Manager or HashiCorp Vault. Proper secret management prevents accidental leakage of sensitive credentials in version control.

Set the maximum token limit to 4096 for all requests to ensure uniform response times.

Why it fails: Arbitrarily limiting tokens can truncate helpful responses or cause unexpected failures if the model requires more space for a complex task. Token limits should be set based on the specific requirements of the prompt and expected output. A static limit ignores the diverse needs of different LLM-driven applications.

Disable all request retries to ensure the CLI tool fails fast during network issues.

Why it fails: Failing fast without retry logic is detrimental to user experience in automated environments. Transient network errors are common in cloud environments; retries ensure that minor blips do not cause long-running jobs to fail. Intelligent retries significantly increase the success rate of workflows without requiring developer intervention for minor issues.

Average latency of the user's internet connection.

Why it fails: User-side network latency is outside the control of the agentic system. While important for general UX, it is not an observability metric for the agent's internal reasoning process. Focus should be on the agent's internal performance, not the user's network environment, to optimize the system's logic.

The color profile of the agent's UI dashboard.

Why it fails: The UI's design is purely aesthetic and has no impact on the agent's reasoning or performance. Monitoring this does not provide any insight into the system's reliability. Observability metrics must be technical, objective, and related to the agent's internal state, reasoning steps, and interaction with external tools.

Number of times the user clicks the refresh button.

Why it fails: While refreshing might indicate user frustration, it is a lagging indicator of poor performance. Observability should focus on the internal metrics that explain *why* the agent is failing, such as tool call rates or token usage, rather than relying on external user behavioral signals which are often noisy and delayed.

Increasing the context window size

Why it fails: Expanding the context window allows the agent to process more information but does nothing to stop a repetitive logic loop from occurring. In fact, a larger window might actually prolong the loop by giving the agent more room to generate redundant reasoning before hitting a token limit at a higher cost.

Using a lower temperature setting

Why it fails: While reducing temperature makes the model more deterministic, it does not inherently prevent logic loops where the model repeatedly decides to use the same tool. Determinism can sometimes make loops more likely as the model consistently arrives at the same flawed reasoning path without any stochastic variation to break out.

Enabling prompt caching for tool definitions

Why it fails: Prompt caching is a performance optimization that reduces latency and cost for repeated input prefixes but does not influence the logical flow or termination of an agentic loop. It helps handle the loop more cheaply but does not provide a mechanism to detect or stop the loop from continuing indefinitely.

Disable streaming in the playground to reduce token consumption.

Why it fails: Streaming affects how tokens are delivered, not how many are generated or billed. Disabling it would not reduce cost and would make the playground behave differently from production tools that stream. It also degrades the interactive experience engineers expect. The meaningful cost controls are quotas and visibility, not turning off a delivery mechanism that has no bearing on token totals.

Require engineers to file a ticket for every prompt test so usage is reviewed before execution.

Why it fails: Ticket-gated testing destroys the fast iteration loop a playground is meant to create. Review before every run adds latency that pushes engineers to bypass the tool entirely, undermining adoption. Cost control should come from quotas and dashboards that operate automatically, not from manual approvals. Production parity plus automated budgets keeps velocity high while keeping spend predictable.

Give every engineer an unlimited personal API key so experimentation is never blocked.

Why it fails: Unlimited personal keys remove the very guardrail the team wants. Prompt iteration can generate large volumes of tokens quickly, and without per-user budgets or quotas, costs can spike unpredictably. The goal is to enable experimentation while keeping spend visible and bounded. Centralized keys with per-user quotas, plus usage dashboards, achieve both goals better than unrestricted credentials.

Give every team direct access to the raw HTTP layer and encourage them to call the API without the shared library.

Why it fails: Bypassing the shared library reintroduces the duplicated, inconsistent request handling the library was built to eliminate. Teams would each have to solve authentication, retries, and error parsing independently. This fragments the integration surface and makes org-wide changes such as model upgrades far more expensive.

Keep the client API undocumented so teams are forced to read the source code and understand every internal detail.

Why it fails: Forcing teams to read source increases onboarding time and encourages copy-paste of internal details that may change. It also makes the library harder to evolve because callers depend on implementation specifics. This reduces productivity rather than improving it, and it undermines the goal of a stable shared interface.

Require every consuming team to write its own request wrapper so each can tune retries to its own latency profile.

Why it fails: Duplicating wrappers across teams multiplies maintenance and produces inconsistent behavior when the API evolves. Each team re-discovers the same edge cases around retries, timeouts, and error codes. The shared library exists precisely to avoid this duplication, so this practice works against the stated productivity goal.

Route all Claude traffic through a single shared API key managed by the platform team to simplify billing.

Why it fails: A single shared key simplifies billing but destroys per-team cost attribution, which is the opposite of the accountability goal. It also creates a high-value credential that is hard to rotate and a single point of failure. Incident blast radius grows because one team's runaway usage affects everyone. This practice undermines clear cost accountability.

Let each team choose its own SDK, logging format, and retry strategy to maximize autonomy.

Why it fails: Full autonomy recreates the fragmentation the platform team is trying to eliminate. Divergent SDKs, logs, and retry strategies make incidents harder to diagnose and prevent shared tooling from working. Onboarding slows because there is no common path. Autonomy is valuable, but unconstrained variation conflicts with the goals of faster onboarding and fewer incidents.

Mandate that all teams use the same model version and freeze upgrades for a year to reduce variability.

Why it fails: A year-long freeze reduces short-term variability but blocks access to improved models and security fixes, and it does not address onboarding speed or cost accountability. It also forces teams with different latency and quality needs into one tradeoff. Eventually the freeze creates a risky big-bang upgrade. This practice does not support the stated goals.