CCAR-P Practice Test 18 — 15 Questions

Question 1

Refer to the exhibit. An audit of an Anthropic API configuration reveals the policy shown. What is the primary governance concern with this implementation?

Accepted Answer

The safety settings expose the organization to content moderation and liability risks.. The configuration explicitly disables core safety filters for harassment and hate speech. In an enterprise context, this creates significant legal and brand risk. Governance frameworks mandate that safety guardrails remain enabled to protect the organization from generating or facilitating harmful content. By explicitly setting these to 'block_none', the organization bypasses essential protections that are designed to uphold safety standards and ethical AI usage requirements.

Answer

The temperature setting is too high for professional administrative tasks.

Answer

The max_tokens limit is too low, restricting the capability of the system.

Answer

The system prompt is too generic, leading to poor model performance.

Question 2

An organization requires strict adherence to data residency requirements for PII processed by Claude. Which strategy best ensures that customer prompts and completions remain within a specific geographic boundary while utilizing Anthropic's API?

Accepted Answer

Configure the API client to target region-specific endpoints and verify regional data residency settings.. Implementing region-specific API endpoints combined with Data Residency configurations ensures that data processing and storage occur within authorized borders. This approach is critical for regulatory compliance in jurisdictions like the EU. By limiting the scope of model interaction to regional infrastructure, architects prevent the inadvertent cross-border transfer of sensitive data, thereby aligning with global privacy governance frameworks and minimizing legal exposure for the enterprise.

Answer

Enable global load balancing across all available Anthropic regions to optimize latency.

Answer

Encrypt all outgoing prompts using a third-party gateway before sending to Anthropic.

Answer

Anonymize all data locally and rely on Anthropic's general model training to handle PII.

Question 3

When designing an LLM application, what is the best strategy for managing 'system instructions' (system prompts) to prevent unauthorized alteration?

Accepted Answer

Store the system prompt in a centralized, secured configuration service.. System instructions should be stored in a secured, backend configuration service that is injected at runtime, rather than being hardcoded or exposed to the client-side. This architecture ensures that the system prompt is immutable from the user's perspective. By centralizing the storage and management of these instructions, architects provide a secure, auditable method for updating behavior without exposing the underlying logic to potential client-side manipulation or injection attacks.

Answer

Hardcode the system prompt directly in the client-side JavaScript for speed.

Answer

Allow users to modify the system prompt to customize their experience.

Answer

Use a public Git repository to store the system prompts for transparency.

Question 4

An agentic system often experiences 'goal drift' when managing long-running, multi-step tasks. Which architectural pattern most effectively mitigates this risk during recursive reasoning chains?

Accepted Answer

Integrating a reflection loop that evaluates progress against the initial goal.. State-space re-grounding via periodic reflection loops allows the agent to compare current progress against original user intent. By forcing a dedicated 'evaluator' step that analyzes the chain of thought against the task definition, the system can self-correct before executing irreversible actions. This prevents the agent from spiraling into irrelevant sub-tasks that deviate from the primary objective, ensuring high task fidelity in complex autonomous workflows.

Answer

Increasing the context window size to include all previous turns.

Answer

Implementing a hard-coded decision tree for every possible action.

Answer

Reducing the temperature parameter to zero for all model calls.

Question 5

To ensure organizational security and governance when using Anthropic's API, what is the best practice for managing API keys across a team of 50 developers?

Accepted Answer

Use a centralized secret management service to store and dynamically inject keys.. Using a centralized secret management service (like AWS Secrets Manager or HashiCorp Vault) is the industry standard for managing sensitive credentials. It allows for auditing, automatic rotation, and granular access control, ensuring that only authorized services and developers can access the keys. This approach drastically reduces the risk of accidental exposure and allows security teams to monitor usage effectively, which is essential for enterprise-scale operations.

Answer

Store keys in a shared environment file (.env) in a private GitHub repository.

Answer

Create one master API key and share it among all team members via Slack.

Answer

Hardcode the keys in each microservice for faster startup times.

Question 6

A project stakeholder expresses concern that an Anthropic-powered content moderation system is generating unexpected output bias. As the lead architect, how should you communicate the resolution strategy while managing expectations?

Accepted Answer

Outline a documented evaluation process using a golden dataset to measure bias before and after applying specific safety guardrails.. Effective architect communication requires transitioning from abstract concern to empirical validation. By proposing a phased audit approach, you demonstrate technical rigor and transparency. This builds trust by confirming the issue is being treated as a high-priority architectural defect rather than a minor configuration tweak. Maintaining a clear line of communication regarding the evaluation methodology helps stakeholders understand the inherent limitations of LLMs while ensuring the mitigation path is grounded in verifiable data.

Answer

Immediately disable all moderation features until a full audit is completed.

Answer

Inform the stakeholder that LLM behavior is non-deterministic and inherent to the model.

Answer

Task the engineering team to manually review every output until the stakeholder is satisfied.

Question 7

Which governance practice is most effective for managing 'Third-Party Model Risk'?

Accepted Answer

Conducting vendor due diligence and establishing SLAs.. Managing third-party model risk involves creating a clear vendor management strategy that includes rigorous due diligence, transparency requirements, and contractual guarantees regarding safety and performance. By treating AI providers as critical vendors, organizations ensure that the model provider is held accountable for their platform's safety. This allows the organization to align the provider's capabilities with their own internal risk tolerance and compliance requirements.

Answer

Allowing free access to all models.

Answer

Only using internal models.

Answer

Relying on public trust instead of contracts.

Question 8

A financial services firm is deploying an AI agent to handle diverse requests including balance inquiries, market analysis, and loan applications. To minimize latency and maximize precision, the architect decides to implement a pattern where a primary model classifies the intent and delegates to specialized sub-agents. Which architectural pattern is being described?

Accepted Answer

Router Pattern. Implementing a router architecture allows the system to isolate logic into specialized modules, which significantly reduces the cognitive load on the LLM. By dynamically selecting only the necessary tools for each specific query, the architect ensures that the model remains focused on relevant parameters, leading to higher precision, lower token consumption, and a more maintainable codebase over time.

Answer

Monolithic Chain

Answer

Sequential Pipeline

Answer

Parallel Execution

Question 9

Refer to the exhibit. A stakeholder reports that the output is too verbose. Which change would best address the stakeholder requirement while maintaining model performance and architectural standards?

Accepted Answer

Update the system prompt to explicitly state: 'You are a financial analyst. Provide only bulleted summaries. Do not exceed 100 tokens per response.'. Optimizing the system prompt is the most efficient way to influence model behavior without changing the underlying architecture or model version. By explicitly constraining the output format within the system instruction, you provide a clear boundary for the model. This satisfies the stakeholder's request for brevity while demonstrating proactive lifecycle management and iterative refinement of the AI's utility to the business.

Answer

Replace 'claude-3-5-sonnet-20240620' with 'claude-3-haiku-20240307'.

Answer

Hard-code a truncation script in the application layer to cut off responses after 500 characters.

Answer

Advise the stakeholders that verbosity is a byproduct of the model's intelligence and cannot be altered.

Question 10

A developer wants to reduce the cost and latency of their LLM application, which sends repetitive, long context windows. Which optimization technique is most appropriate?

Accepted Answer

Implement prompt caching for static segments of the context window.. Prompt Caching is designed specifically to handle large, static context segments by allowing the model to reuse the computed state. By caching parts of the prompt, the application avoids redundant processing, which directly reduces both latency and cost. This technique is a crucial operational enabler for developers building complex apps that rely on large knowledge bases or extensive instructions.

Answer

Aggressively compress all input text using standard text compression algorithms.

Answer

Switch to a smaller model version regardless of performance requirements.

Answer

Move all conversation processing to the client-side browser to offload the server.

Question 11

You are leading a project involving Claude integration. Which TWO of the following steps are essential for successful stakeholder expectation management during the model evaluation phase?

Accepted Answer

Define specific, measurable KPIs related to task completion time and accuracy rates.. Managing expectations requires balancing technical transparency with business clarity. By defining clear success metrics and documenting the limitations of LLMs, you prevent misalignment between the technical performance and business outcomes. These actions ensure that stakeholders are aware of both the potential and the constraints of the system, fostering a collaborative environment where iterations can occur based on real data rather than misguided assumptions about artificial intelligence capabilities.

Answer

Promise 100% accuracy for all complex logic tasks to secure stakeholder buy-in.

Answer

Exclude non-technical stakeholders from the evaluation process to maintain focus.

Answer

Automate all stakeholder status reports to avoid any human interaction.

Question 12

When evaluating the performance of Claude for a new feature, which metric is most useful for understanding the impact on end-user experience?

Accepted Answer

Time to First Token (TTFT).. Time to First Token (TTFT) is the most critical metric for perceived latency. Even if the full response takes a long time to generate, users feel that an application is responsive if the initial tokens appear quickly. By prioritizing TTFT, developers can ensure that the application feels 'fast' and interactive, which is the most significant factor in maintaining user engagement and perceived product quality in LLM-powered applications.

Answer

Total number of API calls made per month.

Answer

The total number of tokens generated per response.

Answer

The memory usage of the server hosting the API client.

Question 13

An organization wants to enforce consistent prompt engineering standards across teams. They have a large library of prompts and need to ensure that developers use approved versions while maintaining audit trails of prompt usage. What is the most effective approach?

Accepted Answer

Deploy a central prompt management service that provides versioned API endpoints for prompt retrieval.. Using a centralized Prompt Registry allows teams to version-control, audit, and distribute approved prompt templates. This ensures governance without hindering developer velocity, as teams can reference specific versions through API calls rather than hardcoding prompts. Establishing a registry mitigates risks associated with prompt injection and inconsistent model behavior across different product features, effectively bridging the gap between security compliance and rapid software iteration.

Answer

Require developers to manually copy-paste prompt templates into a shared documentation Wiki.

Answer

Implement a custom middleware layer that encrypts all prompt traffic at the network edge.

Answer

Enforce strict hardcoding of all prompts within the application source code with mandatory PR reviews.

Question 14

An architect needs to implement a 'Red Teaming' process for a new Claude deployment. What is the primary objective of this activity in the context of AI governance and safety?

Accepted Answer

To adversarialy test the model to find safety gaps and potential for misuse.. Red Teaming is a proactive safety practice where a group of testers tries to find vulnerabilities, biases, or ways to make the model fail. This is a critical component of risk management, as it identifies potential issues before they can affect real users in a production environment.

Answer

To optimize the model's latency and throughput for high-volume traffic.

Answer

To automate the generation of unit tests for the application's UI components.

Answer

To verify that the model's billing and usage credits are being tracked correctly.

Question 15

In the Anthropic tool-use workflow, what is the primary purpose of the 'system prompt' relative to tool usage?

Accepted Answer

To establish the agent's persona and tool-use guidelines. The system prompt serves as the 'instruction manual' for the agent. It defines the agent's identity, its overall goals, and the constraints within which it must operate. While tool schemas define the 'how,' the system prompt provides the 'why' and 'when,' guiding the model's decision-making process for tool selection.

Answer

To provide the JSON schema for each tool

Answer

To store the results of previous tool executions

Answer

To define the API keys for the external services