CCAR-P Practice Test 3 — 30 Questions

Question 1

When designing an agentic system, which TWO of these 'observability' metrics are most crucial for monitoring the health of the agent's reasoning process?

Accepted Answer

Total Step Count per Task.. Tracking 'Step count per task' and 'Tool call success rate' provides immediate visibility into whether the agent is diverging or struggling. An unusually high step count indicates potential infinite loops or circular reasoning. A low tool call success rate reveals integration failures or ambiguous tool definitions. By monitoring these, you can detect system degradation before it impacts the end-user, allowing for proactive debugging and iterative improvements to the agent's core instruction set.

Answer

Average latency of the user's internet connection.

Answer

The color profile of the agent's UI dashboard.

Answer

Number of times the user clicks the refresh button.

Question 2

You are building a system that requires strict adherence to a specific output format. Which approach provides the highest reliability in a high-traffic agentic environment?

Accepted Answer

Implement structured schema enforcement at the model interface layer.. Forcing structured output through system-level schema enforcement at the model interface is the only reliable way to guarantee format consistency in production. By utilizing constrained decoding or rigorous post-processing validation, you ensure that the agent consistently produces machine-readable formats like JSON. This architectural pattern is vital because it removes the variability of natural language, allowing downstream systems to process agent outputs without frequent parsing errors or exceptions.

Answer

Include a long prompt instruction asking the model to be very careful with formatting.

Answer

Use a post-generation regex parser to fix up malformed JSON responses.

Answer

Request the model to output the answer in XML because it is easier for agents.

Question 3

An autonomous agent is designed to browse the web and perform research. Which TWO mechanisms are most critical for preventing infinite loops and excessive API consumption during autonomous tool-calling cycles?

Accepted Answer

Implementing a maximum iteration counter. In autonomous agentic loops, the risk of 'stuck' states or recursive logic is high. Implementing both hard iteration limits and state-based detection ensures the system remains within operational bounds. These safeguards are essential for production-grade agents to prevent runaway costs and to provide a predictable user experience in non-deterministic environments.

Answer

Increasing the context window size

Answer

Using a lower temperature setting

Answer

Enabling prompt caching for tool definitions

Question 4

A product owner asks you to add a feature that lets internal users upload customer contracts so Claude can extract renewal dates and auto-populate a CRM. During intake you learn the contracts contain personally identifiable information and commercially sensitive terms. The product owner wants to launch in three weeks with no legal review. As the architect, what is the most appropriate action?

Accepted Answer

Document the data flows and PII exposure, engage legal and security for a review, and propose a phased launch where extraction runs only on redacted or approved contract types first.. When a requested feature involves PII and sensitive commercial terms, the architect's role is to make the risk visible and enable a compliant path rather than to either block or bypass review. Documenting data flows, engaging legal and security, and scoping an initial launch to approved contract types satisfies the product owner's delivery goal while ensuring regulated data is handled under proper controls.

Answer

Agree to the three-week timeline but process all contract text through an external public model endpoint to reduce internal infrastructure work.

Answer

Launch on schedule and add a note in the backlog to complete the privacy review after the first month of production usage once real data volumes are known.

Answer

Reject the feature outright because contracts always contain sensitive data and no AI system should ever be used for extraction tasks.

Question 5

You are preparing a steering committee update for a Claude-based customer support assistant that has been live for two months. The sponsor asks for a concise way to judge whether the investment is paying off and whether the project should continue to the next phase. Which set of measures best answers that question?

Accepted Answer

Deflection rate, customer satisfaction for AI-handled interactions, average handling time change, and adoption among support agents.. A phase-gate decision needs outcome-oriented evidence, not consumption or engineering trivia. Deflection, satisfaction, handling time, and agent adoption together show whether the assistant improves customer service and is actually used. They also reveal the quality-versus-volume tension that sponsors must weigh, making them the right foundation for a continuation recommendation.

Answer

Model parameter count and the number of prompt templates currently maintained in the repository.

Answer

Total number of API calls made and the cumulative token spend since launch.

Answer

Number of incidents logged and the mean time to resolve production issues over the reporting period.

Question 6

A financial services firm runs a Claude-powered agent that can call internal tools to move funds between accounts. Risk management wants a control that prevents the agent from executing a transfer above a threshold without human sign-off, and that remains effective even if the model is manipulated through injected content in a retrieved document. Which control best meets this requirement?

Accepted Answer

Enforce the threshold in the tool-execution layer so that any transfer exceeding the limit is rejected unless a human approval token is presented.. When an agent can take consequential action, the authorization boundary must live outside the model, in the component that executes the action. Enforcing the threshold and requiring a human approval token at the tool layer means a manipulated model cannot exceed its authority, because the code performing the transfer refuses. Prompt rules, reviewer models, and sampling adjustments modify model behavior but cannot guarantee that a hijacked agent will decline a prohibited action.

Answer

Lower the agent's temperature and restrict its tool list to a single transfer function with a fixed daily cap.

Answer

Add a system prompt rule stating that transfers above the threshold must be escalated to a human operator.

Answer

Run a second Claude instance as a reviewer that inspects each proposed transfer and vetoes suspicious ones.

Question 7

A non-technical stakeholder asks why the Claude assistant sometimes takes several seconds to respond while other requests feel instant. Which explanation is most appropriate?

Accepted Answer

Response time varies with how much text the model must read and produce, so longer prompts and longer answers naturally take more time.. Latency in a Claude-based assistant is dominated by how much text the model must read and how much it must generate, because output is produced token by token. Framing variability this way is technically correct and gives the stakeholder practical levers, such as reducing retrieved context or capping response length.

Answer

The model is randomly load-balanced across servers, so identical requests can land on faster or slower hardware.

Answer

The assistant is learning from each conversation, and it pauses to update its weights after difficult questions.

Answer

Slow responses indicate the request triggered a safety review, and those requests are queued for human approval.

Question 8

A fintech platform runs a Claude-powered transaction summarizer in production. Latency spikes during market open, and the team suspects that requests are being retried unnecessarily when the API returns overloaded errors. They want to make retry behavior observable and tunable without redeploying each service. Which design best meets that goal?

Accepted Answer

Centralize retry logic in a shared client with exponential backoff, jitter, and a configurable maximum attempt count exposed as environment variables, and emit structured metrics for retry reasons and attempt counts.. Centralizing retry behavior in a shared client makes it consistent and configurable, while exponential backoff with jitter prevents synchronized retry storms during market open. Structured metrics on retry reasons and attempt counts provide the observability needed to confirm whether overloaded errors are the trigger, and environment-variable tuning allows adjustment without redeploying each service.

Answer

Route all requests through a fixed-size thread pool and rely on occasional gateway timeouts to shed load, with no client-side retry configuration.

Answer

Disable retries entirely and surface overloaded errors to end users, instructing them to resubmit the transaction manually.

Answer

Have each service catch overloaded errors and immediately retry in a tight loop up to ten times, logging only the final success or failure.

Question 9

You are the lead architect on a Claude-powered internal knowledge assistant. Six weeks into the pilot, the executive sponsor asks for a one-page status update that will be forwarded to the CFO, who has never attended a demo. Which artifact should you produce?

Accepted Answer

A one-page executive brief covering pilot objectives, measured adoption and quality metrics, run-rate cost, top risks, and the decision requested.. A CFO with no demo context needs a forwardable artifact that stands alone: objectives, evidence of value, cost, risk, and a clear ask. The executive brief is the only option that is self-contained, brief, and framed for a financial audience, satisfying both the sponsor's request and the CFO's decision needs.

Answer

A live 30-minute demo for the CFO so she can see the assistant answer real questions.

Answer

The full evaluation notebook with prompt variants, scoring rubrics, and per-case results so the CFO can audit the methodology.

Answer

A forward of the weekly engineering standup notes with a short cover message summarizing the highlights.

Question 10

A bank is deploying a Claude agent that can call internal tools to move funds between accounts. Risk leadership wants a control that limits the blast radius if the agent is manipulated into performing unauthorized transfers. Which control best addresses this requirement?

Accepted Answer

Enforce authorization and transaction limits in the downstream banking APIs the agent calls, independent of anything the model outputs.. When an agent can take consequential actions, enforcement must live outside the model in the systems that hold authority. Server-side authorization, per-transaction caps, and velocity limits ensure that even a fully manipulated agent cannot exceed policy, because the downstream API rejects anything outside its rules. Prompt instructions, sampling settings, and post-hoc monitoring cannot provide that hard boundary.

Answer

Route all agent traffic through a proxy that logs every tool invocation and alerts the security operations center after transfers complete.

Answer

Add a system prompt instructing the model to never perform transfers above a defined threshold or to accounts not previously seen.

Answer

Increase the model's temperature to zero so its responses become fully deterministic and cannot be manipulated.

Question 11

You are the lead architect on a Claude-powered claims triage system that has been in production for six months. A newly appointed VP of Operations asks for a quarterly review of the system's lifecycle status. Which deliverable best satisfies this request while maintaining stakeholder alignment?

Accepted Answer

A lifecycle status report covering adoption metrics, cost trends, incident history, model version currency, and the next-quarter roadmap.. The VP requested a lifecycle status review, which is a governance activity, not a technical artifact or a contracting action. Consolidating adoption, cost, incidents, version currency, and the roadmap into a single report gives the stakeholder the decision-ready picture they need and establishes a repeatable review cadence. Pointing to a model card, raw logs, or a statement of work extension each answers a different question than the one asked.

Answer

A raw export of the last 90 days of API request logs for the VP's team to analyze.

Answer

A revised statement of work that extends the engagement by another two quarters.

Answer

A model card update listing the current Claude model version and its evaluation benchmarks.

Question 12

A developer enablement team is building an internal prompt playground so engineers can iterate on Claude prompts without writing API code. They want the playground to reflect production behavior and avoid surprising cost overruns. (Choose two.)

Accepted Answer

Pin the playground to the same model version and system prompt that production uses, so experiments reflect real behavior.. A playground is only useful if its results predict production, so the model version and system prompt must match what ships. Cost safety comes from automated controls: per-user quotas and usage dashboards. Together these give engineers fast, trustworthy iteration without exposing the organization to unbounded spend. Unlimited keys, disabled streaming, and ticket-gated testing each fail either the parity goal or the velocity goal.

Answer

Disable streaming in the playground to reduce token consumption.

Answer

Require engineers to file a ticket for every prompt test so usage is reviewed before execution.

Answer

Give every engineer an unlimited personal API key so experimentation is never blocked.

Question 13

A developer enablement group is standing up a shared Claude integration library that dozens of internal teams will consume. They want to reduce duplicated work and prevent each team from re-implementing fragile request handling. Which TWO practices best improve developer productivity across those consuming teams? (Choose two.)

Accepted Answer

Publish a typed client library that encapsulates authentication, retries, and error normalization behind a small, stable interface.. A typed client library with a small stable interface centralizes fragile request handling so teams do not each reinvent retries and error normalization. Runnable examples and integration tests lower the cost of adoption and document the recommended patterns. Together these reduce duplicated work and support questions, which is exactly what a developer enablement group is trying to achieve.

Answer

Give every team direct access to the raw HTTP layer and encourage them to call the API without the shared library.

Answer

Keep the client API undocumented so teams are forced to read the source code and understand every internal detail.

Answer

Require every consuming team to write its own request wrapper so each can tune retries to its own latency profile.

Question 14

A developer is building a tool that lets engineers query an internal knowledge base through Claude. During testing, Claude sometimes invents plausible but nonexistent document titles when the retrieved context is thin. The team wants a systematic way to detect and reduce these hallucinations before the tool reaches general availability. Which approach is most appropriate?

Accepted Answer

Build an evaluation set of questions with known ground-truth answers, score responses for faithfulness to the retrieved context, and iterate on retrieval and prompting until scores meet a defined threshold.. Hallucinated titles are a grounding failure, so the fix is to measure faithfulness against retrieved context and improve retrieval and prompting where scores fall short. A labeled evaluation set with ground-truth answers makes the problem quantifiable, and a release threshold converts that measurement into a defensible go or no-go decision.

Answer

Add an instruction telling the model to be accurate and never hallucinate, then rely on that instruction in production.

Answer

Raise the temperature setting so responses become more varied and the model is less likely to repeat a fabricated title.

Answer

Switch the knowledge base queries to return a fixed number of documents regardless of relevance, so the model always has something to cite.

Question 15

A fintech company wants to use Claude to generate personalized financial advice for retail customers. The compliance team mandates that all AI-generated advice must be traceable to a specific model version and configuration for audit purposes. Which governance control best satisfies this requirement?

Accepted Answer

Enable logging of all API requests and responses with model version and configuration metadata.. The correct control is logging API requests and responses with model version and configuration metadata. This creates a detailed audit trail that links each output to the exact model version and settings used, enabling full traceability. Other options improve security or oversight but fail to provide the required technical record for auditing AI-generated financial advice.

Answer

Implement a human review step where a compliance officer approves each piece of advice before it is sent.

Answer

Restrict API access to a whitelist of IP addresses and require multi-factor authentication.

Answer

Use a single, static prompt template that never changes and is stored in version control.

Question 16

A developer support team receives repeated reports that Claude responses in an internal tool are truncated mid-sentence. Logs show the stop_reason value is max_tokens on most affected calls. Which change most directly resolves the truncation while preserving response quality?

Accepted Answer

Increase the max_tokens parameter on the affected requests and verify the model's context window can accommodate input plus output.. The stop_reason value is the authoritative signal: max_tokens means the output ceiling, not the model, ended the response. Raising max_tokens lets the model finish, and checking that input plus output fit the context window prevents a new failure mode. Sampling settings, prompt wording, and streaming do not change how many tokens the model may emit, so they cannot resolve this truncation.

Answer

Set temperature to zero so the model produces shorter, more deterministic completions.

Answer

Enable streaming so partial responses are delivered before the token limit is reached.

Answer

Add a system prompt instruction telling the model to always finish its sentences.

Question 17

A team is building an agent that must call a payment provider's API. The API requires an idempotency key on every charge request, and the agent may retry a charge if a network error occurs mid-call. Which design correctly preserves financial correctness when retries happen?

Accepted Answer

Derive the idempotency key deterministically from the logical charge (for example, a hash of order ID and amount) and reuse that same key across all retries of that charge.. The idempotency key is the provider's contract for recognizing repeated attempts as one logical charge. Deriving it deterministically from stable charge attributes means every retry, even after a process restart or agent replan, presents the same key, so the provider returns the original outcome instead of charging again. This preserves correctness under at-least-once retry semantics without sacrificing resilience.

Answer

Generate a fresh idempotency key for each retry attempt so the provider can distinguish a retry from a genuinely new charge.

Answer

Disable automatic retries for charge calls and surface every network error to a human operator for manual reconciliation.

Answer

Omit the idempotency key and instead check the account balance before each retry to see whether the charge already landed.

Question 18

A platform team is standardizing how internal teams integrate Claude. Leadership wants faster onboarding, fewer production incidents, and clear accountability for cost. Which TWO practices best support these goals? (Choose two.)

Accepted Answer

Publish a golden-path reference implementation with sample prompts, error handling, and a checklist for going to production.. A golden-path reference implementation accelerates onboarding and reduces incidents by giving teams proven patterns, while per-service tagging with a shared usage dashboard makes cost attributable and accountable. Freezing model versions, allowing full tooling autonomy, and centralizing one shared key each work against at least one of the stated goals, so they are not the right practices here.

Answer

Route all Claude traffic through a single shared API key managed by the platform team to simplify billing.

Answer

Let each team choose its own SDK, logging format, and retry strategy to maximize autonomy.

Answer

Mandate that all teams use the same model version and freeze upgrades for a year to reduce variability.

Question 19

A software company's internal AI review board is defining escalation criteria for its Claude-powered support assistant. The board wants a rule that reliably routes the highest-consequence cases to human specialists rather than relying on the model's own confidence statements. Which escalation design best achieves this?

Accepted Answer

Route cases to specialists based on objective risk signals such as account tier, regulatory keywords, and prior complaint history.. Reliable escalation must be driven by signals that exist independently of the model's self-assessment. Objective criteria such as account tier, regulatory keywords, and complaint history correlate directly with consequence and can be evaluated deterministically, guaranteeing that the cases the board cares most about reach a specialist before a reply is finalized.

Answer

Ask the assistant to flag any conversation it finds ambiguous and forward those to specialists for a second opinion.

Answer

Instruct the assistant to state a confidence percentage with each answer and escalate whenever it reports below ninety percent.

Answer

Sample five percent of all conversations at random for specialist review after the assistant has already replied.

Question 20

You are the lead architect on a Claude-powered contract-analysis platform. Two weeks before go-live, the General Counsel asks for written assurance that the system will not retain client contract text for model training and that all data stays within the EU. Which artifact should you produce first to satisfy this governance obligation?

Accepted Answer

A data-flow and retention attestation that maps contract text through the Claude API, documents Anthropic's zero-data-retention configuration, and confirms the EU-only processing region.. The stakeholder needs verifiable evidence about two distinct properties: retention behaviour and processing location. Only a data-flow and retention attestation both traces the path of contract text through the Claude API and records the concrete configuration choices, such as zero-data-retention and EU-region processing, that make the assurance true. Schedule changes, redaction habits, and risk logs are adjacent activities that do not answer the question.

Answer

A prompt-engineering guideline telling analysts to strip client names from contract text before submitting it.

Answer

An updated project charter that adds a legal-review milestone to the schedule before the go-live date.

Answer

A risk-register entry logging 'data residency uncertainty' with a medium severity rating and an owner.