CCAR-P Practice Question: Stakeholder Communication and Lifecycle Management
Six months after launch, a logistics company's operations VP reports that the Claude-based exception-handling assistant 'used to be great and now gives worse answers,' though no code has changed. You confirm the application code and system prompt are untouched. What is the most likely explanation you should investigate first?
⚠ Common exam trap
The trap here is assuming that unchanged code implies unchanged behaviour, when the inputs reaching the system are the more likely variable in a maturing deployment.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The assistant is now encountering a broader distribution of exception types than during the pilot, including cases that were out of scope for the original evaluation set.
When code, prompt, and model snapshot are unchanged but users report degradation, input distribution drift is the leading hypothesis. Early pilots exercise a narrow set of exception types chosen by the implementation team, while production usage expands into long-tail cases the evaluation never covered. Measuring the current mix of incoming exception categories against the original test set either confirms drift or rules it out, and it is a cheap first check before investigating infrastructure.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The prompt caching layer has expired its entries, forcing the model to reason without the operational context it previously had.
Why it's wrong here
Prompt caching reduces cost and latency for repeated prefixes; it does not supply reasoning context that the model otherwise lacks, and cache misses do not change the content the model conditions on. An expired or missed cache would show up as a cost or latency change, not as worse answers. This misattributes a content-quality symptom to a performance optimization mechanism.
- ✓
The assistant is now encountering a broader distribution of exception types than during the pilot, including cases that were out of scope for the original evaluation set.
Why this is correct
A stable application facing a shifting input distribution will appear to degrade even when nothing in the code changed. As the assistant gains adoption, users bring exception categories the pilot never covered, and accuracy on those unfamiliar cases is naturally lower. Comparing current input distributions against the original evaluation set is the fastest way to confirm whether the workload itself has drifted.
- ✗
Latency has increased over time, and the operations team is now perceiving slower responses as lower quality answers.
Why it's wrong here
Perceived quality and measured response time are separate signals, and nothing in the report indicates the VP is conflating them. Assuming a perception error dismisses a substantive complaint about answer correctness without any supporting measurement. If latency were genuinely elevated, that would be a distinct issue to diagnose, but it does not explain incorrect or less useful exception handling.
- ✗
The model provider has silently retrained the underlying model, so previously correct behaviours have been overwritten without notice.
Why it's wrong here
If the application pins a dated model snapshot, the served model does not change underneath it, so this cannot explain the regression without evidence of an actual version change. Even where an alias is in use, the first diagnostic step is to verify which snapshot is being served rather than assuming retraining. Asserting silent retraining without checking is an unfounded escalation that misdirects the investigation.
About these practice questions
Courseiva writes every CCAR-P question from scratch — 262 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Anthropic exam blueprint
This CCAR-P practice question is part of Courseiva's free Anthropic certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the CCAR-P exam.