CCAR-P Advanced Agentic Architecture Practice Question
When evaluating the performance of a new 'Orchestrator-Worker' agentic architecture, which THREE metrics provide the most insight into the system's efficiency and reliability?
⚠ Common exam trap
Candidates often select business metrics like 'User Satisfaction' or 'Revenue Impact'. These are lagging indicators that do not provide the granular technical feedback necessary to debug agentic reasoning loops or tool usage.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Task Success Rate (TSR)
Evaluating agents requires looking beyond simple accuracy to understand the operational characteristics of the loop. Task success rate measures the ultimate goal, while steps-per-task and tool accuracy pinpoint where the logic might be breaking down or becoming inefficient. These metrics collectively guide the architect in refining the agent's reasoning path.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Task Success Rate (TSR)
Why this is correct
TSR is the primary indicator of whether the agent is actually fulfilling its intended purpose. It measures the percentage of sessions where the agent reaches a correct and verified conclusion, providing a high-level view of the architecture's effectiveness across a diverse set of test cases or user queries.
- ✗
Total number of tools defined in the schema
Why it's wrong here
The quantity of tools available to an agent is a configuration detail, not a performance metric. While having too many tools can negatively impact model performance, simply counting them does not provide any insight into how well the agent is actually performing its tasks or utilizing those tools in practice.
- ✓
Average Steps per Task
Why this is correct
Tracking the number of turns or iterations required to complete a task helps identify efficiency issues. An agent that takes 20 steps to solve a problem that should take 3 steps is likely stuck in a loop or following an inefficient reasoning path, leading to higher costs and latency.
- ✗
The model's pre-training data cutoff date
Why it's wrong here
The knowledge cutoff is a static property of the underlying model and does not reflect the performance of the agentic architecture. In an agentic system, the model relies on tools to access real-time data, making the training cutoff less relevant than the agent's ability to successfully query and interpret tool results.
- ✓
Tool Call Accuracy Rate
Why this is correct
This metric measures how often the agent selects the correct tool and provides correctly formatted arguments. Low accuracy here indicates that the model is struggling with the tool definitions or the context, which is a critical signal for the architect to improve the documentation or simplify the toolset.
About these practice questions
Courseiva writes every CCAR-P question from scratch — 262 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Anthropic exam blueprint
This CCAR-P practice question is part of Courseiva's free Anthropic certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the CCAR-P exam.