CCAR-P Practice Test 16 — 15 Questions

Question 1

Refer to the exhibit. The developer reports that the model is cutting off summaries for very long inputs. What is the most likely cause?

Accepted Answer

The max_tokens setting is too low for the requested output length.. The 'max_tokens' limit dictates the maximum size of the generated completion, not the input size. If the generated summary exceeds this limit, the output will be truncated. To fix this, developers must ensure the 'max_tokens' parameter is sufficiently large to accommodate the desired output length. Understanding the interaction between token limits and output requirements is crucial for operational stability in LLM-based text summarization tasks.

Answer

The input text exceeds the model's total context window size.

Answer

The system prompt is too short to handle long-form summarization.

Answer

The model version being used does not support large outputs.

Question 2

You are presenting the roadmap for a multi-stage LLM implementation to non-technical stakeholders. What is the most effective approach?

Accepted Answer

Frame the roadmap around business goals and performance milestones.. Focusing on business outcomes rather than model mechanics is key when communicating with non-technical stakeholders. By tying each phase to a specific value proposition or risk reduction, the architect makes the project tangible. This is critical because stakeholders are invested in the business impact, and excessive technical jargon can obscure the ROI, leading to a loss of project support or misaligned expectations regarding deliverables.

Answer

Detail every technical parameter and model architecture variation.

Answer

Present the technical hurdles as the primary focus to manage expectations.

Answer

Exclude all risks to ensure the stakeholders approve the budget.

Question 3

A stakeholder asks why the AI system occasionally provides different answers to the same question. How do you explain this?

Accepted Answer

Explain that the model uses a 'temperature' parameter to balance variety and predictability, which is key to its generative nature.. Explaining the concept of model temperature and non-determinism is crucial for managing expectations about AI behavior. By framing it as a balance between creativity and consistency, you help stakeholders understand that this variability is a fundamental aspect of generative AI. This allows them to make informed decisions about whether to adjust parameters for their specific use case, ensuring they have the right tool for the job while maintaining realistic expectations about performance.

Answer

Tell them it is a bug and that the team is working on a fix for consistency.

Answer

Blame the training data for being inconsistent and poorly structured.

Answer

State that the system is broken and should be shut down until it is perfect.

Question 4

When designing an LLM-based application, what is the primary benefit of using a 'System Prompt' compared to embedding instructions in the user message?

Accepted Answer

It provides a dedicated space for behavioral instructions, improving consistency and safety.. System prompts define the core persona, constraints, and operational boundaries of the model, which remains consistent throughout the session. This separation of concerns improves developer productivity by keeping the application logic clean and separating user-provided data from behavioral instructions. It also helps prevent prompt injection attacks, as the model is explicitly instructed to treat the system prompt as a higher-priority directive compared to the user's input.

Answer

It significantly reduces the latency of every request sent to the model.

Answer

It allows the model to cache the entire user conversation history automatically.

Answer

It eliminates the need for any further user input during the conversation.

Question 5

Refer to the exhibit. Your application is hitting rate limits. A stakeholder is concerned about the user impact. What is your communication strategy?

Accepted Answer

Explain the rate limit, the automatic retry mechanism, and the scaling plan.. When hit by technical constraints, the architect must communicate the limitation, the current mitigation (retry logic), and the plan for long-term scaling. This is vital because stakeholders need to know that the system is self-healing, but also that you are actively working on increasing capacity. Clear communication prevents panic and demonstrates that the system was designed with resilience in mind, maintaining stakeholder trust during temporary service interruptions.

Answer

Tell them the service is unusable and wait for Anthropic to fix it.

Answer

Ignore the issue until the stakeholder asks about it again.

Answer

Blame the engineering team for poor code quality.

Question 6

Your team is building an LLM-powered application and experiencing high latency during peak times. Which TWO actions would best improve developer productivity and operational efficiency when debugging these bottlenecks?

Accepted Answer

Implement distributed tracing with custom spans for model inference and prompt processing.. Observability and granular tracing are critical for diagnosing LLM latency. By capturing token usage and latency metrics per request, developers can pinpoint whether the bottleneck is model inference, networking, or pre-processing. These insights enable targeted optimizations like prompt caching or streaming, which are essential for scaling production-grade generative AI applications while keeping developer workflows focused on high-impact performance improvements rather than guesswork.

Answer

Switch all internal communications to asynchronous polling to avoid blocking operations.

Answer

Force all developers to use the largest available model to ensure high quality results.

Answer

Disable all logging and monitoring to minimize overhead on the network layer.

Question 7

When designing a stakeholder status dashboard for an AI implementation project, which THREE metrics are most critical to include to demonstrate success and maintain alignment?

Accepted Answer

Average latency (time-to-first-token).. Selecting the right KPIs is vital for stakeholder buy-in. By focusing on latency, cost per request, and error rates, you provide a balanced view of system performance and operational health. These metrics allow stakeholders to track ROI and identify potential bottlenecks, ensuring that the architectural decisions align with the business's operational goals and that the lifecycle of the AI application is transparently managed throughout the production deployment phase.

Answer

Model training loss values.

Answer

Number of lines of code written to build the UI.

Question 8

When implementing a 'Human-in-the-loop' (HITL) checkpoint, what is the best way to handle the state persistence during the wait period?

Accepted Answer

Persist the state in an external database and terminate the process.. Externalizing state to a persistent database is essential. Agentic systems are often stateless by design, but a wait period requires 'pausing' execution. Saving the entire conversation history, current tool state, and reasoning chain allows the process to resume exactly where it left off once human approval is received. This prevents data loss and maintains context continuity, which is critical for long-running, complex workflows.

Answer

Keep the connection open to avoid re-initializing the agent.

Answer

Store the state only in the client-side browser memory.

Answer

Retry the tool call periodically until the human responds.

Question 9

When designing a system that uses Claude for data extraction, how should a developer handle non-deterministic outputs to ensure operational reliability?

Accepted Answer

Implement a post-processing validation layer that checks the output schema.. Operational reliability in data extraction is achieved through structural constraints and validation. By forcing the model to output specific formats and validating those outputs against a predefined schema, developers can mitigate the risks of non-deterministic LLM behavior. This approach ensures that downstream systems receive the data they expect, turning the variability of generative AI into a manageable input for traditional software engineering components.

Answer

Set the temperature to zero to guarantee the exact same output every time.

Answer

Ask the model to explain why it extracted the data in a specific way.

Answer

Use a larger model to reduce the probability of hallucinated extraction formats.

Question 10

Refer to the exhibit. Your agent is receiving this error during a high-concurrency operation. Which implementation correctly handles this scenario?

Accepted Answer

Wait for the duration specified in 'retry_after' before retrying.. The correct implementation is to respect the 'retry_after' header and back off before attempting another call. This prevents the system from entering a crash loop where it hammers the API with failed requests. By implementing an exponential backoff strategy that honors the provided delay, you ensure the agent remains resilient to transient load issues while complying with the service provider's rate constraints, ultimately increasing the reliability of the overall system.

Answer

Discard the task and return a failure message to the user.

Answer

Immediately retry the request without any delay.

Answer

Switch to a backup API key and continue immediately.

Question 11

Which TWO metrics are most useful for evaluating developer productivity in an LLM-driven organization? (Choose TWO)

Accepted Answer

Mean time to deploy a prompt improvement to production.. Measuring productivity in LLM development requires balancing speed of iteration with the quality of the output. Metrics like mean time to deploy prompt changes and the success rate of automated evaluations provide a clear picture of how quickly and effectively a team can work. These indicators help identify bottlenecks in the CI/CD pipeline and ensure that improvements are actually enhancing the system's overall performance rather than just adding complexity.

Answer

Total number of prompts written by a developer each week.

Answer

Total API usage cost per day for the entire organization.

Answer

Number of lines of code written in the application backend.

Question 12

Which THREE technical strategies best support scaling prompt engineering across a large organization?

Accepted Answer

Implement automated CI/CD pipelines for prompt testing and deployment.. Scalable prompt engineering requires treating prompts as code. This includes using version control, automated testing, and modular prompt design. By implementing these practices, organizations create a repeatable and transparent workflow that allows developers to iterate safely. These strategies reduce the risk of regressions and enable effective collaboration, which is fundamental to maintaining high-quality AI outputs at scale without slowing down the overall development velocity.

Answer

Encourage developers to share prompt snippets via chat platforms.

Answer

Limit access to the Claude API to a single dedicated team.

Question 13

A team notices that Claude's performance fluctuates for a specific task. What is the most logical step to stabilize the output?

Accepted Answer

Embed several high-quality examples of the task into the prompt.. Stability in LLM outputs is best achieved through Few-Shot Prompting. By providing high-quality, representative examples within the prompt, developers reduce the ambiguity the model faces, ensuring more consistent reasoning. This technique is a cornerstone of professional prompt engineering, as it guides the model towards the desired behavior and format, significantly improving reliability across varying inputs and reducing the need for constant, manual fine-tuning or excessive prompt complexity.

Answer

Increase the temperature to its maximum value to explore more possibilities.

Answer

Rewrite the instructions to be longer and more descriptive.

Answer

Add a post-processing step to filter out low-quality outputs.

Question 14

A team wants to transition from a proof-of-concept to a production environment. Which task should be prioritized for operational readiness?

Accepted Answer

Implement structured logging and automated evaluation pipelines.. Implementing automated monitoring, logging, and robust error handling is the priority for moving to production. While prototyping focuses on functionality, production focuses on reliability, observability, and security. By establishing these foundations early, teams prevent operational debt and ensure that their AI systems can be maintained and scaled effectively as they grow, which is critical for long-term project success and developer support.

Answer

Hardcode the highest possible system prompt complexity.

Answer

Switch to a private, self-hosted version of the Claude model.

Answer

Remove all caching mechanisms to ensure real-time data accuracy.

Question 15

A team is concerned about data privacy. They want to ensure that no personally identifiable information (PII) is sent to the LLM. What is the most effective way to manage this in a developer-friendly way?

Accepted Answer

Deploy a middleware proxy layer that detects and redacts PII before model submission.. Building a middleware proxy for PII redaction ensures that sensitive data is scrubbed before it ever leaves the company's network. By automating this at the infrastructure level, developers are freed from the responsibility of manual redaction, reducing the risk of human error. This approach balances developer productivity with strict security compliance, making it an essential operational pattern for enterprise-scale LLM adoption.

Answer

Provide all developers with a PII detection manual and perform monthly audits.

Answer

Only use the LLM to process public data that has been vetted by legal teams.

Answer

Require developers to encrypt all prompt text using AES-256 before sending.