CCAO-F Safety and Responsible Use Practice Test 2 — 15 Questions

Question 1

A user asks Claude to help them write a convincing phishing email to steal credentials from their coworkers. Claude refuses. How does this refusal benefit the 'Safety and Responsible Use' ecosystem?

Accepted Answer

It prevents the model from being used to facilitate cyberattacks and social engineering.. By refusing to generate phishing content, Claude prevents its technology from being used as a tool for cybercrime. This aligns with Anthropic's goal of ensuring AI is a force for good. Such refusals protect potential victims, reduce the burden on cybersecurity teams, and maintain the model's reputation as a safe and helpful assistant for legitimate tasks.

Answer

It ensures the model remains competitive with other AI providers who also refuse.

Answer

It helps the user learn how to write better phishing emails on their own.

Answer

It allows the model to save its processing power for more complex coding tasks.

Question 2

An enterprise is concerned about 'indirect prompt injection'—where Claude might process malicious instructions hidden in a third-party website it is summarizing. Which TWO methods are most effective for mitigating this safety risk?

Accepted Answer

Instructing the model in the system prompt to treat all external data as untrusted text.. Indirect prompt injection occurs when the model follows instructions found within the data it is processing rather than from the user. Mitigating this requires a combination of robust system prompts that prioritize user instructions and post-processing filters that detect when the model is deviating from its intended task due to external content influence.

Answer

Increasing the model's temperature to 1.0 to make its responses more creative.

Answer

Disabling all safety filters to allow the model to process the hidden instructions freely.

Answer

Limiting the model's output to only 10 words to prevent complex responses.

Question 3

Which of the following best describes the 'Shared Responsibility Model' for safety when using Anthropic's API?

Accepted Answer

Anthropic secures the model and base safety, while the customer secures their application and use case.. The Shared Responsibility Model clarifies that both Anthropic and the customer have roles in ensuring safety. Anthropic provides a secure and aligned model, while the customer is responsible for how they implement and use that model in their specific application. This partnership is essential for the safe and ethical deployment of AI across different industries.

Answer

Anthropic is 100% responsible for any output the model generates in any context.

Answer

The user is 100% responsible for all model behavior, and Anthropic provides no safety features.

Answer

Both parties are responsible for sharing the financial costs of any legal disputes.

Question 4

When Claude is asked to generate instructions for a dangerous and illegal activity, such as manufacturing a prohibited substance, it provides a refusal. This refusal is a direct result of which Anthropic design choice?

Accepted Answer

The 'harmlessness' training within the Constitutional AI framework.. Anthropic's commitment to safety is baked into the model's architecture through Constitutional AI. The refusal to assist with illegal or dangerous activities is a deliberate design choice to ensure that the model cannot be used to cause physical harm. This is a core part of the 'harmlessness' principle that guides all Anthropic model development.

Answer

The model's inability to understand complex chemical formulas.

Answer

A manual review of the prompt by an Anthropic safety officer.

Answer

A request from the user's internet service provider to block the content.

Question 5

An enterprise developer is building a customer support bot using Claude. During testing, the model refuses to answer a query about a competitor's product, stating it cannot discuss other brands. This behavior is considered an over-refusal. Which pillar of the 'Helpful, Honest, Harmless' (HHH) framework is primarily being misapplied in this instance?

Accepted Answer

Helpfulness. The HHH framework requires a delicate balance between being useful to the user and avoiding harm. In this scenario, the model is prioritizing a perceived safety boundary over its core duty to be helpful. Over-refusals occur when the model interprets safety guidelines too broadly, resulting in a failure to provide legitimate information that does not actually violate any safety policies.

Answer

Harmlessness

Answer

Honesty

Answer

Confidentiality

Question 6

Refer to the exhibit. When this request is sent to Claude, the model responds: 'I cannot fulfill this request. I am programmed to be a helpful and harmless AI assistant, and I cannot assist with requests related to cyberattacks or illegal activities.' Which safety mechanism is primarily responsible for this specific refusal?

Accepted Answer

Internal safety guardrails and alignment training.. Claude's refusal is a result of safety training and alignment, specifically designed to prevent the model from assisting in illegal or harmful activities like cyberattacks. The model identifies the harmful intent in the user's prompt and, based on its internal safety guardrails and Constitutional AI training, generates a refusal message instead of the requested harmful content.

Answer

The system prompt provided in the exhibit.

Answer

A regex-based keyword filter at the API level.

Answer

The user's lack of administrative privileges in the API console.

Question 7

A healthcare organization is using Claude to summarize patient notes. To ensure the responsible use of the AI and protect patient privacy, which action should the organization take before sending data to the Claude API?

Accepted Answer

De-identify or redact all PII/PHI from the notes locally.. Protecting Personally Identifiable Information (PII) and Protected Health Information (PHI) is a critical component of responsible AI use. Organizations must ensure that sensitive data is handled according to legal standards like HIPAA. De-identifying or anonymizing data before it reaches the AI provider is a primary security measure to prevent accidental exposure.

Answer

Enable the 'Public Data Sharing' option in the Anthropic console.

Answer

Instruct the model via the system prompt to ignore all names.

Answer

Use the 'Honest' parameter in the API request body.

Question 8

A researcher is attempting to test Claude's safety limits by using a complex prompt that instructs the model to 'act as a persona that has no moral constraints.' This is an example of which type of safety threat?

Accepted Answer

Jailbreaking. Jailbreaking attempts involve using specific prompt engineering techniques to bypass the model's safety filters or alignment training. By instructing the model to adopt a persona that ignores its core principles, users try to trick the AI into generating harmful, biased, or restricted content that it would otherwise refuse in a standard context.

Answer

Prompt Injection

Answer

Data Poisoning

Answer

Model Inversion

Question 9

Under the Shared Responsibility Model for AI safety, which THREE tasks are primarily the responsibility of the customer (the developer using Claude)?

Accepted Answer

Implementing application-level content moderation and monitoring.. Safety is a shared effort between Anthropic and its customers. Anthropic is responsible for the base model's safety and infrastructure, while customers are responsible for how they implement the model, the data they feed it, and the monitoring of their specific application to prevent misuse in their unique business context.

Answer

Aligning the base model using Constitutional AI principles.

Answer

Maintaining the physical security of the data centers hosting Claude.

Question 10

Refer to the exhibit. A developer receives this JSON response while trying to generate a response from Claude. What does this error message signify regarding Anthropic's safety systems?

Accepted Answer

The model's generated output violated a safety threshold and was suppressed.. This error indicates that the model's output was blocked by an automated safety filter. Anthropic uses various layers of safety, including the model's internal refusal logic and external filters that scan for specific types of harmful content. This ensures that even if a model were to generate a harmful response, it is caught before being returned.

Answer

The user's API key has been permanently banned for a policy violation.

Answer

The request was blocked because the prompt was too long for the model.

Answer

The developer forgot to include the mandatory 'safety_check' parameter.

Question 11

When Claude provides a response that is factually incorrect but delivered with high confidence, this is known as a hallucination. How does Anthropic's 'Honest' pillar address this issue during model training?

Accepted Answer

By training the model to express uncertainty and refuse to answer if unsure.. The 'Honest' pillar aims to make the model's confidence levels match its actual accuracy. During training, Claude is encouraged to admit when it is uncertain or doesn't have enough information to answer. This reduces the frequency of hallucinations and ensures the model is more transparent about its own limitations to the user.

Answer

By forcing the model to cite a source for every single word it generates.

Answer

By training the model to prioritize being polite over being factually correct.

Answer

By connecting the model to a real-time truth-checking database for every query.

Question 12

Which of the following best describes the practice of 'Red Teaming' in the context of Anthropic's model development?

Accepted Answer

Adversarial testing to identify and mitigate safety vulnerabilities and risks.. Red teaming is a proactive safety evaluation where internal or external experts intentionally try to provoke the model into generating harmful outputs. This helps identify vulnerabilities, biases, and potential failure modes that may not have been caught during standard training, allowing Anthropic to refine the model's safety guardrails before public release.

Answer

A process where developers label only 'correct' and 'safe' data for training.

Answer

A method for optimizing the model's latency by reducing safety checks.

Answer

Automating the generation of marketing materials using the Claude API.

Question 13

When fine-tuning a model for a specific industry, which TWO safety considerations are most important to maintain Claude's alignment?

Accepted Answer

Ensuring the fine-tuning dataset does not contain toxic or biased content.. Fine-tuning can inadvertently weaken a model's safety guardrails if not done carefully. It is crucial to ensure that the industry-specific data doesn't introduce new biases or teach the model to ignore its core safety principles. Maintaining alignment during fine-tuning requires balancing specialized knowledge with the original HHH safety framework.

Answer

Maximizing the model's ability to generate content as fast as possible.

Answer

Removing the Constitutional AI framework to allow for more flexibility.

Answer

Disabling all API-level filters to see the model's raw performance.

Question 14

A developer notices that Claude consistently provides more detailed career advice to male-sounding personas than to female-sounding personas in a simulation. This is an example of which safety concern?

Accepted Answer

Algorithmic Bias. Algorithmic bias occurs when a model reflects or amplifies societal prejudices found in its training data. Even with safety alignment, models can exhibit subtle biases in how they treat different demographic groups. Identifying and mitigating these biases is a key part of Anthropic's commitment to responsible and equitable AI development.

Answer

Data Hallucination

Answer

Over-refusal

Answer

Prompt Injection

Question 15

Claude refuses to write a fictional story about a bank robbery for a novelist, claiming it cannot assist with illegal acts. The novelist intends for the story to be part of a crime thriller. This is best described as:

Accepted Answer

An over-refusal due to conservative safety alignment.. This is a classic case of over-refusal, where the model's safety training is applied too broadly. While bank robbery is illegal, writing a fictional story about one is a standard creative task. Over-refusals occur when the model fails to distinguish between 'assisting in a crime' and 'writing about a crime' in a creative context.

Answer

A successful application of the Harmlessness principle.

Answer

A jailbreak attempt by the novelist.

Answer

A violation of the Anthropic Usage Policy by the user.