This chapter covers Azure AI Content Safety, a critical service for ensuring responsible AI deployment by detecting harmful user-generated and AI-generated content. For the AI-900 exam, this topic appears in the Generative AI domain (objective 5.4) and typically accounts for 5-10% of exam questions. You will learn how Content Safety works, its key features, configuration options, and how it integrates with other Azure AI services. Mastery of this topic is essential for understanding Microsoft's approach to responsible AI and content moderation.
Jump to a section
Imagine a large corporate mailroom that receives thousands of letters daily. The mailroom has a sorter who checks each envelope for dangerous materials (like powders or suspicious stains) and inappropriate content (like hate mail or explicit images). The sorter uses a set of rules: if a letter contains certain keywords or images, it is flagged and sent to a manager for review; otherwise, it is delivered. The sorter can also be configured to block certain senders or topics entirely. In this analogy, the mailroom is Azure AI Content Safety, the letters are user-generated content, the rules are the severity levels and filters, and the manager is a human reviewer. The sorter does not read every letter deeply but uses quick checks (like scanning for keywords or image hashes) to decide. Similarly, Azure AI Content Safety uses pre-trained models to detect harmful text and images, assigning severity scores instead of binary pass/fail. Just as the mailroom can be tuned to be more or less strict, Content Safety allows configuration of severity thresholds for each category (hate, sexual, violence, self-harm). The key is that the sorter never opens a letter unless it is flagged — analogous to how Content Safety can be used to filter content before it reaches users or to flag it for moderation.
What is Azure AI Content Safety?
Azure AI Content Safety is a cloud service that detects harmful content in text and images. It is part of Azure AI Services and is designed to help organizations comply with content moderation policies and legal requirements. The service uses pre-trained machine learning models to analyze content across four severity-graded categories: hate, sexual, violence, and self-harm. Each category has four severity levels: 0 (safe), 2 (low), 4 (medium), and 6 (high). The service returns a severity score for each detected category, allowing fine-grained filtering.
Why Content Safety Exists
With the rise of generative AI and user-generated content platforms, the risk of harmful content being published has increased. Manual moderation does not scale. Azure AI Content Safety provides automated, scalable moderation that can be integrated into applications, chatbots, and content pipelines. It supports responsible AI principles by helping to prevent the spread of hate speech, harassment, and graphic violence.
How It Works Internally
The service processes content in a pipeline: 1. Input: Text (up to 10,000 characters per API call) or image (up to 4 MB, supported formats: JPEG, PNG, GIF, BMP, WEBP). 2. Pre-processing: For images, the service extracts frames from GIFs (up to 20 frames). For text, it tokenizes and normalizes input. 3. Model Inference: Pre-trained transformer-based models analyze the content. For text, models are trained on large datasets of harmful and safe content. For images, models analyze visual features using computer vision. 4. Severity Scoring: Each category gets a score from 0 to 7 (but only 0, 2, 4, 6 are used). The score indicates the likelihood and severity of harm. 5. Output: JSON response with an array of category severity scores. Optionally, the service can return a binary 'isHarmful' flag based on a configurable threshold.
Key Components, Values, and Defaults
Categories: Hate, Sexual, Violence, Self-Harm. Each category has specific subcategories (e.g., Hate includes race, religion, gender).
Severity Levels: 0 (safe), 2 (low), 4 (medium), 6 (high). Default threshold for flagging is 4 (medium).
Text Length: Maximum 10,000 characters per API call. For longer texts, split into chunks.
Image Size: Maximum 4 MB, dimensions up to 2048x2048 pixels.
Rate Limits: Standard tier: 1,000 requests per minute (RPM), 10,000 requests per month free. S0 tier: 10,000 RPM.
Latency: Typically < 1 second for text, < 2 seconds for images.
Pricing: Pay per API call. Free tier: 10,000 text records and 10,000 image records per month. Standard tier: $0.50 per 1,000 text records, $1.00 per 1,000 image records.
Configuration and Verification Commands
You can configure Content Safety via Azure portal or REST API. Key parameters: - Blocklist: Manage custom blocklists of terms to block (e.g., specific profanity or brand names). Blocklists can have up to 100,000 terms per list. - Threshold: Set per-category threshold (0,2,4,6) for flagging content. - Output Type: 'FourSeverityLevels' (default) or 'EightSeverityLevels' (finer granularity).
Example API call for text moderation:
POST https://<your-resource-name>.cognitiveservices.azure.com/contentSafety/text:analyze?api-version=2023-10-01
Headers: Ocp-Apim-Subscription-Key: <your-key>
Body:
{
"text": "I hate you!"
}Response:
{
"hateResult": {
"severity": 4,
"isHarmful": true
},
"sexualResult": {
"severity": 0,
"isHarmful": false
},
"violenceResult": {
"severity": 0,
"isHarmful": false
},
"selfHarmResult": {
"severity": 0,
"isHarmful": false
}
}Interaction with Related Technologies
Azure OpenAI Service: Content Safety can be used as a filter for prompts and completions to prevent generating harmful content. Azure OpenAI has built-in content filtering that uses Content Safety models.
Azure Cognitive Search: Can be used to moderate indexed content.
Azure Logic Apps / Power Automate: Automate workflows when harmful content is detected (e.g., send alert, block user).
Azure Monitor: Log moderation results for auditing.
Important Exam Considerations
The AI-900 exam focuses on:
Understanding the four categories (hate, sexual, violence, self-harm).
The four severity levels (0,2,4,6) and default threshold (4).
Use cases: moderating user-generated content, filtering AI-generated content, compliance.
Integration with Azure OpenAI Service for responsible AI.
The fact that it is a pre-trained API, not a custom model (though you can add blocklists).
Pricing model: pay per API call, free tier available.
Common Pitfalls
Confusing Content Safety with Azure AI Content Moderator (older service). Content Safety is the newer, recommended service.
Thinking it only works with text — it also supports images.
Assuming it returns binary pass/fail — it returns severity scores.
Overlooking that it can be integrated with Azure OpenAI to filter both input and output.
Create a Content Safety Resource
In the Azure portal, search for 'Content Safety' and create a resource. Choose a subscription, resource group, region (e.g., East US, West Europe), and pricing tier (Free F0 or Standard S0). The Free tier offers 10,000 text and 10,000 image transactions per month. After deployment, note the endpoint and key. This is the first step to enable API access.
Configure Blocklists and Thresholds
In the Content Safety resource, navigate to 'Blocklists' to add custom terms. You can create multiple blocklists (e.g., 'Profanity', 'BrandNames'). Each blocklist can have up to 100,000 terms. Also set the severity threshold per category (default 4). This configuration determines what content is flagged as harmful. Blocklists are checked before model inference.
Send Text for Moderation
Use the REST API or SDK to send text content. The API expects a JSON body with the 'text' field. The service analyzes the text and returns severity scores for each category. The response includes 'isHarmful' boolean based on your threshold. For example, if threshold is 4, any severity >=4 is harmful. The API can handle up to 10,000 characters per call.
Send Image for Moderation
For images, send a base64-encoded image or a URL. The service supports JPEG, PNG, GIF, BMP, WEBP up to 4 MB. For GIFs, it analyzes up to 20 frames. The response includes severity scores for each category. Image moderation is useful for detecting violent or sexual imagery. Combine with text moderation for comprehensive filtering.
Integrate with Azure OpenAI Service
Azure OpenAI has built-in content filtering that uses Azure AI Content Safety models. You can configure content filters for prompts and completions in the Azure OpenAI Studio. This ensures that harmful content is not generated or displayed. The filtering happens automatically, but you can adjust severity thresholds per category. This integration is a key exam topic.
Enterprise Scenario 1: Social Media Platform Moderation
A social media company uses Azure AI Content Safety to automatically moderate user posts and comments. They integrate the API into their content pipeline: every post is sent for text and image analysis before being published. They set the threshold to 4 (medium) for all categories to catch hate speech and graphic violence. They also maintain a custom blocklist of banned terms and usernames. The service scales to handle millions of requests daily with low latency (<1 second per text). Misconfiguration (e.g., threshold too low) leads to over-blocking legitimate content, causing user complaints. Proper tuning involves A/B testing thresholds.
Enterprise Scenario 2: AI Chatbot Filtering
A company deploys a customer service chatbot using Azure OpenAI. They use Content Safety to filter both user inputs and AI responses. User inputs are checked for profanity or harassment before being sent to the model. AI responses are filtered to ensure they do not contain harmful or inappropriate content. They configure separate thresholds: stricter for user inputs (threshold 2) and more lenient for AI responses (threshold 4) to avoid overly censoring the bot. This dual-filter approach reduces risk but increases latency slightly (additional 200ms per filter).
Enterprise Scenario 3: E-commerce Product Reviews
An e-commerce platform moderates product reviews and images. They use Content Safety to automatically reject reviews containing hate speech or sexual content. They also analyze images for violence (e.g., weaponry). The free tier is insufficient for their volume, so they upgrade to S0 tier at $0.50 per 1,000 text records. They use Azure Monitor to track flagged content and generate reports for compliance. A common issue is false positives for certain cultural expressions; they mitigate by adding a blocklist of allowed terms and using human review for borderline cases.
AI-900 Objective 5.4: Describe Azure AI Content Safety
The exam tests your understanding of what Content Safety is, its categories, severity levels, and integration with Azure OpenAI. Specific focus areas: - Categories: Hate, Sexual, Violence, Self-Harm. Memorize these four. - Severity Levels: 0 (safe), 2 (low), 4 (medium), 6 (high). Default threshold is 4. - Text and Image support: Know that it supports both, with limits (10K chars, 4MB images). - Integration with Azure OpenAI: Built-in content filters use Content Safety models. - Pricing: Free tier available (10K text + 10K image per month).
Common Wrong Answers
Choosing 'Azure Content Moderator' instead of Content Safety: The older service is deprecated; exam focuses on Content Safety.
Thinking severity is 1-5: The actual levels are 0,2,4,6.
Assuming it only filters text: Images are also supported.
Believing you can train custom models: Content Safety is pre-trained; you can only add blocklists.
Confusing with Azure AI Vision: Vision detects objects, not harmful content categories.
Exam Tips
If a question mentions 'responsible AI' or 'moderation', think Content Safety.
For integration with OpenAI, remember that filters can be applied to both prompts and completions.
The free tier is often tested: 'Which Azure service offers a free tier for content moderation?' Answer: Content Safety.
Eliminate answers that mention 'custom training' or 'machine learning model creation' — Content Safety is API-based, no training.
Edge Cases
GIFs: Only first 20 frames analyzed.
Multiple languages: Supports over 100 languages, but accuracy is best for English.
Blocklist priority: Blocklist terms are checked before model inference; if a term matches, it is flagged regardless of context.
Azure AI Content Safety detects harmful content across four categories: hate, sexual, violence, self-harm.
Severity levels are 0 (safe), 2 (low), 4 (medium), 6 (high); default threshold for flagging is 4.
Supports text (up to 10,000 characters) and images (up to 4 MB, JPEG, PNG, GIF, BMP, WEBP).
Integrates with Azure OpenAI Service to filter both prompts and completions.
Free tier offers 10,000 text and 10,000 image transactions per month.
Custom blocklists allow up to 100,000 terms per list to block specific content.
The service is pre-trained; no custom model training is possible.
For GIFs, only the first 20 frames are analyzed.
These come up on the exam all the time. Here's how to tell them apart.
Azure AI Content Safety
Newer service, recommended by Microsoft
Supports text and images with severity scoring (0,2,4,6)
Integrated with Azure OpenAI for content filtering
Custom blocklists (up to 100,000 terms)
Free tier: 10K text + 10K images per month
Azure Content Moderator (Deprecated)
Older service, being phased out
Text moderation with binary pass/fail or category flags
No direct integration with Azure OpenAI
Custom terms via moderation lists (smaller limits)
Different pricing model, no free tier
Mistake
Azure AI Content Safety can be used to train custom moderation models.
Correct
Content Safety is a pre-trained API; you cannot train or fine-tune models. You can only use blocklists and adjust severity thresholds.
Mistake
Content Safety only works with text content.
Correct
It supports both text (up to 10,000 characters) and images (up to 4 MB, various formats including GIFs).
Mistake
The severity levels are 1, 2, 3, 4.
Correct
Severity levels are 0 (safe), 2 (low), 4 (medium), 6 (high). There is no level 1, 3, 5, or 7.
Mistake
Content Safety is the same as Azure Content Moderator.
Correct
Content Safety is the newer, recommended service. Azure Content Moderator is deprecated. The exam focuses on Content Safety.
Mistake
You can set different thresholds for different categories.
Correct
Yes, you can configure per-category thresholds (e.g., hate threshold 2, violence threshold 4). The default is 4 for all.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
The four categories are Hate, Sexual, Violence, and Self-Harm. These are the core categories the service analyzes. Each category returns a severity score from 0 to 6 (only 0,2,4,6 used). On the exam, memorize these four exactly.
Yes, it supports image moderation for formats JPEG, PNG, GIF, BMP, and WEBP up to 4 MB. For GIFs, it analyzes up to 20 frames. This is a key differentiator from older services. Exam questions may test that it supports both text and images.
Azure OpenAI has built-in content filtering that uses Azure AI Content Safety models. You can configure filters for both user inputs (prompts) and AI outputs (completions) in Azure OpenAI Studio. This helps prevent generation of harmful content. The integration is automatic but configurable.
The default threshold is 4 (medium). Any content with a severity score of 4 or higher is considered harmful. You can change this per category. The exam may ask: 'What is the default threshold?' Answer: 4.
Yes, the Free F0 tier provides 10,000 text records and 10,000 image records per month at no cost. This is often tested as a cost-effective option for small-scale moderation.
No, the service is pre-trained and does not support custom model training. However, you can add custom blocklists of terms to block. This is a common exam trap: candidates think they can train models, but Content Safety is API-only.
The API returns an error. You must split the text into chunks of 10,000 characters or less and send multiple requests. This limit is important for exam scenarios involving long documents.
You've just covered Azure AI Content Safety — now see how well it sticks with free AI-900 practice questions. Full explanations included, no account needed.
Done with this chapter?