This chapter covers cost management and data retention in Microsoft Sentinel, a critical area for the SC-200 exam. Understanding how to control costs through retention policies, data ingestion configurations, and pricing tiers is essential for any security operations analyst. Approximately 10-15% of exam questions touch on cost management, retention, or data lifecycle concepts, making this a high-yield topic that can differentiate candidates who master the details.
Jump to a section
Managing Microsoft Sentinel costs is like renting a storage unit with two pricing models: pay-as-you-go and reserved capacity. In pay-as-you-go, you pay per box (gigabyte) you store each month, with a higher per-box rate. In reserved capacity, you commit to storing a certain number of boxes for a year, getting a lower per-box rate but paying even if you don't fill them. Additionally, you can choose to store some boxes (interactive retention) in a quick-access closet for fast retrieval, while older boxes (long-term retention) go to a cheaper, slower-access warehouse. You can also set rules to automatically move boxes to the warehouse after a certain time. If you store more than your reserved capacity, you pay a penalty (overage at pay-as-you-go rate). You can also filter out junk mail (data transformation) before it even arrives, reducing the number of boxes you need to store. By understanding your storage patterns, you can choose the right pricing model and retention tiers to minimize costs.
What is Sentinel Cost Management and Data Retention?
Microsoft Sentinel is a cloud-native SIEM that charges based on the volume of data ingested into the Log Analytics workspace. Cost management involves optimizing this ingestion and retention to balance security visibility with budget. Data retention defines how long data is stored in the interactive (hot) tier versus the long-term (cold) tier, and when it is deleted.
Why It Exists
Without cost management, Sentinel can become unexpectedly expensive. Organizations often ingest massive amounts of log data, and without careful planning, costs can spiral. Retention policies help comply with regulatory requirements (e.g., 90 days for security logs) while minimizing storage costs for older data that is rarely queried.
How It Works Internally
Sentinel uses Azure Log Analytics workspaces as its underlying storage. Each workspace has a pricing tier (Pay-as-you-go or Capacity Reservations) and a retention setting (interactive retention and long-term retention). Data flows into the workspace through data connectors, and each table (e.g., SecurityEvent, SigninLogs) can have its own retention and archiving policies.
- Ingestion: Data is ingested into tables. Each table has a cost per GB ingested. You can use Data Collection Rules (DCRs) and transformations to filter or modify data before ingestion, reducing volume. - Interactive Retention: Default is 30 days for most tables, but can be extended up to 90 days (or 2 years for some tables like SecurityEvent) at higher cost. Data in interactive tier is available for fast queries and analytics. - Long-Term Retention (Archiving): After interactive retention expires, data can be moved to long-term retention (archive) for up to 7 years. Archive storage costs are lower, but querying archived data requires a search job, which takes time and incurs additional cost per GB scanned. - Total Retention: The sum of interactive + archive retention cannot exceed 7 years (2556 days). You can set different policies per table. - Pricing Tiers: - Pay-as-you-go: $2.46 per GB ingested (varies by region). No commitment. - Capacity Reservations: Commit to a minimum of 50 GB/day, get a discount (e.g., $2.00 per GB for 50 GB, lower tiers for higher commitments). Overage above commitment is charged at pay-as-you-go rate. - Cost Analysis: Use Azure Cost Management + Billing to view costs by resource, tag, or meter. Sentinel-specific costs appear under the 'Microsoft Sentinel' meter.
Key Components, Values, Defaults, and Timers
Default interactive retention: 30 days (for most tables). For SecurityEvent, default is 90 days.
Maximum total retention: 7 years (2556 days).
Minimum archive retention: 30 days after interactive retention ends.
Capacity Reservation minimum: 50 GB/day.
Capacity Reservation discount tiers: 50 GB, 100 GB, 200 GB, 300 GB, 400 GB, 500 GB, 600 GB, 700 GB, 800 GB, 900 GB, 1000 GB, 2000 GB, 5000 GB per day. Discounts increase with commitment size.
Overage rate: Pay-as-you-go rate (no discount).
Data transformation: Can be applied at ingestion via DCRs to filter out specific events (e.g., benign events from security logs).
Basic Logs: A cheaper ingestion tier for verbose logs (e.g., AppTraces) that have limited analytics value. Basic Logs cost less to ingest but have reduced query capabilities and limited retention (30 days).
Auxiliary Logs: Even cheaper tier for high-volume, low-value logs. 30-day retention, no interactive analytics.
Configuration and Verification Commands
To set retention policy on a table:
# Using PowerShell
$table = Get-AzOperationalInsightsTable -ResourceGroupName "myRG" -WorkspaceName "myWS" -TableName "SecurityEvent"
$table.RetentionInDays = 90
$table.TotalRetentionInDays = 365
Set-AzOperationalInsightsTable -InputObject $tableTo view current pricing tier:
az monitor log-analytics workspace show --resource-group myRG --workspace-name myWS --query "sku"To check data ingestion volume:
Usage
| where TimeGenerated > ago(30d)
| where IsBillable == true
| summarize TotalVolumeGB = sum(Quantity) / 1000.How It Interacts with Related Technologies
Azure Policy: Can enforce retention policies across workspaces.
Azure Cost Management: Provides detailed cost breakdowns and budgets.
Data Collection Rules (DCRs): Control which data is ingested and can apply transformations to reduce volume.
Workbooks: Can visualize cost trends and ingestion patterns.
Sentinel Optimization Workbooks: Built-in workbook to analyze cost drivers and suggest savings.
Exam Focus
The SC-200 exam tests your ability to:
Understand the difference between interactive and long-term retention.
Know default retention values (30 days, 90 days for SecurityEvent).
Identify scenarios where Capacity Reservations save money.
Recognize that data transformation reduces ingestion costs.
Understand that Basic Logs and Auxiliary Logs are cheaper tiers for specific use cases.
Know that total retention cannot exceed 7 years.
Trap Patterns
Wrong answer: "Setting retention to 0 days disables retention." Reality: Minimum retention is 30 days for interactive, and you cannot set 0.
Wrong answer: "Capacity Reservations apply to a single table." Reality: Reservations apply to the entire workspace ingestion.
Wrong answer: "Archived data is free to query." Reality: Queries against archived data incur costs per GB scanned.
Wrong answer: "You can set retention per workspace only." Reality: Retention can be set per table.
Edge Cases
If you delete a table, data is permanently lost.
Changing retention to a lower value does not immediately delete data; it marks it for deletion.
Data in archive cannot be directly queried using standard KQL; you must use a search job or restore to interactive tier.
Summary
Cost management in Sentinel is about balancing ingestion volume, retention duration, and pricing tier. Use data transformations to reduce ingestion, set appropriate retention per table, and consider Capacity Reservations if your daily ingestion is stable and above 50 GB.
Assess current ingestion volume
Start by analyzing your current data ingestion volume using the Usage table in Log Analytics. Run a query to see daily ingestion by table: Usage | where TimeGenerated > ago(30d) | where IsBillable == true | summarize by DataType, sum(Quantity)/1000. This tells you which tables consume the most space. Also check the cost in Azure Cost Management to understand your baseline spending.
Choose appropriate pricing tier
Based on daily ingestion volume, decide between Pay-as-you-go and Capacity Reservations. If your ingestion is consistently above 50 GB/day, a Capacity Reservation will save money. Use the Azure Pricing Calculator to compare costs. For example, at 100 GB/day, pay-as-you-go costs $246/day, while a 100 GB reservation costs $200/day (assuming $2.00/GB). The reservation requires a 31-day commitment, but you can change it once per month.
Set interactive retention duration
Determine how long you need fast access to data. Typical compliance requirements are 90 days for security logs. Set interactive retention per table using Azure portal, PowerShell, or ARM templates. For tables like SecurityEvent, the default is 90 days; for others, 30 days. Extending beyond defaults increases cost. Consider using a shorter retention for tables with low query frequency.
Configure long-term retention (archive)
For data that must be kept longer than interactive retention, enable archiving. Set TotalRetentionInDays to the sum of interactive + archive. For example, interactive=90 days, archive=275 days, total=365 days. Archive storage is cheaper, but querying requires a search job. Ensure you don't exceed 7 years total. Use per-table policies to optimize costs.
Implement data transformations
Use Data Collection Rules (DCRs) to filter or modify incoming data. For example, you can drop benign events like Windows Event ID 4688 (process creation) for known good processes. This reduces ingestion volume and cost. Write KQL transformations in the DCR. Test in a staging workspace first. Remember: transformations are applied before ingestion, so they reduce billable data.
Monitor and adjust regularly
Use the Sentinel Optimization workbook to track cost trends and identify anomalies. Set up budget alerts in Azure Cost Management to notify you when spending exceeds a threshold. Review retention policies quarterly to ensure they still meet compliance needs without overspending. Adjust Capacity Reservation levels if ingestion changes significantly.
Scenario 1: Large Enterprise with 500 GB/day Ingestion
A multinational company ingests logs from all on-premises servers, cloud resources, and SaaS applications. Their daily ingestion is around 500 GB. Initially on Pay-as-you-go, they were spending $1,230/day. After moving to a 500 GB Capacity Reservation, the cost dropped to $800/day (assuming $1.60/GB at that tier), saving $430/day. They also set interactive retention to 90 days for security tables and archive for up to 1 year. They use DCRs to filter out verbose debug logs from application servers, reducing ingestion by 10%.
Scenario 2: Small Business with 20 GB/day Ingestion
A small company with 500 employees has a daily ingestion of 20 GB. Pay-as-you-go costs about $49.20/day. Capacity Reservations start at 50 GB, which would cost $100/day (at $2.00/GB) – more expensive. So they stay on Pay-as-you-go. They set interactive retention to 30 days and archive to 90 days for compliance. They use Basic Logs for non-critical application logs to save costs.
Scenario 3: Compliance-Driven Financial Institution
A bank must retain security logs for 7 years due to regulatory requirements. They have 200 GB/day ingestion. They use interactive retention of 90 days for fast queries, then archive for 6.9 years (total 7 years). They have a 200 GB Capacity Reservation to save on ingestion. They also use data transformations to exclude irrelevant events like printer logs, reducing volume by 5%. They set up Azure Policy to enforce retention policies across multiple workspaces.
Common Misconfigurations
Setting retention too high: Keeping 2 years of interactive data for low-value logs increases costs unnecessarily.
Not using transformations: Missing opportunities to filter out noise.
Ignoring Capacity Reservations: Staying on Pay-as-you-go when ingestion is stable and above 50 GB.
Not monitoring: Cost surprises when ingestion spikes due to new data sources or misconfigured connectors.
What SC-200 Tests on This Topic (Objective 2.1)
The exam expects you to:
Understand the difference between interactive and long-term retention.
Know default retention values: 30 days (most tables), 90 days (SecurityEvent).
Identify scenarios where Capacity Reservations are cost-effective.
Recognize that data transformations reduce ingestion cost.
Know that Basic Logs and Auxiliary Logs are cheaper tiers.
Understand that total retention cannot exceed 7 years.
Be able to recommend a retention policy based on compliance requirements.
Common Wrong Answers and Why Candidates Choose Them
"Setting retention to 0 days disables retention." – Candidates assume 0 means no retention, but the minimum is 30 days. The exam may ask about invalid retention settings.
"Capacity Reservations apply per table." – They apply to the entire workspace. Candidates confuse with per-table retention.
"Archived data is free to query." – Queries on archived data incur costs. Candidates think archive means cheap storage and free access.
"You can set retention only at workspace level." – Retention is per table. Candidates may not know the granularity.
Specific Numbers and Terms That Appear on the Exam
30 days: Default interactive retention for most tables.
90 days: Default interactive retention for SecurityEvent.
7 years (2556 days): Maximum total retention.
50 GB/day: Minimum Capacity Reservation.
Pay-as-you-go rate: ~$2.46 per GB (varies).
Basic Logs: Lower cost, limited query, 30-day retention.
Auxiliary Logs: Even lower cost, 30-day retention, no analytics.
Edge Cases and Exceptions
If you delete a table, all data is lost.
Changing retention to a lower value does not immediately delete data; it's marked for deletion.
Data in archive cannot be queried with standard KQL; requires search job or restore.
Capacity Reservations can be changed once per month.
Some tables (e.g., SecurityEvent) have higher default retention; the exam may test this.
How to Eliminate Wrong Answers
If an answer says "0 days retention" or "unlimited retention," it's wrong.
If an answer suggests querying archived data without extra cost, it's wrong.
If an answer claims Capacity Reservations are per table, it's wrong.
If an answer says you cannot set per-table retention, it's wrong.
Use the underlying mechanism: data flows through DCRs (transformations) -> ingestion -> retention (interactive -> archive). Each step has cost implications.
Default interactive retention is 30 days for most tables, 90 days for SecurityEvent.
Maximum total retention (interactive + archive) is 7 years (2556 days).
Capacity Reservations require a minimum commitment of 50 GB/day and provide discounted rates.
Data transformations applied via DCRs reduce ingestion volume and cost.
Basic Logs and Auxiliary Logs are cheaper ingestion tiers for low-value logs.
Archived data queries incur cost per GB scanned.
Retention policies can be set per table, not just per workspace.
Pay-as-you-go is best for ingestion below 50 GB/day or variable patterns.
Capacity Reservations can be changed once per month.
Use the Sentinel Optimization workbook to monitor cost drivers.
These come up on the exam all the time. Here's how to tell them apart.
Pay-as-you-go pricing
No commitment required
Higher per-GB cost (e.g., $2.46/GB)
Ideal for variable or low ingestion (<50 GB/day)
Can be used for any workspace size
No penalty for low usage
Capacity Reservations
Requires 31-day commitment
Lower per-GB cost (discount tiers from 50 GB/day)
Best for stable, high ingestion (≥50 GB/day)
Minimum 50 GB/day commitment
Overage charged at pay-as-you-go rate
Interactive retention
Fast queries with standard KQL
Higher storage cost per GB/month
Default 30 days (90 for SecurityEvent)
Maximum 90 days (or 2 years for some tables)
Data available for real-time analytics
Long-term retention (archive)
Requires search job or restore to query
Lower storage cost per GB/month
Can be set up to 7 years total retention
Minimum 30 days after interactive retention ends
Data not available for standard queries
Mistake
Setting interactive retention to 0 days disables data storage.
Correct
The minimum interactive retention is 30 days. You cannot set it to 0. The correct way to reduce cost is to use shorter retention (e.g., 30 days) and archive.
Mistake
Capacity Reservations apply to individual tables.
Correct
Capacity Reservations apply to the entire workspace's daily ingestion. You cannot reserve capacity for a specific table. All ingestion counts toward the commitment.
Mistake
Archived data can be queried instantly at no cost.
Correct
Archived data requires a search job that scans the archive, incurring cost per GB scanned. It is not free and takes time to complete.
Mistake
Data transformations increase cost because they require processing.
Correct
Data transformations reduce cost by filtering out unwanted data before ingestion. The processing cost is negligible compared to the savings from reduced ingestion volume.
Mistake
You can only set retention at the workspace level.
Correct
Retention can be set per table, allowing granular control. For example, you can keep SecurityEvent for 90 days and AppTraces for 30 days.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
The default interactive retention for the SecurityEvent table is 90 days. This is higher than the default 30 days for most other tables because security events are often needed for longer compliance periods. You can extend it up to 2 years if needed, but that increases cost.
Use data transformations to filter out benign events, set appropriate retention policies (shorter interactive, longer archive), consider Capacity Reservations if ingestion is stable and above 50 GB/day, and use Basic Logs for verbose logs that don't need full analytics. Also, regularly review the Sentinel Optimization workbook.
Yes, but not with standard KQL queries. You must use a search job that scans the archive, which incurs cost per GB scanned. Alternatively, you can restore data to the interactive tier for a limited time. Archived data is not available for real-time analytics or alerts.
Basic Logs are a cheaper ingestion tier for verbose logs that have limited analytics value. They cost less to ingest but have reduced query capabilities and a fixed 30-day retention. Auxiliary Logs are even cheaper, also with 30-day retention, but they cannot be queried interactively at all; they are only for search jobs.
In the Azure portal, navigate to your Log Analytics workspace, select 'Usage and estimated costs', then 'Pricing tier'. Choose 'Capacity Reservations' and select the commitment tier (e.g., 100 GB/day). You can also use PowerShell or CLI. The commitment is for 31 days, and you can change it once per month.
Any ingestion above the committed amount is charged at the pay-as-you-go rate. For example, if you have a 100 GB/day reservation and ingest 120 GB, the first 100 GB is at the reservation rate, and the extra 20 GB is at the pay-as-you-go rate. There is no penalty other than the higher per-GB cost.
Yes, you can delete data by reducing retention or using the purge API (with appropriate permissions). However, Azure Sentinel does not support immediate deletion; data is marked for deletion and removed within 30 days. Also, deleting data may violate compliance policies.
You've just covered Sentinel Cost Management and Data Retention — now see how well it sticks with free SC-200 practice questions. Full explanations included, no account needed.
Done with this chapter?