Log Analytics Workspace for Security

A Log Analytics workspace is a unique environment for log data in Azure Monitor. It serves as the central repository where diagnostic logs, activity logs, metrics, and other telemetry from Azure resources are collected, stored, and analyzed. Each workspace has its own data repository, configuration, and permissions. Data is stored in tables, each with a predefined schema, and can be queried using Kusto Query Language (KQL). The workspace is the foundation for Azure Monitor features like alerts, dashboards, and workbooks, and it is the primary data store for Microsoft Sentinel.

Security operations require centralized logging to detect threats, investigate incidents, and meet compliance requirements. Without a Log Analytics workspace, logs would be scattered across individual resources, making correlation nearly impossible. The workspace provides a single pane of glass for security monitoring, enabling:

Data flows into a Log Analytics workspace through several ingestion paths: 1. Azure Diagnostics – Azure resources (VMs, App Services, etc.) can send diagnostic data directly to the workspace using Azure Diagnostics extension or via Azure Monitor agent. 2. Azure Activity Log – Subscription-level events (e.g., resource creation, policy actions) are automatically sent to the workspace when configured. 3. Data Connectors – Microsoft Sentinel uses connectors to ingest logs from Microsoft 365, Azure AD, firewalls, and other sources. 4. HTTP Data Collector API – Custom applications can push data using a REST API with a workspace ID and key.

Once ingested, data is processed and stored in tables. Each table has a fixed schema defined by the data source. For example, the AzureActivity table contains columns like OperationName, Caller, Status, etc. The workspace uses a columnar storage format optimized for analytics. Data is automatically indexed based on the schema, and you can create custom fields using custom logs.

To create a Log Analytics workspace via Azure CLI:

az monitor log-analytics workspace create --resource-group MyResourceGroup --workspace-name MyWorkspace --location eastus --sku PerGB2018

To set retention to 365 days:

az monitor log-analytics workspace update --resource-group MyResourceGroup --workspace-name MyWorkspace --retention-time 365

To view workspace details:

az monitor log-analytics workspace show --resource-group MyResourceGroup --workspace-name MyWorkspace

To query data using KQL (example from Azure Portal or using REST API):

AzureActivity
| where TimeGenerated > ago(7d)
| where OperationName == "Create Virtual Machine"
| project TimeGenerated, Caller, ResourceGroup

For the exam, remember these numbers and the fact that you can increase retention up to 730 days, and beyond that use data export to Storage.

A large enterprise with 50+ Azure subscriptions needs to monitor security events across all subscriptions. They deploy a single Log Analytics workspace in a central subscription and configure diagnostic settings on all resources to send logs to this workspace. They use Azure Policy to enforce the deployment of the Azure Monitor Agent on all VMs and enable diagnostic settings on key services (e.g., Azure SQL, Key Vault). The workspace receives over 100 GB of data per day. To manage costs, they set a daily cap of 200 GB and configure data archiving for data older than 90 days to a storage account. They use cross-workspace queries (using workspace() function) to correlate events from other workspaces used by isolated teams. The security team uses KQL to detect brute-force attacks across all subscriptions by querying the SigninLogs table from Azure AD connector. One common issue is that the daily cap is reached too early, causing critical logs to be dropped. They mitigate this by setting the cap to a higher value and monitoring the _LogOperation table for ingestion failures. They also use table-level RBAC to restrict access to SecurityEvent and SigninLogs tables to the SOC team only.

A healthcare organization must retain diagnostic logs for 6 years to comply with HIPAA. They use a Log Analytics workspace with 730-day retention and configure data export to an Azure Storage account for long-term archival. They set up a daily export rule that moves logs older than 2 years to a cold storage tier (Azure Blob Storage - Cool or Archive). They also use customer-managed keys (CMK) for encryption to meet compliance requirements. The workspace is configured with private endpoints to ensure logs never traverse the internet. They encounter a problem where the export rule fails because the KQL query is too complex—they simplify it to filter on TimeGenerated only. They also set up alerts for any failed export jobs. The security team runs quarterly queries on archived data by restoring it to a staging workspace (paying retrieval fees). They use Azure Policy to audit that all resources have diagnostic settings enabled and that the workspace is using CMK.

A global company with operations in multiple regions decides to use a separate Log Analytics workspace per region to keep data local for compliance (e.g., GDPR). They use Azure Sentinel in each region but also need a central view for global threat hunting. They use cross-workspace queries in Azure Sentinel workbooks to combine data from all workspaces. They also configure a central workspace for Azure Activity Logs from all subscriptions. The challenge is managing multiple workspaces: they use Azure Lighthouse to delegate permissions across tenants. They also set up a log analytics workspace for each business unit with its own retention and RBAC. They find that querying across workspaces is slow, so they use materialized views (via Azure Data Explorer) for common aggregations. They also use Azure Monitor Workbooks to create a single dashboard that queries all workspaces using the workspace() function. The key lesson is to plan the workspace architecture carefully to balance performance, cost, and compliance.