Sentinel Watchlists

A watchlist in Microsoft Sentinel is a collection of data items (such as IP addresses, file hashes, or user accounts) that you can use to enrich logs, filter events, or correlate with threat intelligence. Watchlists are stored as a dedicated table in your Log Analytics workspace (Watchlist), but they are optimized for fast lookups. They are essentially key-value pairs with a schema you define. You can create up to 30 watchlists per workspace (soft limit), each with a maximum of 10 MB of data (or 10,000 rows, whichever is smaller).

Security analysts often need to reference static or semi-static data sets: lists of known malicious IPs, sensitive admin accounts, asset inventories, or employee IDs. Without watchlists, you would have to either hardcode values in queries (which is inflexible) or join against external tables (which is slow and costly). Watchlists provide a lightweight, cached, and easy-to-manage solution. They are designed for scenarios where the data changes infrequently but needs to be referenced frequently in queries.

When you create a watchlist, Sentinel stores the data in a hidden table in the Log Analytics workspace. The table name follows the pattern Watchlist_<GUID> but you access it via an alias you define (e.g., MyIPList). The data is stored as rows with columns: SearchKey (the key you search on), SearchValue (the value), and additional custom columns. The SearchKey is the column you will use for lookup operations. Internally, Sentinel uses Azure Data Explorer's storage engine to host this data. When a query uses the _GetWatchlist('alias') function, the entire watchlist is loaded into memory for the duration of the query. This is why watchlists are limited in size—they are not meant for large datasets (use Threat Intelligence for that).

To create a watchlist via the portal: 1. Navigate to Microsoft Sentinel > Your workspace > Configuration > Watchlist. 2. Click "Add new". 3. Provide a name, alias, and description. 4. Upload a CSV file (first row must be column headers). 5. Set the SearchKey column (the column used for lookups). 6. Optionally set a validity period (start/end date) for the watchlist items. 7. Click "Add".

To create via PowerShell:

$watchlistProps = @{
    Name = 'MyIPList'
    Alias = 'MyIPList'
    Description = 'List of known bad IPs'
    DisplayName = 'Bad IPs'
    Provider = 'Custom'
    Source = 'Local file'
    RawContent = (Get-Content -Path 'C:\badIPs.csv' -Raw)
    ItemsSearchKey = 'IPAddress'
}
Set-AzSentinelWatchlist -ResourceGroupName 'rg-sentinel' -WorkspaceName 'sentinel-workspace' -Watchlist $watchlistProps

To verify:

_GetWatchlist('MyIPList')
| take 10

Watchlists are loaded entirely into memory for each query that uses _GetWatchlist. Therefore, large watchlists (near the 10 MB limit) can cause query performance degradation. For large datasets, consider using Threat Intelligence or custom Log Analytics tables with appropriate indexing. Also, watchlists are not meant for real-time updates; they are eventually consistent within a few seconds after creation/update.

A large enterprise receives thousands of sign-in attempts daily. The SOC maintains a list of known malicious IPs from threat feeds and internal analysis. They create a Sentinel watchlist called 'MaliciousIPs' with columns: IPAddress, Source, and DateAdded. Every 5 minutes, a scheduled analytics rule queries SigninLogs and joins against this watchlist. When a match occurs, an incident is created with high severity. The watchlist is updated weekly by the threat intelligence team via a PowerShell script that downloads the latest CSV from a secure storage account. The watchlist currently has 5,000 entries, well within the 10,000 limit. Performance is acceptable; the query completes in under 30 seconds. One challenge: if the watchlist grows to 9,000 rows, the query may take longer. They plan to split into two watchlists or migrate to Threat Intelligence if needed.

A company wants to enrich all security events with asset owner information. They have a CSV file mapping hostnames to owners and departments. They create a watchlist 'AssetOwners' with SearchKey=Hostname and columns: Owner, Department. In hunting queries, they use: SecurityEvent | lookup _GetWatchlist('AssetOwners') on $left.Computer == $right.SearchKey. This adds owner info to every event, making it easier to contact the right person during an incident. The watchlist is updated quarterly. Because the watchlist has only 2,000 rows, performance is excellent. They also use it in a fusion rule to correlate events from the same owner's machines.

A SOC uses a watchlist to suppress alerts for known good IPs (e.g., internal scanners, legitimate partners). They create a watchlist 'SafeIPs' and modify analytics rules to exclude those IPs: SigninLogs | where IPAddress !in (_GetWatchlist('SafeIPs') | project SearchKey). This reduces noise significantly. However, they must ensure the watchlist is kept current; stale entries can cause missed detections. They set a validity period so that entries older than 90 days are automatically excluded. Misconfiguration: initially they used !in with a subquery that returned multiple columns, causing errors. They fixed by projecting only SearchKey.

In production, watchlists are cached per query. If multiple queries run concurrently, each loads the watchlist separately. For high-frequency queries (e.g., every 1 minute), consider using a smaller watchlist or caching the results in a Log Analytics table. Also, watchlist data counts toward your workspace ingestion and retention costs. A 10 MB watchlist stored for 90 days incurs storage costs. Large watchlists may cause query throttling if the query consumes too much data. Use _GetWatchlist only when necessary; for very large datasets, prefer Threat Intelligence or custom tables with indexing.