Sentinel Notebooks with Jupyter

Microsoft Sentinel notebooks are Jupyter notebooks that run in the Azure Machine Learning compute environment, integrated directly into the Sentinel portal. They provide an interactive, code-driven approach to security analysis, complementing the built-in hunting queries and rules. The primary purpose is to enable advanced analysts to perform complex investigations that are not possible with KQL alone—such as machine learning model inference, custom data enrichment, and multi-step correlation across disparate data sources.

When you launch a notebook in Sentinel, the following happens: 1. Authentication: The notebook uses the logged-in user's Azure AD credentials via the azure-identity library to authenticate to the Sentinel workspace. 2. Compute Provisioning: An Azure Machine Learning compute instance (or serverless Spark compute) is allocated. By default, a compute instance with 2 cores and 8 GB RAM is created, but you can customize it. 3. Kernel Selection: The notebook runs a Python kernel (3.8 or later) with pre-installed libraries: msticpy, pandas, matplotlib, seaborn, scikit-learn, Kqlmagic, etc. 4. Data Access: Using msticpy's QueryProvider, the notebook connects to the Sentinel LogAnalytics workspace and executes KQL queries, returning results as pandas DataFrames. 5. Execution Flow: Cells are executed sequentially. Output (tables, charts, maps) is rendered inline. The notebook state persists across cell executions, allowing iterative analysis.

To create a new notebook from the Sentinel portal:

To verify the environment:

# Check MSTICpy version
import msticpy
print(msticpy.__version__)

# Check connection to Sentinel
from msticpy.data import QueryProvider
qry_prov = QueryProvider('LogAnalytics')
qry_prov.connect(connection_str='workspace://your_workspace_id/your_tenant_id')
# Test query
df = qry_prov.exec_query('SigninLogs | take 10')
df.head()

For SC-200, know that:

# Cell 1: Connect to Sentinel
from msticpy.data import QueryProvider
qry_prov = QueryProvider('LogAnalytics')
qry_prov.connect()  # Uses default credentials

# Cell 2: Get failed sign-ins
query = """
SigninLogs
| where ResultType != 0
| project TimeGenerated, UserPrincipalName, IPAddress, ResultType
| take 1000
"""
failed_signins = qry_prov.exec_query(query)

# Cell 3: Enrich with geo-location
from msticpy.vis import foliummap
geo_df = qry_prov.geoip_lookup(df=failed_signins, ip_column='IPAddress')
foliummap.folium_plot(geo_df, ip_column='IPAddress')

# Cell 4: Cluster suspicious IPs
from msticpy.analysis import cluster_events
clusters = cluster_events(data=failed_signins, time_column='TimeGenerated', freq='1h', min_cluster_size=5)
clusters.plot()

A large financial institution uses Sentinel for SIEM. The SOC analysts use built-in analytics rules to detect common attacks, but they need to hunt for subtle lateral movement patterns that evade signature-based detection. They deploy a custom Jupyter notebook that queries Windows Event Logs (Event ID 4624, 4634, 4648) across all domain controllers. The notebook uses MSTICpy's event_cluster function to group logon events by source IP, target computer, and time window. It then enriches the IPs with geo-location and threat intelligence feeds. The notebook produces a heatmap of anomalous logon patterns. This allows the team to identify compromised accounts that authenticate from unusual locations. The notebook is parameterized with papermill to run daily on a schedule, and the output is saved to a shared Azure Blob storage. Misconfiguration: If the compute instance is too small (e.g., Standard_DS1_v2), the notebook may fail with memory errors when processing millions of events. The team learned to use a larger VM (Standard_DS3_v2) and set the idle timeout to 240 minutes to avoid losing progress during long runs.

A managed security service provider (MSSP) uses Sentinel to monitor multiple customer tenants. They receive thousands of low-severity alerts daily. To reduce analyst fatigue, they built a notebook that uses a pre-trained Random Forest classifier (from scikit-learn) to score alerts based on historical incident data. The notebook connects to each customer's Sentinel workspace via a loop, extracts alert details using KQL, runs the model, and assigns a priority score (0-100). Alerts with scores above 80 are automatically promoted to incidents via the Sentinel API. The notebook runs on a serverless Spark compute to handle the multi-tenant workload. Performance consideration: The notebook must handle authentication for each tenant using service principals. A common pitfall is not handling API rate limits (30 requests per minute per workspace). The team implemented exponential backoff in their Python code.

A healthcare organization suspects an insider is exfiltrating patient data. They use a notebook to correlate DLP alerts (from Microsoft 365) with user activity logs and Azure AD sign-ins. The notebook uses MSTICpy's browser_session extraction to identify user behavior anomalies. It also queries the Microsoft Graph API to check if the user has accessed sensitive SharePoint sites. The investigation produces a timeline visualization showing the user's actions before and after the DLP alert. The notebook is shared with the legal team as an HTML export for evidence. Misconfiguration: The notebook's compute instance was left running for weeks, incurring significant cost. The organization now enforces a policy to stop compute instances after 2 hours of inactivity.