CEF and Syslog Connector Configuration

CEF (Common Event Format) and Syslog connectors are data ingestion methods used by Microsoft Sentinel to collect security logs from on-premises and third-party security appliances that cannot directly send logs to Azure via the Log Analytics agent or Azure Monitor Agent. These connectors allow Sentinel to receive logs from devices such as firewalls (e.g., Palo Alto, Fortinet), intrusion detection systems (e.g., Snort), and other security tools that output logs in Syslog format (RFC 3164 or RFC 5424) or CEF (a format originally by ArcSight).

The connector is implemented as a Log Analytics agent (or Azure Monitor Agent) running on a Windows or Linux machine that acts as a log collector. This machine receives Syslog or CEF messages over UDP or TCP, parses them, and forwards them to Azure Sentinel as CommonSecurityLog (for CEF) or Syslog (for Syslog) tables.

Step 1: Deploy Log Analytics Agent on a Linux machine - Use the Azure portal or manual installation script. - The agent must be connected to the same workspace as Sentinel.

Step 2: Configure Syslog/CEF data sources - In Sentinel, go to Data connectors -> Common Event Format (CEF) or Syslog. - Open the connector page and click "Open connector page". - Follow the instructions to run the deployment script on the log forwarder. The script installs the agent, configures Syslog daemon (rsyslog or syslog-ng), and sets up the connector.

Step 3: Configure the security device to send logs - Point the device to the log forwarder's IP address and port (e.g., UDP 514). - Ensure the device sends logs in CEF format if using the CEF connector.

Verification Commands: - Check agent connectivity: sudo /opt/microsoft/omsagent/bin/omsadmin.sh -l - Check Syslog reception: tcpdump -i any port 514 - Check the agent logs: tail -f /var/opt/microsoft/omsagent/log/omsagent.log - In Sentinel, query the CommonSecurityLog table: CommonSecurityLog | take 10

The CEF and Syslog connectors work alongside the Azure Monitor Agent (AMA) for newer deployments. AMA supports Syslog via the Syslog data collection rule (DCR), but does not support CEF natively. For CEF, the Log Analytics agent is still required. The connectors integrate with Sentinel's built-in parsers and analytics rules. For example, the "Palo Alto PAN-OS" solution uses CommonSecurityLog data. Troubleshooting involves checking network connectivity, firewall rules, and agent logs. Common issues include message truncation (UDP has 64 KB limit), incorrect timestamp parsing, and mismatched field mappings.

A large enterprise uses multiple Palo Alto Networks firewalls across global offices. They need to centralize logs in Azure Sentinel for threat detection and compliance. The security team deploys a Linux log forwarder in each region (e.g., US, Europe, Asia) running the Log Analytics agent. Each firewall is configured to send CEF logs via TCP 514 to the nearest forwarder. The forwarders then forward the logs to a single Sentinel workspace. Performance considerations: each forwarder handles up to 10,000 EPS (events per second). At peak, the workspace receives 30,000 EPS, which is below the 5 MB/s limit (assuming average log size 500 bytes, 30,000 EPS = 15 MB/s, which exceeds the limit). The team implements filtering on the forwarders to drop low-severity logs (e.g., informational) and uses multiple workspaces for high-volume regions. Misconfiguration: if the forwarder's clock drifts, logs may be timestamped incorrectly, causing detection rules to fail. They use NTP and monitor clock skew with a KQL query.

A financial institution uses an older Snort IDS that only outputs Syslog (RFC 3164) and cannot be upgraded to CEF. They deploy a Windows-based Log Analytics agent as the forwarder. The Snort sensor sends UDP 514 logs to the forwarder. The agent parses the Syslog and populates the Syslog table. However, the message body is unstructured, so threat detection requires custom parsing. The SOC team writes KQL functions to extract IPs and signatures from the 'Message' field. They also set up a data collection rule to filter out known benign events (e.g., ARP scans) to reduce noise and cost. Common issue: Syslog messages longer than 1024 bytes are truncated by the default rsyslog configuration; they increase the message size limit to 8192 bytes.

A cloud gaming company uses a Radware DDoS mitigation appliance that sends CEF logs to Sentinel. The device generates up to 50,000 EPS during attacks. The log forwarder is a high-performance Linux server with 16 vCPUs and 32 GB RAM, using TCP with TLS to prevent loss. Despite this, they experience data gaps during attacks due to Sentinel's 5 MB/s ingestion limit. They implement a two-tier approach: the forwarder writes logs to a local buffer (e.g., a Kafka queue) and then batches them to Sentinel at a controlled rate. They also use a dedicated workspace for security logs and another for operational logs. Misconfiguration: if the TLS certificate expires, the connection drops and logs are lost. They set up monitoring for certificate expiry and automatic renewal.