This chapter covers Shadow IT Discovery in Microsoft Defender for Cloud Apps, a critical feature for identifying and managing unsanctioned cloud applications used within an organization. For the SC-900 exam, this topic falls under Domain 3: Security Solutions, Objective 3.2: Describe the capabilities of Microsoft Defender for Cloud Apps. Approximately 5-10% of exam questions touch on Defender for Cloud Apps capabilities, with Shadow IT Discovery being a key sub-topic. Understanding how Shadow IT Discovery works—from log collection to risk assessment—is essential for answering scenario-based questions about cloud app governance and security.
Jump to a section
Shadow IT Discovery in Microsoft Defender for Cloud Apps is like having a corporate private investigator who monitors all the apps employees use on company devices. Imagine a company with 1,000 employees, each using a company-issued laptop. The company's IT department provides approved tools (like Microsoft 365, Salesforce, and Zoom Enterprise). However, employees often download unauthorized apps—like personal file-sharing services (e.g., Dropbox), messaging apps (e.g., WhatsApp), or note-taking apps (e.g., Evernote)—to get work done faster. The private investigator (Defender for Cloud Apps) doesn't block these apps initially; instead, it watches traffic logs from the company's firewall and proxy servers. It identifies each app by its unique digital fingerprint (e.g., destination IP ranges, TLS certificate details, HTTP headers). For each app detected, the investigator records: how many employees use it, how much data they upload/download, and whether the app has any known security risks (e.g., low encryption, no multi-factor authentication, data residency issues). The investigator then compiles a report categorizing apps as 'sanctioned' (approved), 'unsanctioned' (risky and blocked), or 'monitored' (not blocked but watched). This allows the company to make informed decisions: block high-risk apps, negotiate enterprise licenses for popular unsanctioned apps, or accept low-risk apps. The key mechanic is passive observation via log analysis—not active scanning or blocking—until the company decides to act.
What is Shadow IT Discovery?
Shadow IT refers to cloud applications and services used by employees without explicit approval or knowledge of the IT department. These apps can pose significant security risks, including data leakage, compliance violations, and malware introduction. Microsoft Defender for Cloud Apps provides Shadow IT Discovery by analyzing traffic logs from your network (via firewalls, proxies, or cloud app connectors) to identify which cloud apps are being used, by whom, and with what risk profile.
How It Works: The Mechanism
Shadow IT Discovery operates through a series of automated steps:
1. Log Collection: Defender for Cloud Apps ingests traffic logs from network devices (e.g., firewalls, proxies, CASBs). Supported log formats include Common Event Format (CEF), Web Proxy logs, and Syslog. Logs must be uploaded manually or via automated log collectors (on-premises or cloud-based).
2. Log Parsing and Normalization: The service parses logs to extract key fields: source IP (user), destination URL, bytes sent/received, timestamp, and user agent. It normalizes these fields into a standard schema.
3. App Identification: Using a proprietary cloud app catalog of over 31,000 apps, Defender for Cloud Apps matches destination URLs against app signatures. Signatures include domain names, IP ranges, TLS certificate attributes, and known API endpoints. For example, traffic to dropbox.com is identified as Dropbox; traffic to api.github.com is identified as GitHub.
4. Risk Assessment: Each identified app is scored against 80+ risk factors, including: security (encryption, authentication methods), compliance (SOC 2, ISO 27001, GDPR), legal (data residency, subpoena history), and business continuity (uptime, support). The risk score ranges from 0 (lowest risk) to 10 (highest risk).
5. User Attribution: Source IP addresses in logs are correlated with Azure Active Directory (Azure AD) user identities using IP-to-user mapping. This requires Azure AD logs or integration with identity providers.
6. Reporting and Alerts: Discovered apps are displayed in the Defender for Cloud Apps dashboard under 'Cloud Discovery'. Reports show:
- Top apps by user count - Data upload/download volumes - Risk categories (low, medium, high) - Compliance status Alerts can trigger on: new high-risk app detected, data upload exceeding threshold, or unsanctioned app usage.
Key Components, Values, Defaults, and Timers
Cloud App Catalog: Over 31,000 apps indexed. Each app has a predefined risk score. The catalog is updated weekly.
Risk Score: 0 (lowest) to 10 (highest). Default thresholds: Low risk (0-3), Medium risk (4-6), High risk (7-10).
Log Upload: Logs can be uploaded manually (via Defender for Cloud Apps portal) or automatically using log collectors. Automatic upload interval: every 1 hour (configurable).
Data Retention: Parsed discovery data is retained for 90 days. Raw logs are retained for 30 days.
Anonymization: By default, usernames are anonymized in reports to comply with privacy regulations. Can be disabled.
Continuous vs. Snapshot Reports: Snapshot reports are single log uploads for immediate analysis. Continuous reports aggregate logs over time (requires log collector).
App Tagging: Apps can be tagged as 'Sanctioned' (approved), 'Unsanctioned' (blocked), or 'Monitored' (watched). Tagging affects risk score and alerts.
Configuration and Verification Commands
Shadow IT Discovery is configured in the Microsoft Defender for Cloud Apps portal under 'Cloud Discovery' > 'Settings'. Key steps: 1. Create a data source: Define log format (e.g., Cisco ASA, Palo Alto Networks, Zscaler). 2. Upload logs: Via portal or configure log collector. 3. Generate continuous report: For automatic analysis.
To verify discovery is working:
- Check 'Cloud Discovery' > 'Dashboard' for app usage data. - Use PowerShell to export discovery data:
Connect-MicrosoftService
$token = Get-MicrosoftServiceToken
Get-MicrosoftServiceCloudAppDiscovery -ReportId <ReportId>- Use Defender for Cloud Apps API to query discovered apps:
GET https://api.security.microsoft.com/api/discovery/apps
Authorization: Bearer <token>Interaction with Related Technologies
Microsoft Defender for Identity: Correlates discovered app usage with user behavior analytics to detect compromised accounts.
Microsoft Defender for Endpoint: If Defender for Endpoint is deployed, it can provide cloud app usage data directly from endpoints, reducing reliance on network logs.
Azure AD Conditional Access: Discovered apps can be used as conditions for access policies. For example, block access to unsanctioned apps.
Microsoft Information Protection (MIP): Data classification labels can be applied to files discovered in shadow apps.
Microsoft Cloud App Security (MCAS): Shadow IT Discovery is a core feature of MCAS, now part of Defender for Cloud Apps.
Real-World Example: Log Format
A typical log entry from a Cisco ASA firewall:
<166>May 14 14:34:22 192.168.1.1 %ASA-6-302013: Built outbound TCP connection 12345 for outside:10.0.0.10/443 (10.0.0.10/443) to inside:192.168.1.100/54321 (192.168.1.100/54321)Defender for Cloud Apps parses this to extract: source IP (192.168.1.100), destination IP (10.0.0.10), destination port (443), timestamp. It then queries the app catalog for 10.0.0.10 to identify the app (e.g., Microsoft 365).
Configure Log Source
In the Microsoft Defender for Cloud Apps portal, navigate to 'Cloud Discovery' > 'Settings' > 'Log collectors'. Add a new data source by selecting the log format that matches your network device (e.g., Cisco ASA, Palo Alto Networks, Blue Coat). Provide a name and optional description. The system will generate a script to deploy the log collector on a Windows Server or Linux machine. This script installs the collector service that periodically fetches logs from your device via Syslog or FTP. The default collection interval is 1 hour, but you can set it to as low as 5 minutes for real-time discovery. Ensure firewall rules allow the collector to communicate with Defender for Cloud Apps endpoints.
Upload Traffic Logs
If using a snapshot report, you manually upload a log file via the portal. Go to 'Cloud Discovery' > 'Create snapshot report'. Select the data source you configured, then upload the log file (supports .log, .txt, .csv, .zip). The file size limit is 1 GB for snapshot reports. The system parses the log and identifies cloud apps. For continuous reports, the log collector automatically sends logs every hour. The logs are stored in an Azure Blob Storage account managed by Microsoft. The collector compresses and encrypts logs before transmission using TLS 1.2.
Parse and Normalize Logs
Defender for Cloud Apps receives the raw logs and applies a parsing engine that converts device-specific log formats into a normalized schema. The normalized fields include: timestamp, source IP, destination IP, destination URL, user agent, bytes sent, bytes received, and action (allow/block). The parser handles variations in date formats, IP address representation, and field ordering. If the log format is unrecognized, you can define a custom parser using regex. The parser outputs a structured JSON object for each log entry.
Identify Cloud Apps
For each normalized log entry, the system extracts the destination domain name (e.g., 'dropbox.com') from the URL or reverse DNS lookup of the destination IP. It then queries the cloud app catalog, which contains over 31,000 app signatures. Signatures include: domain names, IP ranges, TLS certificate common names, and known API endpoints. If a match is found, the app is identified. If multiple apps share a domain (e.g., 'google.com' hosts Google Drive, Gmail, YouTube), the system uses additional heuristics like URL path and user agent to disambiguate. Unidentified destinations are flagged as 'Unknown' and can be manually classified.
Assess Risk and Generate Report
Each identified app is assigned a risk score based on 80+ factors. The score is calculated from security controls (e.g., encryption strength, MFA support), compliance certifications (e.g., SOC 2, ISO 27001), legal history (e.g., data retention policies), and business practices (e.g., uptime SLA). The risk score is a weighted average, with security factors given 50% weight, compliance 25%, legal 15%, and business 10%. The system then generates a report summarizing: total apps discovered, top users, data volume, and risk distribution. Reports are available in the dashboard and can be exported as CSV or PDF.
Enterprise Scenario 1: Financial Services Compliance
A global bank with 50,000 employees uses Defender for Cloud Apps to discover unsanctioned file-sharing apps. The bank's compliance team needs to ensure no customer data is uploaded to unapproved cloud storage. They configure continuous log collection from their Palo Alto Networks firewalls across all branches. Discovery reveals that 2,000 employees use Dropbox, 500 use Google Drive, and 300 use WeTransfer. Risk assessment shows Dropbox has a medium risk (score 5) due to lack of enterprise-grade encryption, while WeTransfer scores 8 (high risk) because it stores files on third-party servers with no audit trail. The bank tags Dropbox as 'Monitored' and WeTransfer as 'Unsanctioned'. They then use Azure AD Conditional Access to block access to WeTransfer from corporate devices. Within a week, WeTransfer usage drops to near zero. The bank also negotiates an enterprise Dropbox license to gain better control.
Enterprise Scenario 2: Healthcare Data Protection
A hospital network with 10,000 employees wants to prevent PHI leakage via cloud apps. They deploy Defender for Cloud Apps and integrate with Microsoft Defender for Endpoint to collect app usage data directly from endpoints (since many doctors use personal devices). Discovery shows that 1,200 employees use WhatsApp Web for messaging, which is high risk (score 9) because it lacks enterprise controls and data is stored on Meta servers. The hospital tags WhatsApp as 'Unsanctioned' and blocks it via Conditional Access. They also discover that 800 employees use Microsoft Teams (already sanctioned) but 200 use Slack, which is medium risk (score 4). The hospital decides to migrate Slack users to Teams for better compliance. The discovery data is used to justify the migration project.
Common Pitfalls and Misconfigurations
Incomplete Log Coverage: If firewalls only cover outbound traffic from corporate networks, remote workers using VPN may not be logged. Solution: Use Defender for Endpoint to capture endpoint app usage.
Log Format Errors: Custom log formats may not parse correctly. Always test with a sample log. Use the 'Test parser' feature in the portal.
User Attribution Failures: If source IPs are NATed (e.g., multiple users share one public IP), user attribution breaks. Solution: Enable user agent logging or integrate with Azure AD sign-in logs.
Performance at Scale: Log collectors can handle up to 50 GB of logs per day. For larger environments, deploy multiple collectors. Ensure the collector server has at least 4 cores and 8 GB RAM.
What SC-900 Tests on Shadow IT Discovery
The SC-900 exam focuses on the *concept* of Shadow IT Discovery and its role in cloud security, not on deep technical configuration. Key objective: Describe the capabilities of Microsoft Defender for Cloud Apps (3.2). Specific points tested:
What Shadow IT is and why it's a security risk.
How Defender for Cloud Apps discovers apps (via log analysis from network devices or endpoints).
The cloud app catalog (over 31,000 apps).
Risk assessment factors (security, compliance, legal, business).
The difference between sanctioned, unsanctioned, and monitored apps.
Integration with Conditional Access to block unsanctioned apps.
Common Wrong Answers and Why
'Shadow IT Discovery requires installing agents on all endpoints.' This is false. Discovery primarily uses network logs; endpoint agents are optional (via Defender for Endpoint). Candidates assume agent-based discovery because many security tools require agents.
'Defender for Cloud Apps can automatically block all unsanctioned apps.' False. Discovery only identifies apps; blocking requires separate Conditional Access policies or integration with a proxy. Candidates confuse identification with enforcement.
'The risk score is based solely on security factors.' False. It includes compliance, legal, and business factors. Candidates remember 'risk' and assume security only.
'Shadow IT Discovery only works with Microsoft 365 apps.' False. It discovers any cloud app in the catalog, including third-party apps.
Specific Numbers and Terms to Memorize
31,000+ apps in the catalog.
80+ risk factors.
Risk score range: 0-10.
Default risk categories: Low (0-3), Medium (4-6), High (7-10).
Log retention: 90 days for parsed data, 30 days for raw logs.
Snapshot vs. Continuous reports.
Edge Cases and Exceptions
If an app is not in the catalog, it appears as 'Unknown' and has no risk score. Candidates may think it's automatically low risk—wrong. Unknown apps should be investigated manually.
Discovery can identify apps even if TLS inspection is not enabled, as long as the destination IP is known. However, without TLS inspection, the full URL may not be visible, reducing accuracy.
User attribution requires the source IP to be mapped to a user. If logs show only NAT IPs, attribution fails. The exam may test that you need Azure AD sign-in logs to correlate.
How to Eliminate Wrong Answers
If a question asks 'What is required for Shadow IT Discovery?' and an option says 'Install agents on all devices', eliminate it—network logs are sufficient.
If an option says 'It blocks high-risk apps automatically', eliminate it—blocking requires separate configuration.
If an option says 'It only works for Microsoft apps', eliminate it—the catalog includes 31,000+ apps from all vendors.
If a question asks about risk score factors, look for answers that include compliance, legal, and business—not just security.
Shadow IT Discovery identifies unsanctioned cloud apps by analyzing network traffic logs from firewalls, proxies, or endpoints.
The cloud app catalog contains over 31,000 apps, each with a risk score from 0 (lowest) to 10 (highest).
Risk score is based on 80+ factors across security, compliance, legal, and business categories.
Discovery does not automatically block apps; blocking requires Conditional Access or proxy integration.
Snapshot reports are manual, point-in-time analyses; continuous reports use log collectors for ongoing monitoring.
User attribution requires mapping source IPs to Azure AD users; NAT IPs can break attribution.
Parsed discovery data is retained for 90 days; raw logs for 30 days.
Apps can be tagged as Sanctioned, Unsanctioned, or Monitored to manage risk.
These come up on the exam all the time. Here's how to tell them apart.
Snapshot Report
Manual upload of a single log file.
Provides a point-in-time view of app usage.
No log collector required; ideal for ad-hoc analysis.
Data is retained for 90 days from upload.
Cannot track usage trends over time.
Continuous Report
Automatic log uploads via log collector (every 1 hour by default).
Provides ongoing, historical view of app usage.
Requires deploying a log collector on a Windows or Linux server.
Data is retained for 90 days from each upload.
Enables trend analysis and anomaly detection.
Mistake
Shadow IT Discovery requires installing software on every endpoint device.
Correct
Shadow IT Discovery primarily analyzes network traffic logs from firewalls, proxies, or cloud app connectors. Endpoint agents are optional and only needed if you use Microsoft Defender for Endpoint integration for additional visibility.
Mistake
Defender for Cloud Apps automatically blocks unsanctioned apps once discovered.
Correct
Discovery only identifies and reports app usage. Blocking requires manual action, such as tagging the app as 'Unsanctioned' and creating a Conditional Access policy or integrating with a proxy to enforce blocking.
Mistake
The risk score only considers security factors like encryption and authentication.
Correct
The risk score is based on over 80 factors across four categories: security (50% weight), compliance (25%), legal (15%), and business continuity (10%).
Mistake
Shadow IT Discovery can identify apps even if HTTPS traffic is encrypted and TLS inspection is not enabled.
Correct
If TLS inspection is not enabled, Defender for Cloud Apps can still identify apps by destination IP addresses and TLS certificate metadata (SNI). However, without decryption, it cannot see the full URL, which may reduce accuracy for apps hosted on shared IPs.
Mistake
Shadow IT Discovery only works for Microsoft 365 apps.
Correct
The cloud app catalog includes over 31,000 apps from various vendors, including popular third-party apps like Dropbox, Salesforce, and Slack. Discovery is not limited to Microsoft apps.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Shadow IT Discovery is a feature that identifies cloud applications being used within an organization without IT approval. It analyzes traffic logs from network devices (firewalls, proxies) or endpoints to detect app usage, assess risk, and provide visibility to security teams. It does not block apps automatically but enables informed governance decisions.
It ingests traffic logs from network devices (e.g., firewalls, proxies) or from Microsoft Defender for Endpoint. The logs are parsed to extract destination URLs, IPs, and user info. These are matched against a catalog of over 31,000 cloud apps. Each app is assigned a risk score based on 80+ factors. Reports show usage patterns and risk levels.
A snapshot report is a one-time manual upload of a log file for immediate analysis. A continuous report uses a log collector to automatically upload logs at regular intervals (default every hour), providing ongoing visibility and trend analysis. Continuous reports are better for long-term monitoring.
No, Discovery itself only identifies and reports app usage. To block apps, you must tag them as 'Unsanctioned' and then use integration with Azure AD Conditional Access or a proxy to enforce access controls. The exam tests that Discovery is a detection, not enforcement, tool.
The risk score considers over 80 factors across four categories: security (e.g., encryption, authentication, vulnerability history) weighted at 50%, compliance (e.g., SOC 2, ISO 27001, GDPR) at 25%, legal (e.g., data residency, subpoena history) at 15%, and business continuity (e.g., uptime, support) at 10%.
Parsed discovery data (reports and app usage records) is retained for 90 days. Raw log files are retained for 30 days. After these periods, data is automatically deleted.
User attribution maps source IP addresses in logs to Azure AD user identities. This requires Azure AD sign-in logs or integration with an identity provider. If logs only show NAT IPs (multiple users sharing one IP), attribution fails and users appear as 'Unknown'.
You've just covered Shadow IT Discovery in Defender for Cloud Apps — now see how well it sticks with free SC-900 practice questions. Full explanations included, no account needed.
Done with this chapter?