This chapter covers malware analysis, specifically the two primary approaches: static analysis and dynamic analysis. For the CS0-003 exam, understanding the distinction between these methods, their tools, and their appropriate use cases is critical. Approximately 10-15% of Security Operations domain questions touch on malware analysis techniques. You must know which tools belong to which category, what information each method reveals, and how to combine them for comprehensive threat intelligence.
Jump to a section
Think of static malware analysis as an autopsy. The forensic pathologist examines a corpse (the malware file) without ever bringing it back to life. They take photographs, measure the body, weigh organs, and look for external wounds. They may dissect (disassemble) to see the internal structure, but the body never moves or acts. In contrast, dynamic analysis is like performing surgery on a living patient under anesthesia. The patient is alive (the malware runs in a sandbox), and the surgeon monitors vital signs (system calls, network traffic, memory usage) in real time. The surgeon can see how the heart responds to incisions (how malware reacts to triggers) and can intervene if the patient crashes (sandbox reset). The key difference: autopsy reveals structure and static clues; surgery reveals behavior and dynamic interactions. Just as an autopsy cannot show how a person walked or talked, static analysis cannot reveal runtime behaviors like network connections or encryption. And just as surgery cannot show the full anatomy without cutting, dynamic analysis may miss code paths that require specific triggers. A competent analyst uses both: autopsy first to understand the corpse, then surgery to see it in action.
What is Malware Analysis?
Malware analysis is the process of dissecting malicious software to understand its purpose, functionality, capabilities, and origin. Analysts perform malware analysis to develop signatures, determine indicators of compromise (IOCs), understand attack vectors, and inform incident response. The two overarching categories are static analysis and dynamic analysis.
Static Analysis: Examining Without Execution
Static analysis involves examining the malware's code, structure, and metadata without actually running the program. It is safe because the malware never executes. Static analysis can be basic (file properties, strings, hashes) or advanced (disassembly, decompilation).
Basic Static Analysis includes:
- File fingerprinting: Calculating hashes (MD5, SHA-1, SHA-256) to uniquely identify the sample and check against known malware databases like VirusTotal.
- File type identification: Using tools like file command or TrID to determine the true file type, as attackers may disguise executables with double extensions (e.g., invoice.pdf.exe).
- String extraction: Extracting human-readable strings using strings utility. Strings can reveal IP addresses, domain names, registry keys, file paths, error messages, and embedded commands.
- PE header analysis: For Windows executables, examining the Portable Executable (PE) structure using tools like PEview, pefile, or CFF Explorer. Key fields include entry point, section names, imported/exported functions, and compile timestamp.
- Packing detection: Packed executables have compressed or obfuscated code. Tools like PEiD or Detect It Easy (DIE) can identify common packers (UPX, ASPack, Themida). Packed files often have high entropy and few imports.
Advanced Static Analysis involves:
- Disassembly: Converting machine code into assembly language using a disassembler like IDA Pro, Ghidra, or Radare2. Analysts examine the control flow graph, look for suspicious API calls (e.g., CreateRemoteThread, WriteProcessMemory), and identify anti-analysis techniques.
- Decompilation: Tools like Ghidra or Hex-Rays can reconstruct high-level C-like code from assembly, making analysis easier.
Limitations of Static Analysis:
Cannot reveal runtime behavior such as network connections, decrypted payloads, or triggered conditions.
Malware can use anti-static techniques: packing, obfuscation, junk code, anti-disassembly (e.g., opaque predicates).
Encrypted or compressed payloads are invisible until executed.
Dynamic Analysis: Executing in a Controlled Environment
Dynamic analysis involves running the malware in a safe, isolated environment (sandbox) and observing its behavior. It reveals what the malware actually does at runtime.
Sandbox Types: - Hardware sandbox: A dedicated physical machine that can be reimaged after analysis. Expensive but evasive malware can detect virtual environments. - Virtual machine sandbox: Using VMWare, VirtualBox, or Hyper-V. Common and cost-effective, but malware may detect virtualization artifacts (e.g., MAC addresses, registry keys). - Cloud-based sandbox: Services like Cuckoo Sandbox, Joe Sandbox, or Any.Run provide automated analysis.
What Dynamic Analysis Captures: - Process activity: Which processes are created, terminated, or injected. Tools: Process Monitor (Procmon), Process Hacker. - File system changes: Files created, modified, deleted. Tools: Procmon, Regshot. - Registry changes: Keys created or modified. Tools: Regshot, Procmon. - Network activity: Connections made, DNS queries, HTTP requests, data exfiltration. Tools: Wireshark, tcpdump, FakeNet. - Memory dumps: Capturing process memory to extract decrypted payloads or injected code. Tools: Volatility, WinDbg. - API calls: Intercepting system calls (e.g., NtCreateFile, NtWriteVirtualMemory). Tools: API Monitor, Procmon.
Analysis Steps: 1. Setup: Configure the sandbox with networking (simulated or real). Use tools like INetSim or FakeNet to simulate network services. 2. Execution: Run the malware and monitor for a set period (e.g., 5 minutes). 3. Capture: Log all changes and network traffic. 4. Post-analysis: Review logs, extract IOCs, and correlate with static findings.
Limitations of Dynamic Analysis:
Malware may detect sandbox environment and behave innocently (sleep, check for debuggers).
Not all code paths execute; condition-triggered payloads (e.g., specific date, command) may not activate.
Resource-intensive and time-consuming.
Malware can delete itself or alter system state, requiring snapshots.
Combining Static and Dynamic Analysis
A thorough analysis uses both approaches iteratively: 1. Initial static: Get hash, check VirusTotal, extract strings, identify packer. 2. Unpack if needed: Use static unpacking tools or run the malware to dump unpacked code from memory. 3. Dynamic analysis: Run in sandbox to capture behavior, confirm network IOCs, observe process injection. 4. Refined static: Disassemble unpacked code to understand core logic, identify C2 protocols. 5. Repeat: If new IOCs found, search for them in other samples.
Tools Summary
Static Analysis Tools:
- file, strings, md5sum, sha256sum (Linux)
- PEview, CFF Explorer, PEiD, Detect It Easy (Windows)
- IDA Pro, Ghidra, Radare2 (Disassemblers)
- VirusTotal, Hybrid Analysis (Online scanners)
Dynamic Analysis Tools:
- Cuckoo Sandbox (Open-source automated sandbox)
- Procmon, Process Hacker, Regshot (Windows monitoring)
- Wireshark, tcpdump, FakeNet, INetSim (Network monitoring)
- Volatility, Rekall (Memory forensics)
- Any.Run, Joe Sandbox (Online sandbox services)
Exam Relevance
For CS0-003, you need to know:
Which tools are static vs dynamic.
The pros and cons of each.
How to interpret static analysis results (e.g., suspicious strings, packed PE).
The importance of sandbox evasion and how malware detects virtual environments.
The role of malware analysis in threat intelligence (e.g., extracting IOCs).
Common exam scenario: A suspicious file is found. You must choose the best first step (usually static analysis for safety) or interpret a static analysis report to identify malicious indicators.
Step-by-Step Malware Analysis Process
A structured approach ensures thoroughness:
Preserve the sample: Create a forensic copy (bit-for-bit). Compute hash for integrity.
Basic static: Determine file type, extract strings, identify any packer.
Advanced static: Disassemble key sections if needed.
Dynamic setup: Configure sandbox, ensure network simulation.
Execute and monitor: Run malware for sufficient time.
Analyze logs: Identify IOCs: IPs, domains, registry keys, filenames.
Correlate: Match dynamic behavior with static code.
Document and report: Produce indicators, signatures, and mitigation recommendations.
Anti-Analysis Techniques
Malware authors employ techniques to evade both static and dynamic analysis:
Packing/Obfuscation: Compresses or encrypts original code. Static analysis sees only the unpacker stub.
Anti-disassembly: Junk bytes, opaque predicates, call stack tampering.
Anti-debugging: IsDebuggerPresent, NtGlobalFlag, timing checks.
Virtual machine detection: Checking for VM artifacts (MAC vendor, registry keys, hardware IDs).
Sandbox detection: Checking for user activity (mouse movements, uptime), processes (vmtoolsd), or small disk size.
Conditional execution: Payload only executes if specific conditions met (e.g., date, command presence).
Time-based evasion: Sleeps for extended periods or uses timers to delay malicious activity.
Analysts counter these with: - Unpacking: Use generic unpackers or dump process memory after execution. - Debugger hiding: Use stealth debuggers or kernel-level debugging. - Custom sandbox: Use physical machines or modify VM artifacts. - Longer analysis time: Run sandbox for hours or days. - Human interaction: Simulate mouse clicks or keyboard input.
Conclusion
Malware analysis is a critical skill for cybersecurity professionals. Static analysis provides a safe starting point and reveals structural information, while dynamic analysis uncovers actual behavior. Mastering both and understanding their limitations is essential for effective threat analysis and incident response. For the CS0-003 exam, focus on the tools, processes, and common evasion tactics.
Preserve and fingerprint the sample
Before any analysis, create a forensic copy of the malware file using a write-blocker or dedicated imaging tool. Compute cryptographic hashes (MD5, SHA-1, SHA-256) to ensure integrity and to query threat intelligence databases like VirusTotal. Hash values are unique identifiers; a single bit change produces a completely different hash. Record the file size, creation date, and any metadata. This step ensures that the original evidence remains unaltered and that you can later prove the sample's integrity.
Perform basic static analysis
Run the `file` command to determine the true file type (e.g., PE32 executable, PDF, script). Extract strings using `strings` with a minimum length (e.g., 6 characters) to find URLs, IP addresses, registry keys, and filenames. Examine the PE header with tools like `pefile` (Python) or `PEview`. Look for suspicious imports (e.g., `URLDownloadToFile`, `WinExec`), unusual section names, or high entropy indicating packing. Check the compile timestamp; a file claiming to be from 2023 but compiled in 2020 may be suspicious. Document all findings.
Detect packing and obfuscation
Use tools like `Detect It Easy` (DIE) or `PEiD` to identify common packers (UPX, ASPack, MEW). Packed executables have few imports and high entropy. If packed, attempt to unpack using the packer's own decompressor (e.g., `upx -d`) or a generic unpacker like `unpacme`. If static unpacking fails, you may need to execute the malware and dump the unpacked code from memory. Record the packer type and version; this information helps in understanding the malware's complexity.
Perform advanced static analysis
Load the unpacked sample into a disassembler (IDA Pro, Ghidra). Analyze the control flow graph, identify main functions, and look for suspicious API calls. Focus on network-related APIs (e.g., `socket`, `connect`, `send`), process manipulation APIs (e.g., `CreateRemoteThread`, `WriteProcessMemory`), and file/registry operations. Look for encoded strings or data; common encoding includes base64, XOR, or custom algorithms. Document any anti-analysis techniques (e.g., anti-debugging calls, junk code).
Set up dynamic analysis environment
Configure a sandbox (e.g., Cuckoo Sandbox, VirtualBox with snapshots). Ensure the virtual machine has internet connectivity (simulated or real) and monitoring tools installed (Procmon, Wireshark). Use fake network services (INetSim) to simulate DNS, HTTP, and SMTP. Take a snapshot of the clean VM. If using a physical machine, prepare a reimaging solution. Important: Disable any security software that might interfere. Record the baseline state of the system (registry, files, processes).
Execute and monitor malware
Run the malware in the sandbox. Monitor all activity for a defined period (e.g., 5-10 minutes). Use Procmon to log file system, registry, and process activity. Capture network traffic with Wireshark. Take memory snapshots if needed. Note any delays or conditional behavior (e.g., malware sleeping for 60 seconds before connecting). If the malware does not exhibit malicious behavior, try triggering different conditions (e.g., simulate command line arguments, run as administrator, or change system time). After execution, revert the VM to clean state.
Analyze dynamic behavior and extract IOCs
Review Procmon logs for processes created, files written, and registry modifications. Identify dropped files (e.g., `%AppData%\malware.exe`). From network traffic, extract IP addresses, domains, HTTP headers, and any exfiltrated data. Use tools like `tcpflow` to reconstruct TCP streams. Correlate with static findings: e.g., a suspicious IP from strings should appear in network logs. Document all IOCs: hashes, IPs, domains, file paths, registry keys, mutexes.
Document and report findings
Compile a comprehensive report including: sample hash, file type, packer used, static analysis findings (suspicious strings, imports, disassembly highlights), dynamic behavior (process tree, network connections, registry changes), and all IOCs. Provide detection rules (e.g., YARA rules, Snort signatures) and mitigation recommendations. The report should be clear enough for incident responders to use. Store all raw data (logs, pcap, memory dumps) for future reference.
Enterprise SOC: Phishing Incident Response
A large enterprise receives a phishing email with an attached Word document containing macros. The SOC analyst performs static analysis first: they extract the hash and check VirusTotal, which shows no hits. Using olevba (a tool for analyzing OLE objects), they extract the macro code and see it downloads a payload from a hardcoded URL. The URL is a known malicious domain from a threat feed, confirming the email is malicious. The analyst then performs dynamic analysis: they open the document in a sandbox with simulated network services, and the macro executes, downloading a portable executable (PE) file. The PE is then analyzed statically (packed with UPX) and dynamically (it connects to a C2 server, performs reconnaissance, and attempts to exfiltrate data). The SOC can now block the URL, create a YARA rule for the macro, and deploy endpoint detection signatures.
Malware Research Lab: Reverse Engineering a Ransomware Variant
A cybersecurity vendor receives a new ransomware sample. The analyst starts with static analysis: file shows it's a PE32 executable; strings reveals a Bitcoin wallet address and a ransom note template. The PE header shows it's packed with a custom packer. Advanced static analysis with IDA Pro reveals anti-debugging tricks (calling IsDebuggerPresent in a loop). The analyst then runs the sample in a hardware sandbox (physical machine) to avoid VM detection. Dynamic analysis shows the ransomware encrypts files with AES-256 and writes a ransom note. Memory forensics extracts the encryption key before the malware terminates. The analyst combines static and dynamic findings to produce a comprehensive report, including decryption tools and detection rules.
Common Misconfigurations and Pitfalls
Running malware on a production network: Always use isolated sandboxes. A misconfigured sandbox with real network access can result in actual infection or C2 communication.
Insufficient monitoring time: Some malware uses sleep timers (e.g., 30 minutes) to evade sandboxes. Set the analysis duration long enough or use time acceleration techniques.
Over-reliance on automated sandboxes: Automated tools may miss conditional payloads. Manual analysis is often needed to trigger specific code paths.
Ignoring static analysis: Jumping straight to dynamic analysis risks missing key IOCs and may trigger malware that detects the sandbox. Always start with static.
Failure to clean the sandbox: Not reverting to a clean snapshot between analyses can contaminate results and allow malware to persist.
Exactly What CS0-003 Tests
Under Domain 1.0 (Security Operations), Objective 1.2: "Given a scenario, analyze indicators of compromise and formulate an appropriate response." This includes malware analysis techniques. The exam expects you to:
Distinguish between static and dynamic analysis.
Select the appropriate analysis method for a given scenario.
Identify tools associated with each method.
Interpret outputs from static analysis (e.g., strings, PE header info) and dynamic analysis (e.g., process monitor logs).
Recognize anti-analysis techniques and how to overcome them.
Common Wrong Answers and Why Candidates Choose Them
Choosing dynamic analysis as the first step: Many candidates think running the malware is faster. But dynamic analysis can alert the malware or cause damage. Static analysis is always the safe first step.
Mixing up tools: e.g., thinking Procmon is a static analysis tool. Procmon captures runtime behavior; it's dynamic. Conversely, strings is static.
Believing static analysis can reveal all behavior: Static analysis cannot show network connections or decrypted payloads; dynamic analysis is needed for runtime behavior.
Assuming a packed file is definitely malicious: Legitimate software also uses packing (e.g., UPX for compression). Packing is suspicious but not definitive.
Specific Numbers and Terms That Appear on the Exam
Hash algorithms: MD5, SHA-1, SHA-256 (know that SHA-256 is preferred for integrity).
Common packers: UPX, ASPack, Themida.
PE sections: .text (code), .data (initialized data), .rdata (read-only data), .rsrc (resources).
Anti-debugging API: IsDebuggerPresent, NtGlobalFlag.
Sandbox detection: checking for VM processes (vmtoolsd, VBoxService), small disk size (<60GB), lack of user activity.
Dynamic analysis tools: Cuckoo Sandbox, Procmon, Wireshark.
Static analysis tools: IDA Pro, Ghidra, strings, PEview.
Edge Cases and Exceptions
Fileless malware: Malware that runs in memory (e.g., PowerShell scripts) may not leave a PE file for static analysis. Dynamic analysis (memory forensics) is essential.
Multi-stage malware: Initial dropper may be benign, but subsequent stages are malicious. Dynamic analysis over longer periods is needed.
Conditional execution: Malware that only activates on a specific date or after receiving a command. Dynamic analysis may require manual triggering.
How to Eliminate Wrong Answers
If the question asks for the first step in analyzing a suspicious file, the answer is always static analysis (e.g., compute hash, check VirusTotal, extract strings).
If the question asks which tool to use for observing network connections, the answer is a dynamic tool (e.g., Wireshark, tcpdump).
If the question describes a packed executable, the correct next step is to attempt unpacking or proceed to dynamic analysis to obtain unpacked code from memory.
For questions about extracting IOCs, both static and dynamic can produce IOCs, but dynamic provides behavioral IOCs (C2 IPs, registry keys) while static provides structural IOCs (hashes, strings).
Always start with static analysis (hash, strings, file type) before dynamic analysis to ensure safety and gather initial IOCs.
Static analysis cannot reveal runtime behavior; dynamic analysis is needed for network connections, decrypted data, and process injection.
Common static analysis tools: strings, PEview, IDA Pro, Ghidra, VirusTotal.
Common dynamic analysis tools: Cuckoo Sandbox, Procmon, Wireshark, Volatility.
Malware uses anti-analysis techniques: packing, obfuscation, anti-debugging, sandbox detection, conditional execution.
To overcome packing, use unpacking tools or dump process memory during dynamic analysis.
Always document IOCs from both static and dynamic analysis for threat intelligence.
For the exam, know which tools belong to which category and the appropriate first steps in malware analysis.
These come up on the exam all the time. Here's how to tell them apart.
Static Analysis
Examines malware without execution; safe.
Reveals structure, metadata, and embedded strings.
Cannot observe runtime behavior like network connections.
Tools: file, strings, IDA Pro, Ghidra, PEview.
Effective against packed code only if unpacked first.
Dynamic Analysis
Executes malware in a controlled environment; risk of escape.
Reveals actual behavior: process, file, registry, network activity.
Can capture decrypted payloads and runtime IOCs.
Tools: Cuckoo Sandbox, Procmon, Wireshark, Volatility.
May miss code paths that require specific triggers.
Mistake
Static analysis is always faster and easier than dynamic analysis.
Correct
Basic static analysis is fast, but advanced static analysis (disassembly) can be extremely time-consuming and complex, especially with obfuscated code. Dynamic analysis can often reveal behavior more quickly, but requires proper sandbox setup.
Mistake
If a file has no suspicious strings, it is safe.
Correct
Malware can obfuscate strings (e.g., XOR-encoded, base64) that are not visible with a simple `strings` command. Additionally, fileless malware may have no strings at all.
Mistake
Dynamic analysis in a sandbox always reveals the full behavior of malware.
Correct
Malware can detect sandbox environments and avoid malicious actions. It may also require specific triggers (date, command, user interaction) that are not present in the sandbox.
Mistake
A file with a valid digital signature is safe.
Correct
Attackers can steal or misuse legitimate code signing certificates. Malware signed with stolen certificates is not uncommon. Always verify the certificate chain and reputation.
Mistake
Packed executables are always malicious.
Correct
Many legitimate applications use packers like UPX to reduce file size. However, packing is common in malware to evade signature detection, so it warrants further investigation.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
The first step is always static analysis. Start by computing the hash (MD5, SHA-1, SHA-256) and querying threat intelligence databases like VirusTotal. Then determine the file type using the `file` command, extract strings, and examine the PE header. This approach is safe because the malware is not executed, and it can quickly reveal known IOCs.
Use tools like Detect It Easy (DIE) or PEiD to identify common packers. Indicators of packing include high entropy, few imports (often only LoadLibrary and GetProcAddress), and unusual section names (e.g., UPX0, UPX1). If the file is packed, you may need to unpack it using the packer's own tool (e.g., `upx -d`) or run the malware and dump the unpacked process memory.
Static analysis examines the malware without executing it, focusing on file properties, code structure, and embedded strings. Dynamic analysis executes the malware in a sandbox and monitors its behavior (process, file, registry, network). Static analysis is safe but limited; dynamic analysis reveals runtime behavior but requires isolation and may miss conditional code paths.
Common dynamic analysis tools include Cuckoo Sandbox (automated), Process Monitor (Procmon) for file/registry/process activity, Wireshark for network traffic, and Volatility for memory forensics. Online sandboxes like Any.Run and Joe Sandbox also provide dynamic analysis.
Malware checks for artifacts of virtual machines: MAC address prefixes (00:0C:29 for VMWare), processes like vmtoolsd or VBoxService, registry keys (e.g., HKLM\HARDWARE\DESCRIPTION\System\BIOS containing 'VMware'), small disk size (<60GB), low memory, or lack of user activity (no recent mouse movements, short uptime).
Malware analysis helps identify the scope of an infection, extract indicators of compromise (IOCs) for detection and containment, understand the attack vector, and develop signatures for prevention. It informs the response strategy and helps prevent future attacks by feeding threat intelligence.
Static analysis is limited for fileless malware because there is no file to analyze. However, you can analyze scripts (PowerShell, VBS) statically by examining their code. For runtime behavior, dynamic analysis and memory forensics are essential.
You've just covered Malware Analysis: Static vs Dynamic — now see how well it sticks with free CS0-003 practice questions. Full explanations included, no account needed.
Done with this chapter?