Malware analysis is the process of taking a suspicious file or email attachment and figuring out exactly what it does, how it works, and how to stop it. For the CyberOps Associate exam, you must understand the different techniques and tools analysts use, not become a full-time malware analyst. This chapter gives you the core concepts and vocabulary you need for the test questions.
Jump to a section
A simple way to picture Malware Analysis Basics
A police detective's investigation is a precise match for malware analysis. The detective does not just look at a crime photo; she must examine the entire scene, the digital record of events, to find clues.
Imagine a burglary at a house. The detective first talks to the victim, who says "I think I left the back door unlocked." This is like receiving a malware sample from a system administrator. The detective then preserves the scene, taking photos and collecting fingerprints. This is static analysis: she looks at the suspicious file's properties, its creation date (the timestamp), where it came from (the file path), and its digital signature (the equivalent of a driver's license). She does not run the file.
Next, the detective might set up a monitored environment, a safe room, and run the burglar's tools (the malware) to see what they do. This is dynamic analysis. She watches the malware connect to a remote server, just as the detective watches the burglar try to sell stolen goods. She records the network address the malware calls, the files it changes, and the registry keys (like the burglar's list of locks) it modifies. The detective's notebook is the analyst's report, filled with Indicators of Compromise (IOCs) – the unique identifiers, such as the burglar's shoe size or the specific file hash. Each clue helps the detective prevent future crimes, just as each IOC helps an IT team block the same malware variant tomorrow.
Malware analysis is the IT equivalent of a crime lab for software. When a security system flags a suspicious file, someone must investigate it. That investigation is malware analysis. The goal is to understand the malware's capabilities, its origin, and its intended goal. This information is then used to write detection rules, clean infected systems, and prevent future infections.
There are two main branches of malware analysis: static analysis and dynamic analysis. They are complementary, like a detective first looking at a crime scene from a distance (static) and then stepping into it to look for footprints (dynamic).
Static Analysis Static analysis means examining the file without actually running it. It is a safe starting point because the malware never gets a chance to activate. Analysts look at the file's metadata – information about the file itself. This includes:
File Hash: A unique digital fingerprint of the file, like a person's DNA. The hash (often using SHA-256 algorithm) is a long string of numbers and letters. If you find a file with the same hash on another system, you know it is an exact copy.
Strings: Readable text hidden inside the file. Malware often contains text like a criminal's hidden message. These strings might show URLs, file paths, or error messages. For example, a string might be "http://bad-server.com/malware.exe" which tells the analyst where the malware wants to connect.
File Structure: The file's format, such as an EXE (executable) or DLL (Dynamic Link Library). Analysts examine the headers of the file to see if it is packed. A packed file is like a zipped folder that contains the real malware inside. Packers make static analysis harder because you cannot see the real code until it is unpacked.
Static analysis tools include strings command (to extract text), file command (to identify the file type), and VirusTotal (a free online service that checks a file against many antivirus engines).
Dynamic Analysis Dynamic analysis is the controlled execution of the malware. You run the malware in a safe, isolated environment called a sandbox. A sandbox is a virtual machine (VM), an emulated computer that runs on your real machine but is completely separate. If the malware destroys the VM, it does not affect your real system.
Analysts watch what the malware does in the sandbox. They monitor:
File System Changes: Does the malware create new files or modify existing ones?
Registry Changes (Windows): Does it add entries to the Windows registry (a database of settings)?
Network Traffic: Does it try to connect to a remote server? Analysts use tools like Wireshark (a packet analyser) to see the network requests.
Process Creation: Does it spawn other programs or hide itself?
Dynamic analysis reveals the malware's true functionality. A key technique is the use of a debugger, such as x64dbg or OllyDbg. A debugger lets the analyst step through the malware's code one instruction at a time, looking at the contents of CPU registers (temporary storage locations) and memory. This is the most detailed level of analysis, often called reverse engineering.
Why Both Are Needed Static analysis is quick, safe, and provides a broad overview. Dynamic analysis shows the malware in action, revealing behaviour that static analysis cannot. The exam expects you to know the difference, the tools used for each, and why a fast static check is always the first step before running anything dangerous.
Preserve the Evidence
When you first receive a suspicious file, you must preserve its state. This means making a bit-for-bit copy (forensic copy) of the file and storing it safely. Do not modify the original file. This ensures that if you need to re-analyse it later, you have an untainted sample. This is a basic principle of digital forensics.
Perform Initial Static Analysis
This is the safe first step. Use tools like 'file' to identify the file type and 'sha256sum' to compute the file hash. Check the hash against VirusTotal and internal threat intelligence feeds. Extract readable strings from the file using the 'strings' command. This step gives you clues about the malware's purpose without activating it.
Determine if the File is Packed
Use a tool like PEiD or PEview to analyse the file's executable structure. If the file is packed, unpack it using the appropriate unpacker (like UPX for UPX-packed files). Packed files hide their true code and make static analysis less effective. Unpacking reveals the actual malware code.
Set Up the Sandbox Environment
Prepare an isolated virtual machine (VM) for dynamic analysis. Ensure the VM has no network access to your real network, or use simulated network services. Take a clean snapshot of the VM's state so you can easily restore it after analysis. This environment must be completely separated from your production systems.
Execute and Observe Behaviour (Dynamic Analysis)
Copy the file (or unpacked version) into the sandbox VM and run it. Use tools like Process Monitor (procmon) to watch file system, registry, and process activity. Use Wireshark to capture network traffic. Record all behaviour: connections, new files, registry modifications. This reveals the malware's true intentions, such as installing a backdoor or stealing data.
Document Findings and Produce IOCs
After analysis, create a report summarising the malware's functionality, techniques used (e.g., packer, persistence mechanism), and specific IOCs. IOCs include file hashes, registry keys, IP addresses, and domain names. This report is used by the security team to block the malware and clean infected systems.
Imagine you are a security analyst working for a mid-sized company, Acme Corp. The company's email security gateway alerts you that an attachment in an email to a sales representative is suspicious. The attachment is called "Invoice.pdf.exe". Your day-to-day task is to investigate this alert using malware analysis basics.
Step 1: Initial Static Triage
You do not run the file. First, you copy it to a secure, isolated workstation. You run the file command on it. The result shows it is a Windows executable, not a PDF. This immediately confirms it is suspicious. You then compute its hash using a tool like sha256. You paste that hash into VirusTotal. The results show that three out of 60 antivirus engines flag it as a known trojan. You also run the strings command. You find a string that says "confidential_data.zip" and an IP address 192.168.1.105. This suggests the malware might try to steal files and send them to an internal server.
Step 2: Static Analysis of Structure
You use a tool like PEview to look at the PE (Portable Executable) structure of the file. The PE structure is the layout of every Windows executable file. You see that the file's entry point (where execution starts) is in a suspicious section of memory. You also notice the file is packed with UPX, a common packer. You use a UPX unpacking tool to decompress it. After unpacking, you rerun strings and find more clues, including a user agent string that mimics a web browser.
Step 3: Dynamic Analysis in a Sandbox You upload the unpacked file to a controlled sandbox environment (like Cuckoo Sandbox or a manual VM setup). The sandbox takes a snapshot of its state (a clean reference point) before running the file. The sandbox runs the file and records:
It creates a file C:\Users\profile\AppData\Local\temp\confidential_data.zip.
It makes a DNS query to try to resolve badserver.acmecorp.com.
It attempts to connect to the IP address you found earlier on port 443 (HTTPS).
This confirms it is data-stealing malware that tries to exfiltrate data over HTTPS to hide its traffic.
Step 4: Writing Detection Rules Based on the analysis, you write a rule for your network intrusion detection system (NIDS). The rule checks for outbound connections to the specific IP address on port 443 and for the creation of the specific filename pattern. You also update your endpoint protection tool to block the file hash. You send an alert to the sales representative's team to immediately power down that machine and call IT.
Step 5: Reporting You write a one-page report summarising the malware's functionality (data stealer), its IOCs (the hash, IP, filename), and the recommended actions. This report is shared with the security team and the CISO.
This is a real workflow. The exam will test your understanding of these steps: the difference between static and dynamic, what tools like strings and VirusTotal do, and why sandboxes are used.
The 200-201 exam tests your conceptual understanding of malware analysis, not the ability to use every tool. You need to know the definitions, the purpose of each technique, and the order of operations.
Exact Concepts Tested:
- The Difference Between Static and Dynamic Analysis: This is a core question you will see multiple times. Exam questions will describe a scenario and ask which analysis type is being performed. For example: "An analyst runs the strings command on a file without executing it. Which type of analysis is this?" Answer: Static. Trap: They might say "the analyst is monitoring network traffic from a sandbox" – that is dynamic.
- The Purpose of a Sandbox: You must know that a sandbox is an isolated environment used for dynamic analysis. You must know it prevents the malware from spreading. A common trap is confusing a sandbox with an antivirus program. An antivirus program detects malware, but a sandbox is used to analyse it safely.
- Common Analysis Tools: You must recognise the names and purposes of specific tools. For example:
- file command: Identifies the file type.
- sha256 or md5sum: Computes file hash.
- strings: Extracts text from a binary.
- VirusTotal: Online service for checking file hashes against multiple AV engines.
- Wireshark: Captures and analyses network packets.
- Indicators of Compromise (IOCs): The exam will test that IOCs are artefacts of an intrusion, such as file hashes, IP addresses, domain names, and registry keys. You must know that IOCs are used to detect the same malware on other systems.
- Packed vs. Unpacked Malware: You need to know that packers compress or encrypt malware to evade detection. Static analysis of a packed file is less effective. Unpacking (decompressing) is a common step before deeper analysis.
- Behavioural vs. Static IOCs: A behavioural IOC is something the malware
does (e.g. connecting to a specific domain at boot time). A static IOC is a characteristic of the file itself (e.g. its hash). The exam might ask you to classify a given IOC.
Trap Patterns on the Exam:
Trap: Overcomplication. The exam will present a scenario where a sophisticated tool is mentioned (like a debugger), but the question is simply about the type of analysis (static or dynamic). Do not overthink it.
Trap: False Equivalence. A question might ask about the purpose of a sandbox, offering an answer like "to clean the infected file" or "to block network traffic." The sandbox does not clean or block; it observes.
Trap: Sequencing. You will be asked what the first step of analysis should be. The correct answer is always static analysis (e.g. checking the file hash) before running it. Do not choose dynamic analysis first.
Key Definitions to Memorise for the Exam:
Static Analysis: Analysing a file without executing it.
Dynamic Analysis: Analysing a file by executing it in a controlled environment.
Sandbox: An isolated environment for dynamic analysis.
File Hash: A unique string that identifies a file.
Packed File: A file that has been compressed or encrypted to hide its contents.
IOC (Indicator of Compromise): An artefact of an intrusion that indicates a system has been compromised.
Malware analysis is split into two types: static analysis (without running the file) and dynamic analysis (running it in a sandbox), and you always perform static analysis first.
A file hash is a unique digital fingerprint used to identify and track malware, not to prove its trustworthiness.
A sandbox is an isolated virtual environment used to safely execute malware and observe its behaviour without harming the real network.
Common static analysis tools include 'file', 'strings', and VirusTotal, while dynamic analysis often uses Wireshark and debuggers.
Indicators of Compromise (IOCs) are the clues left behind by malware, such as file hashes, IP addresses, and domain names, and they are used to detect future infections.
Packed malware has been compressed or encoded to hide its true content, making static analysis harder and requiring unpacking before deeper analysis.
The exam tests your ability to distinguish between static and dynamic analysis techniques, not the mastery of every tool command.
VirusTotal checks a file against multiple antivirus engines; it is not a removal tool or a guarantee of safety.
These come up on the exam all the time. Here's how to tell them apart.
Static Analysis
Examines the file without executing it.
Safe: no risk of infection to the analysis system.
Provides quick overview of file metadata and strings.
Dynamic Analysis
Executes the file in a controlled sandbox.
Risk of infection if sandbox is misconfigured.
Reveals actual behaviour like network connections.
File Hash (e.g. SHA-256)
Unique fingerprint of file contents.
Does not verify the publisher.
Used for malware identification and integrity checks.
Digital Signature
Cryptographic stamp from a software publisher.
Verifies the file's source and authenticity.
Used to confirm legitimate software.
Packed Malware
Compressed or encoded to hide code.
Harder to analyse with static tools.
Often used to evade signature-based detection.
Unpacked Malware
Original code fully visible after unpacking.
Easier to analyse statically and dynamically.
More signs to detect with antimalware.
Sandbox
Isolated environment to run and observe malware.
Does not actively block or remove the file.
Used by analysts for in-depth behavioural analysis.
Antivirus Software
Detects and blocks known malware signatures.
Can remove or quarantine infected files.
Used for real-time protection on end-user machines.
Mistake
Malware analysis is only done by running suspicious files on a real computer.
Correct
The first step of analysis is always static analysis, which does NOT involve running the file. Running a file without proper isolation is dangerous and can corrupt the entire network.
Beginners may assume that the only way to know what a file does is to run it, but IT professionals always prioritise safety first. The exam explicitly tests the sequence and safety of analysis methods.
Mistake
VirusTotal is a tool for removing malware from your computer.
Correct
VirusTotal is an online scanning service that checks a file against many antivirus engines to see if it is known malicious. It does not remove malware or clean files. It only provides a detection report.
The name 'VirusTotal' sounds like it is a total virus removal tool. Beginners often confuse detection and removal, which is a frequent exam trap.
Mistake
A file hash is the same as a digital signature; both verify the file is legitimate.
Correct
A file hash is a unique fingerprint of the file's contents. It does not verify the file's publisher or trustworthiness. A digital signature is a cryptographic stamp that proves the file came from a specific publisher and has not been modified.
The words 'hash' and 'signature' are used loosely in everyday conversation, but in cybersecurity they have distinct meanings. The exam loves testing the difference between integrity (hash) and authenticity (digital signature).
Mistake
All malware analysis is done automatically by software; humans only review the results.
Correct
Automated sandboxes handle basic analysis, but skilled human analysts are needed for advanced static and dynamic analysis, especially for novel or heavily obfuscated malware. Humans interpret tool output, reverse engineer code, and write custom detection rules.
Movies often depict cybersecurity as fully automated, leading beginners to underestimate the human role. The exam covers both manual and automated techniques, and you need to know which technique requires a human (e.g., debugging).
Mistake
If a file passes a static hash check (its hash is not on a blacklist), then it is safe.
Correct
A clean hash check does not guarantee the file is safe. Malware can be slightly modified to change its hash (a technique called hash poisoning) and thus evade static detection. Dynamic analysis and other behavioural checks are then needed.
People think blacklists are comprehensive, but malware authors actively change their files to avoid known hashes. This is a common beginner trap that the exam tests through 'which analysis would catch this?' scenarios.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Static analysis examines a file without running it, checking its metadata, strings, and structure. Dynamic analysis runs the file in a controlled environment (sandbox) to observe its behaviour. Static is safe and quick; dynamic gives deeper insight but requires isolation.
A sandbox is an isolated virtual machine that prevents the malware from spreading to the real network. It allows the analyst to safely run the malware and watch what it does, without risking the actual systems.
A file hash is a unique string (like a fingerprint) generated from the contents of a file using an algorithm like SHA-256. It is important because it can be used to identify a specific file across different systems, and to check if a file matches known malware hashes.
No, VirusTotal is a scanning service. You upload a file or a hash, and it checks against multiple antivirus engines to see if the file is known malicious. It does not remove malware or clean files.
A packed executable is a program that has been compressed or encoded by a packer tool. This hides the original code and makes static analysis more difficult. Analysts often need to unpack the file before they can analyse its deeper behaviour.
IOCs are pieces of forensic data that indicate a system may have been compromised. Common IOCs include file hashes, IP addresses, domain names, registry keys, and filenames. They are used to detect if the same malware appears on other machines.
You've finished Malware Analysis Basics. Continue through the 200-201 study guide to build a complete picture of the exam.
Done with this chapter?