# Advanced Nmap: Scripting Engine (NSE)

> Chapter 45 of the Courseiva PENTEST-PLUS curriculum — https://courseiva.com/learn/pentest-plus/nmap-advanced-pt

**Official objective:** 2.2 — Recon Enumeration

## Introduction

One of the most powerful features for penetration testers, the Nmap Scripting Engine (NSE), is also a key exam topic under Recon Enumeration (Objective 2.2). You will learn how NSE works internally, how to use it effectively, and how to write custom scripts. Expect roughly 10-15% of exam questions to touch on NSE, often in scenarios requiring service enumeration, vulnerability detection, or custom automation. Mastery of NSE separates a basic Nmap user from an advanced penetration tester.

## NSE as a Programmable Swiss Army Knife

Picture a Swiss Army knife whose pre-installed tools — blades, screwdrivers, scissors — are ready for immediate use. But the knife also has an expansion slot where you can insert custom modules, each designed for a specific task like opening a particular type of lock or testing a specific material. Each module is a script that contains step-by-step instructions: first apply pressure here, then rotate 90 degrees, then listen for a click. The knife's handle has a sensor that reads the environment (like a port scan) and automatically selects the right module based on what it detects (e.g., if it sees a screw head, it picks the screwdriver module). If you want a new task, you write a new module and slide it into the slot. The knife doesn't just use the tools blindly — it can combine them, run them in sequence, and report back what it found. In the same way, NSE scripts are modular, event-driven, and can be chained. The NSE engine is the handle that provides common services (like socket handling, HTTP parsing, SSL negotiation) so each script doesn't need to reinvent the wheel. Just as a custom lock-picking module might fail if the lock is a different brand, an NSE script might fail if the target service doesn't match the expected banner — but the engine catches that gracefully and moves on.

## Core explanation

### What is NSE and Why Does It Exist?

The Nmap Scripting Engine (NSE) is a powerful extension that allows users to write and run scripts to automate a wide variety of networking tasks. Before NSE, Nmap was primarily a port scanner and OS/fingerprinting tool. With NSE, it becomes a full-fledged vulnerability scanner, service enumerator, and even a brute-forcing tool. The key motivation: penetration testers need to not only find open ports but also determine what services are running, their versions, and whether they have known vulnerabilities. NSE scripts can perform banner grabbing, SSL/TLS testing, brute-force attacks on authentication, and even exploit certain vulnerabilities.

### How NSE Works Internally

NSE is built on the Lua programming language (version 5.3 in modern Nmap). Each script is a .nse file that contains a set of functions and rules. The NSE engine (the Nmap executable) loads scripts, parses their metadata (categories, dependencies, port rules), and executes them in parallel using a cooperative multitasking scheduler. The scheduler uses Lua coroutines — each script runs as a coroutine that can yield control back to the engine when waiting for network I/O. This allows hundreds of scripts to run concurrently without the overhead of threads or processes.

When Nmap starts with NSE enabled (e.g., via `-sC` or `--script`), it first performs the normal Nmap phases: target enumeration, host discovery, port scanning, and service/version detection. After that, it enters the script phase. The engine checks each script's `prerule` (a function that determines if the script should run before any scan), `hostrule` (runs once per host after scanning), `portrule` (runs per open port), or `postrule` (runs after all scans). Most scripts use `portrule` to target specific services (e.g., HTTP on port 80, SSH on port 22). The engine evaluates these rules and schedules scripts for execution.

### Key Components, Values, and Defaults

- **Script Categories**: NSE scripts are organized into categories like `safe`, `default`, `discovery`, `auth`, `brute`, `intrusive`, `exploit`, `dos`, `fuzzer`, `malware`, `vuln`, and `external`. The `-sC` flag runs all scripts in the `default` category. The `safe` category includes scripts that are unlikely to crash services or cause disruption. `intrusive` scripts may be noisy or disruptive. `vuln` scripts check for specific vulnerabilities.
- **Script Arguments**: Many scripts accept arguments via the `--script-args` option. For example, `--script-args=userdb=users.txt,passdb=pass.txt` for brute-force scripts.
- **Script Database**: Nmap ships with over 600 scripts. The script database file is `scripts/script.db` (a plain text file listing script names and their categories). You can update it with `nmap --script-updatedb` which downloads the latest scripts from the Nmap repository.
- **Default Scripts**: The `default` category includes scripts that are considered safe and provide useful information. Running `-sC` is equivalent to `--script=default`.
- **Script Execution Order**: Scripts are executed in parallel, but dependencies can be defined using the `dependencies` field in the script metadata. For example, a script that needs HTTP title extraction may depend on `http-title`.
- **Timing and Performance**: NSE scripts are subject to Nmap's timing templates (`-T0` to `-T5`). More aggressive timing reduces delays between script probes. The `--script-timeout` option sets a maximum time (in seconds) per script.

### Configuration and Verification Commands

- **List all available scripts**: `ls /usr/share/nmap/scripts/` (or `nmap --script-help` to get descriptions).
- **Run default scripts**: `nmap -sC target` or `nmap --script=default target`.
- **Run specific scripts**: `nmap --script=http-title,ssl-heartbleed target`.
- **Run scripts by category**: `nmap --script=vuln target`.
- **Run all scripts except intrusive**: `nmap --script "not intrusive" target`.
- **Provide script arguments**: `nmap --script=http-brute --script-args=userdb=users.txt,passdb=pass.txt target`.
- **Update script database**: `nmap --script-updatedb`.
- **Write custom scripts**: Scripts are stored in `~/.nmap/scripts/` or `/usr/share/nmap/scripts/`. After adding a script, run `nmap --script-updatedb` to register it.

### How NSE Interacts with Related Technologies

- **Nmap Service Detection**: NSE scripts often rely on Nmap's service detection (`-sV`) to determine which service is running on a port. The script's `portrule` can check the service name (e.g., `port.service == "http"`).
- **Nmap OS Detection**: Some scripts use OS detection results (`-O`) to tailor their behavior (e.g., Windows-specific checks).
- **SSL/TLS**: The NSE library `tls.lua` provides functions for SSL/TLS handshakes, certificate parsing, and cipher suite testing. Scripts like `ssl-enum-ciphers` and `ssl-heartbleed` leverage this.
- **HTTP**: The `http.lua` library handles HTTP requests, cookie management, and response parsing. Many web application scripts depend on it.
- **Brute-Force**: NSE includes a `brute` library that provides a framework for writing brute-force scripts with threading, delay, and retry logic.

### Writing Custom NSE Scripts

A basic NSE script has the following structure:

```lua
local nmap = require "nmap"
local shortport = require "shortport"
local stdnse = require "stdnse"

-- Rule to run only on open port 80
portrule = shortport.portnumber(80, "tcp")

-- Action function
action = function(host, port)
    local socket = nmap.new_socket()
    socket:connect(host, port)
    socket:send("GET / HTTP/1.0\r\n\r\n")
    local response = socket:receive_lines(1)
    socket:close()
    return response
end
```

This script connects to port 80, sends an HTTP GET request, and returns the first line of the response. For the exam, you should understand the structure: `description` (metadata), `author`, `license`, `categories`, `portrule`, and `action`.

### Advanced Scripting Concepts

- **Dependencies**: A script can declare dependencies on other scripts. For example, `http-title` depends on `http-headers` to ensure headers are fetched first.
- **Running Scripts in Parallel**: NSE's coroutine-based model means scripts run concurrently. You can control parallelism with `--max-scan-delay` and `--min-rate`.
- **Script Output**: Scripts return strings or tables. The NSE engine formats output into the standard Nmap output (normal, XML, grepable). Use `stdnse.format_output` for structured output.
- **Error Handling**: Use `nmap.registry` to share data between scripts. Use `stdnse.debug` for debugging.

### Common NSE Scripts for PenTesting

- **http-title**: Grabs the title of a web page. Useful for identifying web applications.
- **http-enum**: Enumerates directories and files on a web server using a wordlist.
- **ssl-heartbleed**: Checks for the Heartbleed vulnerability (CVE-2014-0160).
- **smb-enum-shares**: Enumerates SMB shares on Windows systems.
- **ftp-anon**: Checks if FTP anonymous login is allowed.
- **mysql-empty-password**: Checks for MySQL accounts with empty passwords.
- **dns-brute**: Brute-forces DNS subdomains.
- **vuln**: Category including many vulnerability checks like `ms17-010` (EternalBlue).

### NSE Performance Considerations

Running many scripts can be slow. Use `--script-timeout` to limit each script's runtime (default is 10 minutes). The `--script-trace` option shows detailed script execution (useful for debugging). For large scans, consider running only `safe` scripts first, then `vuln` on identified services.

### NSE and the Exam

The PT0-002 exam expects you to know:
- How to run NSE scripts (`-sC`, `--script`).
- Common script categories and their safety levels.
- How to provide script arguments.
- How to update the script database.
- How to write a basic custom script (structure: description, portrule, action).
- The difference between `safe` and `intrusive` scripts.
- How to use NSE for service enumeration and vulnerability detection.

Trap: Candidates often confuse `-sC` (default scripts) with `-sV` (version detection). Remember: `-sC` is for scripts, `-sV` is for version detection. Another trap: using `--script=all` runs every script, including intrusive ones that may crash services or trigger alarms. In a pentest, you typically start with `safe` and escalate.

## Real-world context

### Enterprise Scenario 1: Internal Network Penetration Test

A pentester is engaged to assess an internal network with 500 hosts. The goal is to identify vulnerable services without causing disruption. The pentester starts with a ping sweep (`nmap -sn 10.0.0.0/16`) to find live hosts, then runs a TCP SYN scan on top 1000 ports (`nmap -sS -T4 --top-ports 1000 10.0.0.0/16 -oA initial`). After identifying open ports, the pentester runs NSE default scripts (`-sC`) on each host. This reveals that several Windows servers have SMB signing disabled (`smb-security-mode`), a web server is running Apache 2.4.49 (vulnerable to path traversal via `http-vuln-cve2021-41773`), and an FTP server allows anonymous login (`ftp-anon`). The pentester then runs `--script=vuln` on those specific hosts to confirm vulnerabilities. The NSE output is saved as XML and imported into a reporting tool. Scale: scanning 500 hosts with default scripts takes about 30 minutes with `-T4`. Performance consideration: using `--max-scan-delay 1000` prevents overwhelming the network. Misconfiguration: running `--script=all` would have included `dos` scripts that could crash services, alerting the blue team prematurely.

### Enterprise Scenario 2: External Web Application Assessment

A pentester targets a public-facing web application. After discovering open ports 80 and 443, they run NSE scripts against the web server: `nmap -p 80,443 --script=http-* target`. This runs all HTTP-related scripts, including `http-enum` (directory brute-force), `http-headers`, `http-methods`, `http-csrf`, `http-xssed`, and `http-sql-injection`. The `http-enum` script uses a wordlist to find hidden directories like /admin and /backup. The `http-methods` script checks for dangerous HTTP methods (e.g., PUT, DELETE). The `http-sql-injection` script attempts basic SQL injection payloads. The pentester finds that the server allows PUT on /uploads, allowing file upload. They then use `http-put` script (custom) to upload a test file. Common pitfall: running `http-sql-injection` on a login page may cause account lockouts due to failed login attempts. The pentester should first run `safe` scripts, then `intrusive` ones with caution.

### Enterprise Scenario 3: Cloud Environment (AWS)

A pentester is testing an AWS VPC. Since Nmap scans can be noisy and may trigger IDS/IPS, they use a stealthy approach: `nmap -sS -T2 --script=default --script-args=userdb=users.txt target`. They focus on a single instance to minimize logs. The `default` scripts reveal an open Redis port (6379) without authentication (`redis-info`). The pentester then runs `redis-brute` to guess the password. They also run `ssl-enum-ciphers` on port 443 to check for weak ciphers. In cloud environments, NSE scripts like `http-enum` can be used to enumerate S3 buckets if the web application leaks bucket names. Performance: scanning across the internet requires careful timing to avoid rate limiting. Misconfiguration: forgetting to exclude the Nmap scanner's own IP from the target list (e.g., scanning a subnet that includes the scanner's own host).

## Exam focus

The PT0-002 exam tests NSE under Objective 2.2 (Recon Enumeration) and also within vulnerability scanning (Objective 3.1). Expect 3-5 questions directly referencing NSE. Key areas:

1. **Script Categories**: Know the difference between `safe`, `default`, `intrusive`, `vuln`, `exploit`. The exam loves to ask: 'Which script category would you use to avoid disrupting services?' Answer: `safe`. Another: 'Which category includes scripts that may crash services?' Answer: `intrusive`.

2. **Command Syntax**: You must know `-sC` (default scripts), `--script=<script>`, `--script-args=<args>`, `--script-updatedb`. Trap: The exam may show `--script=default` and `-sC` as equivalent. Another trap: `-sV` is version detection, not scripts.

3. **Common Scripts**: Be familiar with `http-title`, `ssl-heartbleed`, `smb-enum-shares`, `ftp-anon`, `dns-brute`, `http-enum`. The exam may describe a scenario and ask which script to use. For example: 'You find an open SMB port. Which script enumerates shares?' Answer: `smb-enum-shares`.

4. **Custom Script Structure**: The exam may show a partial Lua script and ask what is missing or what the `portrule` does. Remember that `portrule` returns true/false to determine if the script runs on that port. The `action` function returns the output.

5. **Performance and Timing**: Know that `--script-timeout` limits script runtime (default 10 minutes). `-T4` increases speed but may cause dropped packets. The exam may ask: 'Which option prevents a script from hanging indefinitely?' Answer: `--script-timeout`.

6. **Common Wrong Answers**: 
   - Choosing `-sV` when the question asks for service enumeration via scripts. 
   - Thinking `-sC` runs all scripts. 
   - Believing that NSE scripts can only be run after a full port scan (they can be run with just a target, but they rely on port scan results). 
   - Confusing `--script` with `--script-args`. 

7. **Edge Cases**: 
   - If no open ports are found, most scripts won't run (except those with `prerule` or `hostrule`). 
   - Scripts can be run against hosts that are down if using `-Pn` (skip host discovery). 
   - The `external` category scripts may send data to third-party services (e.g., whois lookups). 

8. **How to Eliminate Wrong Answers**: Read the scenario carefully. If it mentions 'stealth' or 'avoid disruption', eliminate `intrusive` and `exploit` categories. If it mentions 'vulnerability check', look for `vuln` category or specific script names. If it mentions 'custom script', look for Lua code. If it mentions 'update', look for `--script-updatedb`. If it mentions 'arguments', look for `--script-args`.

## Step by step

1. **Identify Target and Port Scan** — Begin by running a standard Nmap scan to discover live hosts and open ports. Use `nmap -sn 192.168.1.0/24` for ping sweep, then `nmap -p- 192.168.1.10` for full port scan. This step is crucial because NSE scripts run after port discovery; without open ports, most scripts will not execute. The engine evaluates port rules only for ports marked as open or filtered. You can combine with `-sV` to detect service versions, which many scripts use to tailor their probes. For example, `nmap -sV -p 80,443 target` will detect HTTP/HTTPS and then NSE scripts like `http-title` will automatically target those ports.
2. **Run Default NSE Scripts** — Execute `nmap -sC target` to run all scripts in the `default` category. This includes safe scripts like `http-title`, `ssh-hostkey`, `ssl-cert`, and `banner`. The engine loads each script, evaluates its `portrule` against the scan results, and runs matching scripts concurrently. Output is interleaved in the console but structured in XML. On the exam, remember that `-sC` is equivalent to `--script=default`. Common mistake: thinking `-sC` runs all scripts — it only runs the `default` category. To run all scripts, you'd use `--script=all` (not recommended in production).
3. **Run Category-Specific Scripts** — Use `--script=vuln` to check for known vulnerabilities, or `--script=safe` for non-disruptive enumeration. For example, `nmap --script=vuln target` will run all scripts in the `vuln` category, such as `ssl-heartbleed`, `ms17-010`, and `http-shellshock`. The engine filters scripts by category metadata. You can combine categories: `--script=default,safe` or exclude: `--script "not intrusive"`. On the exam, know that `intrusive` scripts may cause service crashes or log alerts, so they are avoided in stealthy engagements.
4. **Provide Script Arguments** — Many scripts accept arguments to customize behavior. For instance, `http-brute` needs a username and password list: `nmap --script=http-brute --script-args=userdb=users.txt,passdb=pass.txt target`. Arguments are passed as key=value pairs, separated by commas. The engine parses these and makes them available to scripts via `nmap.registry.args`. On the exam, expect questions about how to pass arguments, especially for brute-force scripts. Trap: forgetting to quote paths with spaces, or using `--script-args` without the `=` sign (correct: `--script-args=userdb=users.txt`).
5. **Analyze Output and Iterate** — Review the NSE output to identify services, versions, and potential vulnerabilities. For example, if `http-title` returns "Welcome to Apache Tomcat", you might then run `tomcat-default-logins`. If a vulnerability is found (e.g., Heartbleed), you can proceed to exploitation. The output includes script names and their results. Use `-oX` to save XML for parsing. On the exam, you may be asked to interpret NSE output to determine the next step. For instance, if `smb-enum-shares` shows a writable share, the next step might be to upload a payload.

## Comparisons

### NSE (Nmap Scripting Engine) vs Manual Testing with Netcat/OpenSSL

**NSE (Nmap Scripting Engine):**
- Automates hundreds of tests with a single command.
- Built-in parallel execution via coroutines.
- Over 600 pre-written scripts for common tasks.
- Structured output (XML, grepable) for reporting.
- Can be extended with custom Lua scripts.

**Manual Testing with Netcat/OpenSSL:**
- Requires manual commands for each test (e.g., `nc -v target port`).
- Single-threaded; must script loops in bash.
- No pre-built library; must know exact probes.
- Output is raw text; parsing is manual.
- No built-in framework; reinventing the wheel.

## Common misconceptions

- **Misconception:** -sC runs all available NSE scripts. **Reality:** -sC only runs scripts in the `default` category, which is a curated set of safe and useful scripts. To run all scripts, you need `--script=all`, which includes intrusive and potentially disruptive scripts.
- **Misconception:** NSE scripts can only be run after a full port scan. **Reality:** NSE scripts can be run at any time, but they rely on Nmap's scan results. If you run `nmap --script=http-title target` without a port scan, Nmap will first perform a default port scan (top 1000 ports) to find open ports. You can also provide a list of ports with `-p` to limit scanning.
- **Misconception:** NSE scripts are written in Python. **Reality:** NSE scripts are written in Lua (version 5.3). Nmap embeds a Lua interpreter. Python is not used for NSE; however, Nmap can be controlled via Python using the python-nmap library, but that is not NSE.
- **Misconception:** Running `--script=default` is the same as `-sV`. **Reality:** `-sV` enables version detection, which probes services to determine exact version numbers. `-sC` (or `--script=default`) runs NSE scripts. They are separate features, though often used together (`-sCV`). The exam tests this distinction.
- **Misconception:** You cannot run a single script without running others. **Reality:** You can run a single script by specifying its name: `nmap --script=http-title target`. Only that script will execute, along with any scripts it depends on. Dependencies are automatically resolved.

## Key takeaways

- NSE scripts are written in Lua and stored in .nse files.
- The `-sC` flag runs all scripts in the `default` category.
- Categories include `safe`, `default`, `intrusive`, `vuln`, `exploit`, `auth`, `brute`, `dos`, `fuzzer`, `malware`, `discovery`, and `external`.
- Use `--script=<script-name>` to run a specific script.
- Script arguments are passed with `--script-args=key=value`.
- Update the script database with `nmap --script-updatedb`.
- The `portrule` function determines if a script runs on a given port.
- The `action` function returns the script's output.
- NSE scripts run concurrently using Lua coroutines.
- Default timeout per script is 10 minutes; use `--script-timeout` to change it.

## FAQ

**How do I run all NSE scripts in the 'vuln' category?**

Use `nmap --script=vuln target`. This runs every script categorized as `vuln`. You can combine categories with commas, e.g., `--script=vuln,safe`. To exclude a category, use `--script "not intrusive"`. Remember that `vuln` scripts are generally safe but may send exploit-like payloads that could trigger IDS.

**What is the difference between -sC and --script=default?**

There is no difference. `-sC` is a shorthand for `--script=default`. Both run the scripts in the `default` category. The exam uses both notations, so you must recognize them as equivalent. Never confuse `-sC` with `-sV` (version detection).

**How do I write a custom NSE script?**

Create a .nse file in `~/.nmap/scripts/` or `/usr/share/nmap/scripts/`. The script must define a `description` table (with `description`, `author`, `license`, `categories`), a `portrule` function (returns true/false), and an `action` function (returns output). Example: `portrule = shortport.portnumber(80, "tcp")` and `action = function(host, port) return "Hello" end`. Then run `nmap --script-updatedb` and use `--script=yourscript`.

**Can NSE scripts be used for brute-force attacks?**

Yes. NSE includes a `brute` library and many scripts like `http-brute`, `ssh-brute`, `mysql-brute`, `smb-brute`. Use `--script-args=userdb=users.txt,passdb=pass.txt` to provide wordlists. These scripts are in the `brute` category and are considered `intrusive` because they generate many login attempts. On the exam, know that brute-force scripts require argument files.

**What does --script-timeout do?**

It sets the maximum time (in seconds) that a single script can run before being terminated. Default is 600 seconds (10 minutes). Use `--script-timeout 30s` to limit to 30 seconds. This prevents a hanging script from delaying the entire scan. The exam may ask which option prevents indefinite hangs.

**How do I update the NSE script database?**

Run `nmap --script-updatedb`. This downloads the latest scripts from the Nmap repository and updates the `scripts/script.db` file. You need internet access. On the exam, this is the correct answer for 'how to get the latest scripts'.

**Why did my NSE script not run?**

Possible reasons: (1) The target has no open ports that match the script's `portrule`. (2) The script's category was excluded (e.g., you used `--script=safe` but the script is `intrusive`). (3) The script has unmet dependencies. (4) You forgot to run `--script-updatedb` after adding a custom script. Check with `--script-trace` for debugging.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/pentest-plus/nmap-advanced-pt