# Application Control and Intrusion Prevention System (IPS)

> Chapter 6 of the Courseiva FORTINET-NSE4 curriculum — https://courseiva.com/learn/fortinet-nse4/application-control-and-ips

**Official objective:** 4.1 — Configure application control profiles and IPS sensors to protect network traffic from threats.

## Introduction

Application control and Intrusion Prevention System (IPS) are two security features that stop unwanted software and malicious attacks before they can harm your network. For the NSE4 exam, you need to know how to configure these features to protect real business traffic. This chapter explains them in plain language so you can pass the exam and understand what you are doing.

## The Nightclub Bouncer and Security Camera Analogy

First, you decide to open a nightclub, so you need a plan to keep troublemakers out. That leads you to hire two specialised security systems: a bouncer at the door and a set of security cameras inside.

The bouncer checks every person who tries to enter. He has a list of banned individuals (like known malware IP addresses) and a list of acceptable types of people (like 'over 18s' or 'members only'). He doesn't care what they do inside; he just controls who gets in. This is your Application Control. In the network, it decides which applications (like Facebook, BitTorrent, or Skype) are allowed through the firewall. It stops unwanted apps at the door, even if they try to look like something harmless.

The security cameras inside the club watch what people are doing. If someone starts a fight or tries to break into a locked office, the cameras spot the suspicious behaviour and the security team rushes over to stop it. This is your Intrusion Prevention System (IPS). It doesn't just log who enters; it analyses the traffic flowing through the network for malicious activity — like someone trying to exploit a vulnerability in a web server — and actively blocks it in real time.

Together, the bouncer controls what's allowed in, and the cameras stop bad behaviour once it starts. In Fortinet terms, you configure an Application Control profile to decide which apps are permitted, and an IPS sensor to detect and block attacks within the allowed traffic.

## Core explanation

Application control and Intrusion Prevention System (IPS) are two separate but related features on a FortiGate firewall. They work together to protect your network from different types of threats.

First, let's define the key terms.

A 'firewall' is a device that sits between your internal network and the internet. It decides which data packets are allowed to enter or leave based on rules you set. Traditionally, firewalls only looked at IP addresses and port numbers. For example, they would allow web traffic on port 80 and 443. But modern threats hide inside allowed traffic, so we need deeper inspection.

'Application control' is a feature that identifies the specific application (like YouTube, Netflix, or WhatsApp) within the network traffic, not just the port. For instance, many instant messaging apps use port 443 (HTTPS), which is normally allowed for secure web browsing. Without application control, you cannot block WhatsApp or Facebook Messenger without also blocking all secure websites. Application control uses 'signatures' — patterns that uniquely identify each application — to recognise what is really inside the traffic.

An 'Intrusion Prevention System (IPS)' is a security feature that monitors network traffic for malicious patterns. It looks at the content of the data packets to find known attack signatures, such as a hacker trying to exploit a vulnerability in a web server or a virus spreading across the network. When IPS detects a match, it can block the traffic in real time, log the event, or alert the administrator. IPS is different from a simple antivirus because it examines the entire packet and understands the context of the attack.

How do these features work on a FortiGate?

You create two separate objects:
- An 'application control profile' — this is a list of rules that tell the FortiGate which applications to allow, block, or monitor (log without blocking). You can block categories like 'peer-to-peer file sharing' or specific apps like 'BitTorrent'.
- An 'IPS sensor' — this is a set of rules that tell the FortiGate which attack signatures to look for. You can enable predefined groups of signatures for common threats like SQL injection, buffer overflow, or cross-site scripting.

These objects are then attached to a 'security policy'. A security policy is a rule that says: 'traffic from this source (like the internal network) going to this destination (like the internet) must be inspected using this application control profile and this IPS sensor'. Without attaching them to a policy, they do nothing.

Why do you need both? Here is an example.

Imagine a user on your network tries to download a file using BitTorrent. The firewall sees traffic on port 443 (HTTPS) and normally allows it. But application control recognises the BitTorrent protocol and blocks the download. That stops the unwanted application.

Now imagine the same user visits a legitimate website, but that website has been hacked and contains a malicious script. The traffic is allowed because it is normal web browsing. The script tries to exploit a vulnerability in the user's browser. The IPS sensor detects the attack pattern in the traffic and blocks just that malicious packet, while allowing the rest of the website to load.

Application control and IPS can be configured with different 'actions':
- 'Allow' — the traffic passes through without inspection.
- 'Monitor' — the traffic is logged, but not blocked. This is useful for seeing what is happening without breaking anything.
- 'Block' — the traffic is dropped immediately.
- 'Quarantine' — in some FortiGate versions, you can quarantine the source IP for a set time.

IPS also has 'severity levels' like critical, high, medium, low, and info. You can configure the sensor to block only attacks above a certain severity, or log everything but block only critical ones.

In the NSE4 exam, you need to know how to navigate the FortiGate web interface to create these profiles and sensors, and how to attach them to a security policy. You will also be tested on the difference between application control and IPS, and when to use each one.

## Real-world context

Let's walk through a realistic business scenario.

You are the IT administrator for a medium-sized company with 200 employees. The company sells products online and has a customer database. Your boss asks you to secure the internet connection so that:
1. Employees cannot use peer-to-peer file sharing (like BitTorrent) because it wastes bandwidth and can introduce malware.
2. Your internal web server must be protected from known attacks, especially SQL injection attempts that could steal customer data.

Here is what you do step by step.

First, you log into the FortiGate web interface. You navigate to the Policy & Objects section and then to Application Control.

You click 'Application Control Profile' and then 'Create New'. You name it 'Block-P2P-Apps'. In the list of applications, you search for 'BitTorrent' and 'uTorrent'. You set their action to 'Block'. You also expand the 'Peer-to-Peer' category and block the entire category. You save this profile.

Next, you create an IPS sensor. Navigate to IPS Sensor and click 'Create New'. You name it 'Protect-Web-Server'. You add a new signature entry. In the signature selection, you choose 'Pre-defined' and then 'Server' and then 'Web Server'. You select the signature 'HTTP.SQL.Injection' and set its action to 'Block'. You also add 'HTTP.Cross-Site.Scripting' and block that. You save the sensor.

Now you need to create a security policy. Go to Policy & Objects > Firewall Policy. Create a new policy. Set the incoming interface to 'internal' (your LAN) and the outgoing interface to 'wan1' (the internet). For the source, select 'all' or a specific subnet. For the destination, select 'all'. In the Services section, select 'ALL' (or just HTTPS if you want to be stricter). Under the Security Profiles tab, enable 'Application Control' and select your 'Block-P2P-Apps' profile. Enable 'IPS' and select your 'Protect-Web-Server' sensor. Set the action to 'Accept' (this allows the traffic, but then applies the profiles). Click OK.

But wait — that policy only inspects traffic from internal users going to the internet. To protect your web server from incoming attacks, you need a separate policy for traffic coming from the internet to your web server.

Create another policy. Set incoming interface to 'wan1', outgoing interface to 'dmz' (where your web server sits). Source is 'all', destination is the IP address of your web server. Services is 'HTTPS'. Under Security Profiles, enable IPS and select the same 'Protect-Web-Server' sensor, but disable Application Control (you don't need to control apps on incoming traffic). Set action to 'Accept'. Save.

Now, test it. Try to access BitTorrent from an internal computer. The download should fail. Check the FortiGate logs under Log & Report > Application Control to see that it was blocked. Try to simulate a SQL injection attack against your web server (in a lab environment). The IPS should block it and log the event under Log & Report > Intrusion Prevention.

In the real world, you would also monitor the logs weekly to see if any applications need to be unblocked or if new attack patterns appear. You might adjust the IPS sensor to log lower-severity events for a few weeks to understand what normal traffic looks like before enabling blocking. This is called 'tuning'.

Common issues you might encounter:
- An application you need for work (like a customer relationship management tool) gets blocked by application control. You then create an exception to allow it.
- IPS blocks legitimate traffic because a signature is too broad. You might need to disable that specific signature or change its action to 'monitor' while you investigate.

In the NSE4 exam, you will not need to actually configure a live firewall, but you will need to know the steps and understand the logic behind each configuration.

## Exam focus

The NSE4 exam tests your understanding of application control and IPS in several specific ways. Knowing the exact traps will help you pass.

First, know that the exam loves to ask about the 'difference between application control and IPS'. They will give you a scenario and ask which feature to use. Remember: application control controls which apps are allowed; IPS detects malicious content within allowed traffic.

Second, the exam tests the 'order of operations' in a security policy. Traffic is inspected in this sequence: Firewall policy match -> SSL/SSH inspection -> Application Control -> IPS -> Web Filter -> DNS Filter -> Antivirus -> Data Leak Prevention. They may ask what happens if application control blocks traffic before IPS can inspect it. The answer is that IPS never sees the blocked traffic.

Third, be ready for questions about 'signature matching'. Both application control and IPS use signatures, but they are different types. Application control signatures identify applications by their behaviour (e.g., the way a chat app sends 'keep alive' packets). IPS signatures identify attack patterns (e.g., a specific byte sequence in a packet that matches a known exploit). The exam may give you a vague signature and ask whether it belongs to application control or IPS.

Fourth, the exam tests 'actions'. You must know the difference between 'monitor' (log only, no block) and 'block'. They often set up a trap: 'You want to see what applications users are running without blocking anything. What action should you use?' The answer is 'monitor', but many students guess 'allow' by mistake. 'Allow' skips inspection entirely; 'monitor' logs and inspects but does not block.

Fifth, the exam covers 'IPS severity levels'. You will be asked: 'A signature has severity level Critical. You want to block only Critical and High threats. Which setting do you use?' The answer is to set the 'action' to 'Block' and set the 'severity threshold' or use a filter for those levels. There is a specific field in the IPS sensor called 'Severity' where you can choose 'Critical', 'High', 'Medium', 'Low', 'Info'.

Key topics to memorise:
- Application control can identify applications even if they use non-standard ports.
- IPS uses both 'pre-defined signatures' (Fortinet updates them) and 'custom signatures' (you write them).
- Both features are 'licensed' — you need a FortiGuard IPS subscription. Without a valid license, the IPS database is outdated (last updated when the device was shipped).
- Application control does not require a subscription? Actually, it does require a FortiGuard App Control subscription to get the latest signatures. But the device ships with a baseline set. The exam may trick you by saying 'Application control works without a license' — but for full functionality, it needs a subscription.
- The 'IPS Sensor' can be configured to 'block' or 'monitor' per signature. You can also set a 'default action' for all signatures.
- 'Application Control' profiles can be set to 'Block', 'Monitor', or 'Allow' per application.
- Both features generate logs that you can view in the FortiGate Log & Report section.
- The exam may ask about 'performance impact'. Both features require CPU and memory, so too many rules can slow the firewall.

Common question traps:
- They ask: 'You want to block all social media applications on your network. Which feature do you use?' Answer: Application Control (not web filter, because social media apps might use HTTPS and web filter only looks at URLs).
- They ask: 'You want to prevent a worm from spreading through your internal network. Which feature?' Answer: IPS (because a worm is malicious content, not an application category).
- They ask: 'Can application control block an attack?' No — it blocks applications, not attacks. That is IPS's job.

Memorise these definitions exactly:
- Application Control: 'identifies and controls application usage, regardless of port or protocol.'
- IPS: 'inspects traffic for malicious patterns and blocks attacks in real time.'

In the lab section of the exam (if you take the practical), you will be asked to create a policy that includes an application control profile and an IPS sensor. You must know where to find these objects in the GUI and how to attach them to a policy. Practice this step on a FortiGate virtual machine if possible.

## Step by step

1. **Create an Application Control Profile** — Log into the FortiGate GUI. Go to Policy & Objects > Application Control. Click 'Create New'. Name your profile (e.g., 'Block-Social-Media'). Search for applications like 'Facebook' or 'Twitter' and set their action to 'Block'. You can also block entire categories like 'Social.Media'. Save the profile.
2. **Create an IPS Sensor** — Navigate to IPS Sensor (under Security Profiles). Click 'Create New'. Name it (e.g., 'Protect-Web-Server'). Add a new signature entry. Choose 'Pre-defined' and then a category like 'Server' > 'Web Server'. Select specific signatures like 'HTTP.SQL.Injection' and set their action to 'Block'. You can also set a default action for unlisted signatures to 'Monitor'. Save the sensor.
3. **Attach Profiles to a Security Policy** — Go to Policy & Objects > Firewall Policy. Create a new policy or edit an existing one. Under the 'Security Profiles' tab, enable 'Application Control' and select your profile. Enable 'IPS' and select your sensor. Ensure the policy's action is set to 'ACCEPT' (so traffic flows, but gets inspected). Save the policy.
4. **Enable SSL/SSH Inspection (Optional but Recommended)** — To inspect HTTPS traffic, go to Policy & Objects > SSL/SSH Inspection. Create a profile (e.g., 'Deep-Inspection'). Select 'Deep Inspection' (or 'Certificate Inspection' for less thorough inspection). Attach this profile to the same security policy under the 'SSL/SSH Inspection' dropdown. Without this, application control and IPS can only see metadata, not the encrypted content.
5. **Verify and Monitor Logs** — After configuration, check the logs. Go to Log & Report > Application Control to see which applications were blocked or monitored. Go to Log & Report > Intrusion Prevention to see any attacks detected. Use this to tune your profiles — for example, if a legitimate application is blocked, add an exception to allow it.

## Comparisons

### Application Control vs Intrusion Prevention System (IPS)

**Application Control:**
- Controls which applications are allowed (e.g., block Facebook).
- Uses application signatures to identify apps by behaviour.
- Action options: Allow, Monitor, Block per application.

**Intrusion Prevention System (IPS):**
- Detects and blocks malicious content within allowed traffic.
- Uses IPS signatures to identify attack patterns (e.g., SQL injection).
- Action options: Monitor, Block, Quarantine per signature or severity.

### Monitor Action (App Control) vs Allow Action (App Control)

**Monitor Action (App Control):**
- Traffic is inspected and logged, but not blocked.
- Useful for 'learning mode' to see what apps are used.
- Generates logs in Application Control log.
- Traffic still reaches its destination.

**Allow Action (App Control):**
- Traffic is not inspected at all by application control.
- Useful for trusted applications you never want to block.
- No logs generated for the allowed app.
- Traffic passes through as if the profile were not attached.

### Pre-defined IPS Signatures vs Custom IPS Signatures

**Pre-defined IPS Signatures:**
- Provided and updated by Fortinet via FortiGuard.
- Cover known vulnerabilities and exploits.
- Cannot be edited, only enabled or disabled.

**Custom IPS Signatures:**
- Written by the administrator for specific needs.
- Used for proprietary applications or internal threats.
- Fully customisable using FortiGate IPS signature syntax.

## Diagram

_This flowchart shows the order in which a FortiGate inspects traffic: first it matches a security policy, then optionally applies SSL inspection, then application control decides if the app is allowed, and finally IPS checks for malicious content._

```mermaid
flowchart TD
    A[User Traffic] --> B[Security Policy Match]
    B --> C[SSL/SSH Inspection]
    C --> D[Application Control Profile]
    D --> E{Allowed?}
    E -- Yes --> F[IPS Sensor]
    E -- No --> G[Blocked Traffic]
    F --> H{Attack Detected?}
    H -- No --> I[Traffic Allowed]
    H -- Yes --> J[Blocked/Logged Attack]
    G --> K[Log Event]
    J --> K
```

## Common misconceptions

- **Misconception:** Application control can block malware, so I don't need IPS. **Reality:** Application control blocks applications (like BitTorrent), not malware. IPS is needed to detect and block malicious content inside allowed applications, like a virus in a PDF attachment. (New learners think that blocking the 'container' (the app) is enough. But malware can ride inside legitimate apps like email or web browsing.)
- **Misconception:** If I allow HTTPS traffic, the firewall cannot inspect it, so application control and IPS do not work on it. **Reality:** With SSL/SSH inspection (deep packet inspection), the FortiGate can decrypt HTTPS traffic, then apply application control and IPS. Without decryption, they can still inspect metadata, but not the encrypted payload. (People know HTTPS is encrypted and assume no inspection is possible. FortiGate can do explicit or full proxy decryption to see inside.)
- **Misconception:** IPS and antivirus are the same thing. **Reality:** IPS inspects network traffic for attack patterns (like SQL injection). Antivirus inspects files for known malware signatures (like a virus in a document). They are different features with different signature databases. (Both are security features that block 'bad things', so beginners lump them together. The exam tests their distinct purposes.)
- **Misconception:** Once I create an application control profile, it automatically applies to all traffic. **Reality:** An application control profile does nothing until it is attached to a security policy. The policy defines which traffic is inspected. (Many assume that configuring a profile in the GUI activates it globally. But security policies are the gatekeepers.)
- **Misconception:** IPS only works on internet traffic, not internal traffic. **Reality:** IPS can inspect traffic between any interfaces, including internal-to-internal traffic (e.g., between VLANs). You just need a security policy that covers that traffic. (Because IPS is often used for internet-facing servers, students think it is only for WAN-to-LAN traffic.)

## Key takeaways

- Application control identifies and controls specific applications (like YouTube or Skype) regardless of the port or protocol they use.
- IPS (Intrusion Prevention System) detects and blocks malicious patterns in network traffic, such as exploit attempts or worm propagation.
- Both features require a valid FortiGuard subscription to receive up-to-date signature databases.
- Application control and IPS profiles must be attached to a security policy to take effect; they do not work globally.
- Application control can block or monitor applications, but it cannot block a malicious attack — that is the job of IPS.
- IPS uses severity levels (Critical, High, Medium, Low, Info) to classify threats, and you can set different actions per level.
- SSL/SSH inspection must be enabled for application control and IPS to inspect the content of encrypted HTTPS traffic.
- The default action for an IPS signature is 'Monitor' (log only), so you must explicitly set it to 'Block' to prevent attacks.

## FAQ

**Do I need a license for application control and IPS on a FortiGate?**

Yes. Both features require a valid FortiGuard subscription (App Control and IPS) to receive updated signatures. Without a license, the device uses a base set that may be outdated and not effective.

**Can application control block traffic without IPS?**

Yes. Application control works independently. It can block applications like WhatsApp or Netflix without needing IPS. IPS adds another layer by inspecting the content of allowed traffic for attacks.

**What is the difference between 'Monitor' and 'Allow' in application control?**

'Monitor' logs the application usage but does not block the traffic. 'Allow' means the application is not inspected at all and passes through. 'Block' drops the traffic.

**Does IPS work on internal traffic between two devices on the same FortiGate?**

Yes. As long as there is a security policy covering that traffic (e.g., between two internal interfaces), you can attach an IPS sensor to inspect it.

**Will IPS block all attacks automatically once enabled?**

No. By default, IPS signatures are set to 'Monitor' (log only). You must explicitly set the action to 'Block' for the signatures you want to block. You can also set a default action in the sensor.

**Can I create custom IPS signatures for proprietary software?**

Yes. FortiGate allows you to create custom IPS signatures using a specific syntax. This is useful for protecting custom applications or detecting company-specific threats.

## Check your understanding

1. **What is the primary difference between application control and IPS?**

   Answer: Application controls which applications are allowed or blocked, while IPS detects and blocks malicious content inside allowed traffic.

2. **Do application control and IPS profiles work without being attached to a security policy?**

   Answer: No, they are only active when attached to a security policy that defines the traffic they inspect.

3. **What is the default action for an IPS signature in a newly created IPS sensor?**

   Answer: The default action is 'Monitor', which logs the event but does not block the traffic.

4. **If you want to block all peer-to-peer file sharing traffic, which feature do you configure?**

   Answer: You configure an Application Control profile and block the 'Peer-to-Peer' category or specific applications like BitTorrent.

5. **Can IPS inspect encrypted HTTPS traffic?**

   Answer: Yes, if SSL/SSH inspection is enabled. Without it, IPS can only see metadata, not the encrypted payload.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/fortinet-nse4/application-control-and-ips