# Audit Devices and Logging

> Chapter 14 of the Courseiva HASHICORP-VAULT curriculum — https://courseiva.com/learn/hashicorp-vault/audit-devices

**Official objective:** 5.3 — Configure audit devices for monitoring and compliance

## Introduction

Audit devices and logging. If you don't know exactly what happened in your Vault — who accessed which secret, and when — you cannot prove compliance, detect a breach, or even troubleshoot a problem. For the VA-003 exam, you must understand how to turn on logging, where logs go, and what happens when the log storage fills up — because these are the exact scenarios the exam loves to test.

## The Hidden Camera Analogy

Ever borrowed a company laptop and wondered, 'Could someone see what I'm doing right now?' That feeling is the heart of audit logging. Let's map it.

Imagine you run a busy coffee shop. You trust your baristas, but you've had a few unexplained spills and a suspicious shortage of milk. You decide to install a hidden camera — not above the till to watch people, but one pointed at the inventory shelves. This camera records every time a milk carton is moved, by whom, and when. It doesn't stop theft itself, but if someone claims 'I only took one carton,' you can rewind the tape and see they actually took four. That tape is your audit device.

Now imagine the coffee shop is a Vault server. The hidden camera is your audit device — it logs every single action: who requested a secret, when, and whether it succeeded. Just as you wouldn't want the camera to stop recording during a rush, Vault can be configured to keep logging even if the whole system is under heavy load. And if the tape gets full? Vault can either stop writing new secrets (fail-close) or keep operating silently (fail-open) depending on how you set it up. The camera analogy works because both solve the same problem: creating an unchangeable, chronological record of who did what, so you can later prove or disprove a claim of wrongdoing. The key difference? Vault's logs are encrypted and can be sent to multiple 'recording studios' (called audit backends) at once.

## Core explanation

Audit devices in HashiCorp Vault are like the black box on an aeroplane. They record every single request made to the Vault server and every single response it sends back. This record is called an audit log. Why do you need this? Imagine you work for a bank. A regulator demands proof that no unauthorised person accessed customer credit card numbers stored in Vault. Without audit logs, you have no answer. With them, you can produce a detailed, tamper-proof history of every access.

Every audit log entry contains several key pieces of information. First, the timestamp — exactly when the request happened. Second, the client IP address and the authenticated user or machine identity (the entity making the request). Third, the request itself — what secret path was accessed, what operation (read, write, delete) was performed. Fourth, the response — what Vault sent back (but note: Vault can be configured to obfuscate or hash the actual secret values so the log doesn't contain plaintext passwords). Fifth, the authentication method used (token, LDAP, Okta, etc.). Sixth, whether the request succeeded or failed, and if it failed, why.

Vault supports multiple audit backends. A backend is just a destination where logs are sent. The most common are the file backend (writes logs to a file on disk), the syslog backend (sends logs to the system's syslog service, which is a standard Unix logging service), and the socket backend (sends logs over a network connection to a remote server — often used for sending logs to a central log management tool like Splunk or Elasticsearch). You can enable more than one backend at the same time. For example, you might send logs to a local file for quick troubleshooting and to a remote socket for long-term archiving in a security information and event management (SIEM) system.

Here is the critical concept the exam tests heavily: what happens when an audit device cannot write its log? This could happen if the disk is full, the network is down, or the remote server is unreachable. Vault has a setting called the 'blocked audit logging mode' or 'fail behaviour'. There are two modes: 

- Fail-close (also called blocking mode): If the audit device cannot write the log, Vault stops processing any further requests. This is the safest for compliance because it guarantees no action goes unlogged. But it means your entire Vault service goes down if the log disk fills up.
- Fail-open (also called non-blocking mode): If the audit device cannot write the log, Vault logs the error but continues processing requests. This keeps your applications running, but you lose audit history for those moments.

The exam expects you to know that by default, audit devices are in blocking mode (fail-close). You can change this using the 'blocked' parameter when enabling the audit device.

Another critical detail: audit logs are not human-readable by default. They are written in JSON format, which is a structured text format that computers find easy to parse. However, Vault provides a command called 'vault audit list' to see which audit devices are enabled, and you can use standard tools like 'jq' (a command-line JSON processor) to read and filter the logs. The exam may ask you to identify the correct command to enable an audit device ('vault audit enable file file_path=/vault/logs/audit.log') or to disable one ('vault audit disable file').

Finally, Vault has a feature called 'audit log entry sanitisation'. By default, Vault will hash (a one-way mathematical process) the values of secrets in audit logs so that the log doesn't contain the actual secret. You can control this with the 'hmac_accessor' option. For the exam, remember that Vault always tries to protect secrets from appearing in logs, but you should verify this behaviour if you are using custom audit devices.

## Real-world context

Let's say you are a security engineer at 'FinSecure', a financial services company that uses Vault to store database passwords, API keys, and TLS certificates. The compliance team comes to you with a new regulation: Sarbanes-Oxley (SOX) requires that all access to sensitive financial data be logged and that logs be retained for seven years. You need to set up audit devices properly.

Here is what you actually do, step by step:

First, you decide where to store logs. You want a local file for recent logs (easy to grep) and a remote syslog server for long-term archiving. You run the command: 'vault audit enable file file_path=/var/log/vault/audit.log'. This creates an audit device that writes each request as a line of JSON to that file. Next, you enable the syslog backend: 'vault audit enable syslog tag=vault facility=AUTH'. This sends logs to the system's syslog daemon, which forwards them to a centralised log server.

Now you test it. You log in to Vault and read a secret: 'vault kv get secret/db_password'. You then check the audit log file: 'tail -f /var/log/vault/audit.log'. You see an entry containing your source IP, the path 'secret/data/db_password', the operation 'read', and a hashed version of the password. You confirm the timestamp matches your action.

A month later, the disk fills up. Because the file backend is in blocking mode by default, Vault stops serving all requests. The CFO cannot access the production database password. The application goes down. This is where you learn the painful trade-off: you could have enabled non-blocking mode when you created the device ('vault audit enable file file_path=/vault/logs/audit.log blocked=false'), which would keep the app running but create a gap in your audit logs. The compliance team prefers the outage to a log gap, so you stick with blocking mode and set up a disk space alert.

Finally, during an audit, the compliance officer asks for logs from six months ago. You use the syslog server to retrieve the archived logs. You verify their integrity by checking that the timestamps are sequential and that no logs were tampered with after they were written (Vault logs are append-only). This real-world scenario shows exactly why the exam tests the difference between blocking and non-blocking modes — because in production, this choice can cause an outage or a compliance failure.

## Exam focus

The VA-003 exam focuses on three big topics within audit devices: enabling and disabling devices, understanding blocking vs non-blocking behaviour, and knowing which audit backends exist. Here is exactly what to expect.

First, exam questions will test your ability to enable an audit device using the correct CLI syntax. The pattern is always: 'vault audit enable <type> <options>'. For example, 'vault audit enable file file_path=/vault/logs/audit.log'. The exam will not ask you to remember every possible option, but you must know the basic structure and that 'file_path' is required for the file backend. Traps include: 'vault audit enable mylog.log' (missing the file keyword) or using 'path' instead of 'file_path'.

Second, the exam loves the blocking vs non-blocking question. They will present a scenario: 'An administrator configures an audit device and the disk becomes full. What happens to Vault operations?' The correct answer depends on whether the device was configured with 'blocked=true' (default) or 'blocked=false'. They might also ask: 'How can you change the behaviour so Vault continues to serve requests even if the audit log cannot be written?' Answer: set 'blocked=false' when enabling the device.

Third, you need to know the three supported audit backends: file, syslog, and socket. They may ask which one is best for sending logs to a remote Splunk server. The answer is socket, because it sends over TCP or UDP to a specified address. File is local only. Syslog is local to the system's syslog daemon (though it can be forwarded). They might also test that Vault supports enabling multiple audit devices simultaneously — true.

Fourth, a tricky question: 'Can audit logs contain plaintext secrets?' The answer is: by default, Vault hashes the secret values in audit logs. However, you can disable this with 'hmac_accessor=false' for specific audit devices. But the exam expects you to know the default is to hash them.

Fifth, the exam may ask about the 'vault audit list' command to see enabled devices, and 'vault audit disable <name>' to remove one. The name is the same as the type (e.g., 'file') unless you provided a custom path.

Finally, a frequent trap: they ask 'What happens if you try to enable a second audit device of the same type without a custom path?' The answer is: it fails, because each device must have a unique path. You can have two file backends if you give them different paths (e.g., one for critical logs, one for debug).

Key definitions to memorise:
- Audit backend: the destination where logs are sent (file, syslog, socket)
- Blocking mode: Vault stops if it cannot write a log
- Non-blocking mode: Vault logs an error but continues
- HMAC: the algorithm Vault uses to hash secret values in logs
- Default behaviour: blocking mode enabled, HMAC enabled

## Step by step

1. **Enable an audit device** — You run 'vault audit enable file file_path=/var/log/vault/audit.log'. This tells Vault to start logging all requests to that file. If the file already exists, Vault appends to it.
2. **Verify the device is active** — Run 'vault audit list' to see all enabled audit devices. The output will show the path, type, and description. This confirms your configuration was accepted.
3. **Generate test traffic** — Perform an action like 'vault kv get secret/test'. This creates an audit log entry with your request and response details, including timestamp, client IP, and operation type.
4. **Inspect the log** — Use 'tail /var/log/vault/audit.log' to view the latest entries. Each line is a JSON object. You can use 'jq .' to pretty-print the JSON and verify fields like 'path', 'operation', and 'client_ip'.
5. **Simulate a disk-full scenario** — Fill the disk (or use a test quota) and attempt another Vault request. In blocking mode, the request fails. This demonstrates why monitoring disk space is critical for production.
6. **Disable the audit device** — Run 'vault audit disable file'. This stops logging to that device but does not delete existing logs. You can re-enable it later with the same or different path.

## Comparisons

### File Backend vs Socket Backend

**File Backend:**
- Writes logs to a local file on the Vault server's disk
- Useful for quick local inspection with tools like tail and grep
- Data is static and stays on the server until rotated or moved

**Socket Backend:**
- Sends logs over a network (TCP or UDP) to a remote server
- Useful for centralised logging with SIEM tools like Splunk
- Data is streamed in real-time; no local persistence unless the remote server stores it

### Blocking Mode (fail-close) vs Non-blocking Mode (fail-open)

**Blocking Mode (fail-close):**
- Vault stops processing requests if it cannot write a log
- Guarantees no action goes unlogged
- Risk of complete service outage if log destination fails

**Non-blocking Mode (fail-open):**
- Vault continues processing requests if it cannot write a log
- Ensures application uptime
- Risk of gaps in audit history

### Hashed (default) Audit Logs vs Plaintext Audit Logs

**Hashed (default) Audit Logs:**
- Secret values are hashed using HMAC, so they appear as alphanumeric strings
- More secure because an attacker who reads the log cannot see secrets
- Cannot be used to directly recover the secret from the log

**Plaintext Audit Logs:**
- Secret values appear exactly as stored in Vault
- Less secure; any access to the log reveals secrets
- Useful for debugging but strongly discouraged in production

## Diagram

_Flowchart showing how a user request is handled by Vault depending on audit device status and blocking mode configuration._

```mermaid
flowchart TD
    A[User Request] --> B[Vault Server]
    B --> C{Audit Device Enabled?}
    C -- Yes --> D[Write to Audit Log]
    D --> E{Log Write Successful?}
    E -- Yes --> F[Process Request]
    E -- No --> G{Blocking Mode?}
    G -- Yes --> H[Request Rejected with Error]
    G -- No --> I[Log Error, Process Request]
    C -- No --> J[Process Request without Audit]
    F --> K[Return Response to User]
    H --> K
    I --> K
    J --> K
```

## Common misconceptions

- **Misconception:** Audit logs are only for security breaches and are optional for normal operations. **Reality:** Audit logs are mandatory for compliance and troubleshooting, not just security. Most production environments require them for SOX, PCI-DSS, or HIPAA. (Beginners think 'audit' means 'only for investigations', but in reality, logs are needed for debugging failed requests, capacity planning, and proving safe operation.)
- **Misconception:** If an audit device fails, Vault automatically switches to another device without manual intervention. **Reality:** Vault does not automatically failover between audit devices. Each device must be explicitly enabled, and if one fails in blocking mode, Vault stops processing requests entirely. (Newcomers expect high-availability behaviour that Vault does not provide for audit devices. They confuse Vault's replication features with audit device redundancy.)
- **Misconception:** You can view audit logs directly from the Vault CLI with the 'vault read' command. **Reality:** There is no 'vault read' command for audit logs. You must use standard OS tools (cat, tail, grep) for file backends, or query the remote system for socket backends. (Because Vault is a secrets engine, beginners naturally assume all data is accessed through its API. But audit logs are just files or streams, not stored within Vault itself.)
- **Misconception:** Disabling an audit device deletes all previously collected logs. **Reality:** Disabling an audit device only stops future logging. It does not delete any existing log files. The logs remain wherever they were written (disk, syslog server, etc.). (Users conflate 'disable' with 'delete'. In Vault, 'disable' only removes the configuration; the log files are managed separately by the OS or SIEM.)

## Key takeaways

- Audit devices record every request and response in Vault, creating an immutable chain of evidence for compliance and security.
- Vault supports three built-in audit backends: file, syslog, and socket.
- By default, audit devices are in blocking mode (fail-close), meaning Vault stops processing all requests if it cannot write a log.
- You can change blocking behaviour to non-blocking mode by setting 'blocked=false' when enabling the audit device.
- Secret values in audit logs are hashed by default using HMAC to prevent plaintext leakage.
- You can enable multiple audit devices simultaneously, each with a unique path, to send logs to different destinations.
- The CLI command to enable an audit device is 'vault audit enable <type> <options>'.
- If an audit device fails to write a log in blocking mode, Vault returns an error to the client and does not fulfil the request.

## FAQ

**What happens if the disk where audit logs are stored fills up?**

By default, Vault enters blocking mode and stops processing all requests until disk space is freed. You can change this by setting 'blocked=false' when enabling the device.

**Can I send audit logs to a remote server like Splunk?**

Yes, use the socket backend: 'vault audit enable socket address=192.168.1.100:514 socket_type=tcp'. This sends logs over TCP to a remote server.

**Do audit logs contain the actual secret values, like passwords?**

By default, Vault hashes the secret values using HMAC so they do not appear in plaintext. You can disable this with 'hmac_accessor=false' if needed.

**How do I see a list of all enabled audit devices?**

Use the command 'vault audit list'. It shows each device's path, type, description, and options.

**Can I have two audit devices of the same type, like two file backends?**

Yes, but each must have a unique path. For example: 'vault audit enable file file_path=/logs/audit1.log' and 'vault audit enable file file_path=/logs/audit2.log path=file2'.

**Will disabling an audit device delete my existing log files?**

No. Disabling only stops future logging. Existing log files remain on disk and must be deleted manually if no longer needed.

## Check your understanding

1. **What is the default behaviour of an audit device if it cannot write a log entry?**

   Answer: By default, the audit device is in blocking mode (fail-close), so Vault stops processing all requests until the log can be written.

2. **Which CLI command enables a file-based audit device with the log path /var/log/vault/audit.log?**

   Answer: The command is: 'vault audit enable file file_path=/var/log/vault/audit.log'.

3. **What happens to existing audit log files when you disable an audit device with 'vault audit disable file'?**

   Answer: Nothing. The log files remain on disk; only future logging is stopped.

4. **Name the three built-in audit backends supported by Vault.**

   Answer: The three backends are file, syslog, and socket.

5. **How does Vault protect secret values from appearing in plaintext in audit logs?**

   Answer: Vault hashes secret values using HMAC (a cryptographic hash) by default, so the actual secret is not visible in the log.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/hashicorp-vault/audit-devices