What's the quickest way to fix a broken system when nobody knows why it broke? Troubleshooting is the methodical process of identifying, isolating, and resolving problems in operating systems and networks. For LPIC-2, mastering this process is critical because the exam directly tests your ability to apply systematic diagnostic commands and interpret error output to restore services.
Jump to a section
A simple way to picture Troubleshooting System and Network Issues
Have you ever turned on a tap and got only a dribble of water, or found the sink backing up? That's a live troubleshooting situation. A network or system is like your home plumbing. The water source is your ISP or data centre, the pipes are cables and network routes, the taps are your servers and ports, and the drain is your endpoint device. If water (data) doesn't flow, you need to isolate where the blockage is.
Start at the farthest point — the sink. Is the tap (your computer) open? In IT, that means checking if the service is running and listening. Next, look at the pipe under the sink: are there any leaks? That's like a faulty cable or a misconfigured switch. Then go up to the main line: does the building have water? That is the ISP uplink. Finally, check the water source — is the utility delivering pressure? That's your internet connection. Each step eliminates one part of the chain. Every time you fix a small drip, you have to ask: did it cause the whole blockage, or is there a bigger clog further down? Troubleshooting is always systematic, starting from the symptom and working your way back to the root cause, exactly like a plumber tracking a drain backup by checking each pipe segment.
Troubleshooting is not random clicking or guessing — it's a structured approach much like a doctor diagnosing a patient. In Linux, systems log everything that happens, from normal operations to errors. These logs are stored in files under /var/log. The most important one is /var/log/messages (or /var/log/syslog on some distros). This file contains kernel messages, service start-ups, and errors. To read it, you use commands like tail, head, less, or grep.
A key concept is the boot process. When a Linux machine starts, it goes through several stages: BIOS/UEFI initialises hardware, then the bootloader (GRUB) loads the kernel, the kernel starts the init system (systemd or SysVinit), and finally services are launched. If a machine fails to boot, you need to find which stage failed. For example, if GRUB is broken, you see a 'grub rescue' prompt. That means the bootloader can't find the kernel. You'd fix it by booting from a live USB and reinstalling GRUB.
Network troubleshooting has its own toolkit. The ping command tests basic connectivity by sending ICMP echo requests. If ping fails, the issue might be a dead switch, a wrong IP address, or a firewall blocking ICMP. The traceroute command shows the path packets take to a destination, hop by hop. A sudden stop at hop 3 might mean a router is down or misconfigured.
Another critical command is netstat or its modern replacement, ss. These show open sockets — connections between your machine and others. For instance, you can see if a web server is listening on port 80. If it's not, apache might have crashed. The ip command replaces ifconfig and shows IP addresses, routes, and link status. Use 'ip addr' to see your own IP, 'ip route' to see the routing table.
Firewalls add a layer of complexity. iptables or nftables (the newer tool) control what traffic is allowed in and out. A common mistake is to block all incoming traffic and forget to allow SSH, locking yourself out. To troubleshoot firewall issues, you can list rules with 'iptables -L' or, on modern systems, 'nft list ruleset'.
System resource issues are also common. The top or htop command shows CPU and memory usage. If CPU is 100%, a process might be stuck in an infinite loop. Use ps aux to list all processes and kill the problematic one with kill. Disk space is checked with df -h. If a disk fills up, a service might stop writing logs. Use du -sh /var/log to see which directory is eating space.
Why does this exist? Because modern IT environments are complex piles of interdependent components. A single misconfiguration, a full disk, or a corrupted file can cascade into a total service outage. Without a structured troubleshooting method, you either waste hours guessing or reboot everything, which might fix the symptom but not the cause.
What it replaces? It replaces the old 'hit it with a hammer' approach. Systematic troubleshooting replaces random trial-and-error with a repeatable, scientific method: gather information, form a hypothesis, test it, and confirm the fix.
Identify and reproduce the problem
Get a clear, specific description of the issue from the user or monitoring system. For example, 'The website returns a 502 Bad Gateway error.' Then reproduce it yourself to confirm it is not a one-off glitch. Reproducing ensures you are not chasing a phantom problem.
Check system and service logs
Look at /var/log/messages (or /var/log/syslog) and service-specific logs like /var/log/nginx/error.log. Use journalctl if the service uses systemd. Logs often contain the exact error message, like 'No space left on device' or 'Connection refused'. This step narrows down the layer (OS, network, application).
Verify network connectivity and access
Use ping to test basic IP connectivity to the machine and its gateway. Then use nc -zv <ip> <port> to test if the specific service port is reachable. If the port is blocked, check firewall rules with iptables -L or nft list ruleset. This step separates network issues from application issues.
Inspect resource utilisation (CPU, memory, disk, I/O)
Run top to see if CPU or RAM is saturated. Run df -h to check disk space. Run iostat -x to see disk I/O wait times. High numbers in any of these can cause timeouts or crashes. For example, 100% disk I/O wait will make everything slow.
Isolate and test the root cause
Based on the evidence, form a hypothesis (e.g., 'the disk is full because logrotate failed'). Test it by, for instance, freeing up disk space and restarting the service. If the problem disappears, you have found the root cause. If not, go back to step 2 and re-read the logs.
Apply a permanent fix and document
Implement a solution that prevents recurrence: e.g., configure log rotation, add monitoring alerts, or change a config file. Then write down what happened, what commands you ran, and what the fix was. This documentation helps others (and future you) if it happens again.
Imagine you work as a junior systems administrator for a mid-sized e-commerce company. It's 10 AM on Black Friday, and the website becomes unresponsive. Customers are seeing 'Connection timed out' errors. Your manager shouts, 'Fix it now!' Panic sets in, but you know the troubleshooting method.
Step 1: Define the problem. You confirm it's not just your machine by checking from your phone on a different network — same error. The site is down for everyone.
Step 2: Check the obvious. You SSH into one of the web servers (if you can). SSH connects — good. You run 'systemctl status nginx' (web server software). It says 'active (running)'. Not the web server process itself. Next, check the application. You run 'curl http://localhost' from the server. It returns HTML. So the web server is working locally. The problem must be between the server and the outside world.
Step 3: Networking. You run 'ip addr' on the server. The IP address looks correct. You ping the default gateway (the first router hop). It fails. You ping your own server's IP from another machine in the same data centre — also fails. That suggests a routing or firewall issue at the network level.
Step 4: Isolate the layer. You check the networking logs in /var/log/messages and find a series of 'RTNETLINK answers: File exists' errors. This indicates a duplicate IP address — two machines on the same network have been assigned the same IP, causing a conflict. You check the DHCP server logs and see that a newly provisioned database server was given the same static IP as the web server. The network switch is confused about which MAC address to send packets to.
Step 5: Resolve. You change the IP assignment on the database server to a free address, then restart its network interface. You also flush the ARP cache on the switch (arp -d on the router) to clear the old mapping. Then you test: you ping the web server from your laptop, and it responds. You reload the website — it works. Everything is back up.
What does an IT professional actually do with this? They document every step: what was broken, what logs they checked, and what command fixed it. They also set up monitoring to alert if duplicate IPs appear again. They communicate with the team about the change. In larger environments, they might have to navigate change management procedures, but in a crisis, speed and correctness win. The skill is staying calm, working through the layers (Physical -> Data Link -> Network -> Transport -> Application), and not jumping to conclusions.
LPIC-2 exam objective 202.6 is titled 'Troubleshooting' and it's one of the most practical parts of the test. The exam will not ask you theoretical questions about the OSI model. Instead, it presents a scenario: 'A user cannot connect to a remote server. Which command would you use first to diagnose the issue?' The answer is almost always ping or traceroute, but with a trap. For example, they might describe that the user can ping the server but not access a web page. The next step is then to check the service itself with netstat -tlnp or ss -tlnp, not to check DNS.
Common traps:
They love to set up a scenario where the service is running but the firewall is blocking the port. They might say 'ss -tlnp shows the port is listening, but the client gets connection refused.' The answer is to check iptables rules or nftables.
They also test the boot process. A classic question: 'After editing /etc/fstab, the system fails to boot to a login prompt. What is the most likely cause?' The trap answer is a kernel panic, but the real one is that the root filesystem cannot be mounted because of a typo in fstab. They expect you to know to boot into single-user mode (init 1 or systemd rescue mode) and fix the file.
They test log file locations. A question like 'Where would you look for kernel-level hardware errors?' The correct answer is /var/log/messages or /var/log/kern.log, not /var/log/maillog.
Another trap: 'Which tool shows real-time process activity and resource usage?' Beginners might say ps, but the exam wants top or htop because they show real-time updates, not a snapshot.
Key concepts to memorise:
The difference between 'ping' (layer 3 connectivity) and 'telnet' or 'nc' (layer 4 port connectivity).
The recovery modes: systemd-rescue and systemd-emergency.
How to use journalctl to view systemd logs: journalctl -u nginx.service shows logs for nginx only.
The use of dmesg to view kernel ring buffer messages, especially for hardware issues.
How to interpret strace output: it shows system calls and signals. Useful when a program crashes silently.
The 'lsof' command can show which files are open, including ports. 'lsof -i :80' shows who is listening on port 80.
You will also see questions about performance issues. They might give you output from vmstat or iostat and ask which resource is the bottleneck. For instance, if 'wa' (wait I/O) is high, the disk is the problem. If 'si' and 'so' (swap in/out) are high, the system is running out of RAM.
The exam is multiple-choice, but some questions are 'choose all that apply', so read carefully. The correct answer is rarely the most complex command — often it's the simplest one that gives the necessary information.
Systematic troubleshooting uses the order: symptoms -> logs -> hardware/network -> application -> fix, never skip steps.
The /var/log/messages file (or syslog) is the single most important diagnostic file in Linux.
Ping tests basic IP connectivity, but always verify a specific port using nc (netcat) or telnet for application-level checks.
If a service fails to start, always check 'journalctl -xe' or 'systemctl status --full' first, not just the configuration file.
The 'dmesg' command reveals kernel-level errors like hardware failures, OOM (out of memory) kills, or driver issues.
Always document each diagnostic step — writing down what you tried prevents repeating the same test twice.
In LPIC-2, the correct answer to a troubleshooting question is rarely the most complicated command; it's the step that isolates the layer (network vs. application vs. hardware).
These come up on the exam all the time. Here's how to tell them apart.
Ping
Tests basic IP connectivity to a single host
Uses ICMP echo request and reply
Does not show the path taken
traceroute
Shows every router hop along the path
Measures latency per hop to identify where packets drop or slow down
Helps isolate the specific router or network segment causing failure
ss (socket statistics)
Modern, faster, and recommended tool
Shows detailed socket information including process IDs
Part of the iproute2 package, still actively maintained
netstat
Older tool, now often deprecated
Requires -p flag to show process names, but sometimes fails without root
Output is similar but less efficient on high-traffic systems
systemctl status
Shows a brief summary of a service's current state (active, inactive, failed)
Includes a few recent log lines from the service
Ideal for a quick health check
journalctl
Displays the full, scrollable log history for a specified unit
Supports filtering by time, priority, and unit name
Best for deep diagnosis of recurring failures
dmesg
Displays kernel ring buffer messages (hardware, driver, memory issues)
Shows real-time kernel events, not persistent across reboots (unless syslog captures it)
Useful for hardware detection failures at boot
/var/log/messages
Persistent log file of all system messages, including user-space applications
Archived and rotated, so you can review history from days ago
Best for investigating application errors and user login failures
Mistake
If a server is unresponsive, the first thing to do is reboot it.
Correct
Rebooting wipes all diagnostic evidence. You should first gather logs (dmesg, journalctl, /var/log/messages) to find the cause, then reboot only if necessary.
Beginners panic and assume a reboot always fixes things, but in enterprise environments, losing the state of logs can make root cause analysis impossible.
Mistake
Ping is the ultimate test of network connectivity.
Correct
Ping only tests ICMP echo, which is a simple protocol. A server might respond to ping but have its web server port blocked by a firewall. Use nc or telnet to test specific ports.
Because ping is easy to run and always works when the network is fine, people over-rely on it, missing port-level or application-level issues.
Mistake
If a disk is full, deleting the largest file is the best fix.
Correct
You should identify which process is filling the disk using lsof +L1 or by checking log rotation settings. Deleting a large file without addressing the underlying cause (e.g., a runaway log) leaves the problem unresolved.
Common because 'df -h' shows a full disk, and deleting the first big file seems quick, but log files can grow back in minutes.
Mistake
SELinux or AppArmor errors are rare and can be ignored.
Correct
Security modules often cause 'Permission denied' errors silently. If a service fails to start with no obvious reason, check 'ausearch -m avc' for SELinux denials or 'journalctl -u service' for AppArmor messages.
Beginners do not know these modules exist, so they waste hours checking file permissions and fstab before discovering the security context is blocking access.
Mistake
The 'route' command shows the default gateway — that is the only routing information you need.
Correct
You should use 'ip route show' to see the full routing table, including multiple routes and metrics. The default gateway is only one entry among many, and a missing route might cause asymmetric routing.
Because the old 'route' command is gradually disappearing (deprecated), and beginners learn only the simple default gateway concept, missing the complexity of routing tables in modern networks.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Access the server via SSH (if possible) and run 'top' to check CPU and RAM, 'df -h' for disk space, and 'dmesg | tail' for kernel errors. These three commands give you a quick health snapshot.
Ping tests whether a remote host is reachable at the IP level and shows round-trip time. Traceroute shows the exact path packets take through routers, hop by hop, helping you locate where a delay or failure occurs.
Use 'ss -tlnp' on the server to list all listening ports and their processes. To test from a client, use 'nc -zv server_ip port_number' to see if the port is reachable.
Even as root, you can see 'Permission denied' if SELinux or AppArmor is blocking access. Check with 'ls -Z' for SELinux context or 'apparmor_status' for AppArmor profiles. It is usually not a standard file permission issue.
At the GRUB boot menu, edit the kernel line (press 'e') and append 'single' or 'init=/bin/bash' to the end of the line starting with 'linux'. Then press Ctrl+X or F10 to boot. Once in single-user mode, you can edit /etc/fstab with a text editor.
A kernel panic is a fatal error where the Linux kernel cannot recover, usually displayed as a text screen with a message like 'Kernel panic - not syncing'. You should check the full output, which often points to a specific driver or hardware fault. Boot from a rescue disk to inspect logs.
You've finished Troubleshooting System and Network Issues. Continue through the LPIC-2 study guide to build a complete picture of the exam.
Done with this chapter?