Troubleshooting practice questions

A system administrator is troubleshooting a container that fails to start with the error: 'Error: cannot start container: listen tcp4 :80: bind: address already in use'. The container is intended to serve HTTP traffic on port 80. What is the most appropriate first step to resolve this issue?

Trap 1: Add --force to the podman run command

The `--force` flag is not a valid option for `podman run`; it is used with `podman rm` or `podman stop` to forcefully terminate a running or stopped container without waiting for graceful shutdown. Even if you attempted to pass it, it would be rejected by Podman's CLI parser, and it would not address the underlying port conflict. To fix the error, you must either stop the process already bound to port 80 or map the container to a different host port, such as `-p 8080:80`.

Trap 2: Add --replace to the podman run command

The `--replace` option tells Podman to remove any existing container with the same name before creating the new one, so it only resolves name collisions, not port conflicts. If port 80 is already occupied, the new container will still fail to bind regardless of whether `--replace` is present, because the conflict is with an external process or another container's port forward. It is therefore the wrong tool for this scenario.

Trap 3: Use --net=host to bypass the port mapping

With `--net=host`, the container shares the host's network namespace and binds directly to host interfaces, bypassing the need for `-p` port mappings entirely. However, this does not grant any special ability to re-bind an already-occupied port: if a host process is listening on port 80, the container's service will encounter the same 'address already in use' error. The only way to avoid the conflict is to free the port or change the port that the container's service uses.

A
Add --force to the podman run command
Why it fails: The `--force` flag is not a valid option for `podman run`; it is used with `podman rm` or `podman stop` to forcefully terminate a running or stopped container without waiting for graceful shutdown. Even if you attempted to pass it, it would be rejected by Podman's CLI parser, and it would not address the underlying port conflict. To fix the error, you must either stop the process already bound to port 80 or map the container to a different host port, such as `-p 8080:80`.
B
Check which process is using port 80 and either stop that process or use a different host port
Start by identifying which process holds port 80 with `ss -tlnp` or `lsof -i :80`; the output shows the process ID and name. You can then stop that service with `systemctl stop` or `kill` if it is no longer needed, or simply run the container with a different host port mapping like `-p 8080:80` to avoid the conflict. This is the standard, correct approach because it either frees the required resource or selects an unused port.
C
Add --replace to the podman run command
Why it fails: The `--replace` option tells Podman to remove any existing container with the same name before creating the new one, so it only resolves name collisions, not port conflicts. If port 80 is already occupied, the new container will still fail to bind regardless of whether `--replace` is present, because the conflict is with an external process or another container's port forward. It is therefore the wrong tool for this scenario.
D
Use --net=host to bypass the port mapping
Why it fails: With `--net=host`, the container shares the host's network namespace and binds directly to host interfaces, bypassing the need for `-p` port mappings entirely. However, this does not grant any special ability to re-bind an already-occupied port: if a host process is listening on port 80, the container's service will encounter the same 'address already in use' error. The only way to avoid the conflict is to free the port or change the port that the container's service uses.

Question 2hardmultiple choice

A database container crashes repeatedly. The administrator wants to see the last 10 lines of the container's logs before it exited. Which command should be used?

Trap 1: podman logs -f <container>

The `-f` (or `--follow`) flag attaches to the container's log stream and keeps the output open, displaying new lines in real time as they are written. In a crash-looping container this is not the same as viewing a fixed tail; it will wait for future output and may never terminate, so it does not give the administrator a clean snapshot of the last 10 lines that led to the crash. A follow is useful for debugging a running container, not for retrieving a bounded history from a crashed one.

Trap 2: podman logs --since 10m <container>

The `--since 10m` option filters logs to only those entries generated in the last 10 minutes, rather than the last 10 lines. This is a time-based filter, not a count-based one, so it can return hundreds of lines or none at all depending on how long ago the container crashed and how verbose the logs are. For a crash loop, you need the final lines from the most recent exit, not all log output within a time window.

Trap 3: podman inspect <container>

The `podman inspect` command retrieves low-level metadata about the container—such as its configuration, mount points, network settings, and state—as a large JSON object. It does not read the container's stdout/stderr logs, so it provides no visibility into the application error that caused the crash. Inspect is useful for checking the container's exit code or environment, but it cannot replace `podman logs` for diagnosing the actual runtime failure.

A
podman logs --tail 10 <container>
The `--tail 10` flag limits the output to the last 10 lines of the container's log stream, which is exactly what an administrator needs when a database crashes repeatedly. Because the crash reason almost always appears in the final log entries before the process exits, viewing the tail provides the most recent error without wading through the full history. This is the standard, non-interactive way to capture the fatal diagnostics for a container that has already stopped.
B
podman logs -f <container>
Why it fails: The `-f` (or `--follow`) flag attaches to the container's log stream and keeps the output open, displaying new lines in real time as they are written. In a crash-looping container this is not the same as viewing a fixed tail; it will wait for future output and may never terminate, so it does not give the administrator a clean snapshot of the last 10 lines that led to the crash. A follow is useful for debugging a running container, not for retrieving a bounded history from a crashed one.
C
podman logs --since 10m <container>
Why it fails: The `--since 10m` option filters logs to only those entries generated in the last 10 minutes, rather than the last 10 lines. This is a time-based filter, not a count-based one, so it can return hundreds of lines or none at all depending on how long ago the container crashed and how verbose the logs are. For a crash loop, you need the final lines from the most recent exit, not all log output within a time window.
D
podman inspect <container>
Why it fails: The `podman inspect` command retrieves low-level metadata about the container—such as its configuration, mount points, network settings, and state—as a large JSON object. It does not read the container's stdout/stderr logs, so it provides no visibility into the application error that caused the crash. Inspect is useful for checking the container's exit code or environment, but it cannot replace `podman logs` for diagnosing the actual runtime failure.

Question 3mediummultiple choice

An administrator wants to temporarily disable the firewalld service for troubleshooting. Which command will stop the service and prevent it from starting on subsequent boots?

Trap 1: systemctl stop --now firewalld

The --now flag is a systemd modifier that is only valid with action verbs like enable, disable, or mask; it cannot be combined with stop. Since systemctl stop already halts the service immediately, giving --now is syntactically invalid and systemd will abort the command with an 'unrecognized option' error. As a result, firewalld remains active and keeps running.

Trap 2: systemctl mask firewalld

Masking firewalld creates a symlink from the unit file to /dev/null, preventing any process—including root—from starting it even manually, which is far more aggressive than a temporary shutdown. The command does not stop the currently active firewalld process, and to reverse the change you must run systemctl unmask firewalld. For a short troubleshooting window, this is overkill and can mask regressions in service configuration.

Trap 3: systemctl stop firewalld

This halts the running firewalld daemon and flushes its active rules, but it only affects the live state; the service's enablement symlinks in /etc/systemd/system remain intact. Because the unit is still enabled, systemd will automatically restart firewalld on the next boot. To make the firewall stay down until you deliberately restart it, you also need to run systemctl disable firewalld or use the combined --now form.

A
systemctl stop --now firewalld
Why it fails: The --now flag is a systemd modifier that is only valid with action verbs like enable, disable, or mask; it cannot be combined with stop. Since systemctl stop already halts the service immediately, giving --now is syntactically invalid and systemd will abort the command with an 'unrecognized option' error. As a result, firewalld remains active and keeps running.
B
systemctl mask firewalld
Why it fails: Masking firewalld creates a symlink from the unit file to /dev/null, preventing any process—including root—from starting it even manually, which is far more aggressive than a temporary shutdown. The command does not stop the currently active firewalld process, and to reverse the change you must run systemctl unmask firewalld. For a short troubleshooting window, this is overkill and can mask regressions in service configuration.
C
systemctl stop firewalld
Why it fails: This halts the running firewalld daemon and flushes its active rules, but it only affects the live state; the service's enablement symlinks in /etc/systemd/system remain intact. Because the unit is still enabled, systemd will automatically restart firewalld on the next boot. To make the firewall stay down until you deliberately restart it, you also need to run systemctl disable firewalld or use the combined --now form.
D
systemctl disable firewalld
Why it fails: This removes the boot-time enablement symlinks for firewalld, so the service will not be pulled in during future system startup, but it leaves the currently running firewalld process untouched. The firewall continues to enforce whatever rules are loaded right now, and a reboot would also be required to make it go away. For immediate effect, you must stop the service in addition, which is why disable alone is only half the job.
E
systemctl disable --now firewalld
Running systemctl disable --now firewalld performs both operations atomically: it stops the active firewalld daemon and removes the boot-time enablement symlinks so that the service will not start at the next boot. This is the cleanest way to temporarily take the firewall down across reboots while leaving the unit unmasked and available for manual start later. It is the standard single-command answer for 'temporarily disable' scenarios.

Question 4hardmultiple choice

During boot, a server fails to mount an NFS filesystem listed in /etc/fstab. Which troubleshooting step should be taken first to isolate the issue?

Trap 1: Check the status of nfs-client.target with 'systemctl status…

The command 'systemctl status nfs-client.target' is invalid because nfs-client.target is not a standard systemd target in Red Hat Enterprise Linux or systemd-based distributions; the correct target for remote filesystems is remote-fs.target. Running this command will likely return a 'Unit nfs-client.target could not be found' error, so it provides no diagnostic information about the NFS mount failure. The NFS client functionality, such as rpcbind and nfs-idmapd, is managed through other units like nfs-client.service or rpc-statd.service, but they are not tied to a distinct 'nfs-client.target' that controls mounting at boot.

Trap 2: Try to manually mount the NFS share with 'mount /mnt/nfs'

While manually running 'mount /mnt/nfs' can confirm that the NFS server is reachable and the filesystem is mountable at that moment, it bypasses the systemd-managed boot process and does not reveal whether the mount unit was supposed to be activated by remote-fs.target. If the manual mount succeeds, it suggests the network and NFS server are functional, but the boot failure may be caused by ordering dependencies, a failed network-online.target, or a malformed /etc/fstab entry, none of which are diagnosed by this command. Moreover, if the mount unit is in a failed state, systemd will not retry it automatically, so a manual mount does not tell you why the boot-time activation failed.

Trap 3: View kernel messages with 'dmesg | grep -i nfs'

Viewing kernel messages with 'dmesg | grep -i nfs' can expose low-level errors such as 'NFS: mount to server failed: No route to host' or rpc timeout messages, but it is a reactive step that only shows symptoms, not the systemd unit state that governs boot-time mounts. The boot failure might occur because remote-fs.target is not reached due to a unrelated unit failing or because the NFS mount unit is disabled, in which case dmesg would show no NFS-related messages at all. The systematic first step is to check remote-fs.target's status, which aggregates the mount units and their dependency chain; only after seeing a failed unit there should you use dmesg to inspect the kernel-level cause.

A
Check the status of remote-fs.target with 'systemctl status remote-fs.target'
Checking 'systemctl status remote-fs.target' is the correct first step because systemd uses this target to synchronize the activation of all remote filesystem mounts, including NFS, during the boot sequence. If the target is in a 'failed' or 'degraded' state, the status output will directly show which mount unit failed and why, allowing you to then inspect that specific unit's logs or configuration rather than guessing. This target is explicitly ordered after network-online.target and pulls in the mount units, so its state is the authoritative indicator of whether boot-time NFS mounting was attempted and completed.
B
Check the status of nfs-client.target with 'systemctl status nfs-client.target'
Why it fails: The command 'systemctl status nfs-client.target' is invalid because nfs-client.target is not a standard systemd target in Red Hat Enterprise Linux or systemd-based distributions; the correct target for remote filesystems is remote-fs.target. Running this command will likely return a 'Unit nfs-client.target could not be found' error, so it provides no diagnostic information about the NFS mount failure. The NFS client functionality, such as rpcbind and nfs-idmapd, is managed through other units like nfs-client.service or rpc-statd.service, but they are not tied to a distinct 'nfs-client.target' that controls mounting at boot.
C
Try to manually mount the NFS share with 'mount /mnt/nfs'
Why it fails: While manually running 'mount /mnt/nfs' can confirm that the NFS server is reachable and the filesystem is mountable at that moment, it bypasses the systemd-managed boot process and does not reveal whether the mount unit was supposed to be activated by remote-fs.target. If the manual mount succeeds, it suggests the network and NFS server are functional, but the boot failure may be caused by ordering dependencies, a failed network-online.target, or a malformed /etc/fstab entry, none of which are diagnosed by this command. Moreover, if the mount unit is in a failed state, systemd will not retry it automatically, so a manual mount does not tell you why the boot-time activation failed.
D
View kernel messages with 'dmesg | grep -i nfs'
Why it fails: Viewing kernel messages with 'dmesg | grep -i nfs' can expose low-level errors such as 'NFS: mount to server failed: No route to host' or rpc timeout messages, but it is a reactive step that only shows symptoms, not the systemd unit state that governs boot-time mounts. The boot failure might occur because remote-fs.target is not reached due to a unrelated unit failing or because the NFS mount unit is disabled, in which case dmesg would show no NFS-related messages at all. The systematic first step is to check remote-fs.target's status, which aggregates the mount units and their dependency chain; only after seeing a failed unit there should you use dmesg to inspect the kernel-level cause.

Question 5easymultiple choice

A system administrator is troubleshooting a RHEL 9 server that fails to boot and drops into emergency mode. The system console shows an error about mounting /dev/sdb1 on /data. The administrator enters emergency mode, checks /etc/fstab, and sees the line: /dev/sdb1 /data ext4 defaults 0 0. The /data directory exists but /dev/sdb1 is a partition on an external USB drive that was removed. The administrator needs the system to boot normally without the USB drive and plans to fix the mount configuration later. Which course of action should the administrator take?

Trap 1: Remove the line from /etc/fstab and run systemctl daemon-reload,…

Although removing the /dev/sdb1 line from /etc/fstab does stop systemd from attempting that mount at boot, running systemctl daemon-reload will not re-read /etc/fstab — the fstab file is processed by systemd-fstab-generator only during boot/daemon-reload for unit generation, but not for fstab content changes in this context. The option incorrectly pairs daemon-reload with a procedure that actually requires a reboot for the fstab change to take effect, making the suggested command sequence inaccurate.

Trap 2: Add the nofail option to the fstab line, then reboot.

Adding the nofail mount option only instructs systemd to continue booting even if the mount fails; it does not prevent the mount attempt from being made. The /dev/sdb1 device is still missing, so the mount will still fail (silently if nofail is set), but the requirement is to avoid mounting that device altogether — not merely to ignore the failure and proceed.

Trap 3: Delete the /data directory and reboot.

Deleting the /data directory removes only the mount point directory from the filesystem, but the /dev/sdb1 entry remains active in /etc/fstab. Since fstab still lists the device, systemd will still attempt the mount at boot, and the absence of the /data directory will cause the mount operation to fail (or behave unpredictably), so the system will not boot normally as required.

A
Remove the line from /etc/fstab and run systemctl daemon-reload, then reboot.
Why it fails: Although removing the /dev/sdb1 line from /etc/fstab does stop systemd from attempting that mount at boot, running systemctl daemon-reload will not re-read /etc/fstab — the fstab file is processed by systemd-fstab-generator only during boot/daemon-reload for unit generation, but not for fstab content changes in this context. The option incorrectly pairs daemon-reload with a procedure that actually requires a reboot for the fstab change to take effect, making the suggested command sequence inaccurate.
B
Add the nofail option to the fstab line, then reboot.
Why it fails: Adding the nofail mount option only instructs systemd to continue booting even if the mount fails; it does not prevent the mount attempt from being made. The /dev/sdb1 device is still missing, so the mount will still fail (silently if nofail is set), but the requirement is to avoid mounting that device altogether — not merely to ignore the failure and proceed.
C
Delete the /data directory and reboot.
Why it fails: Deleting the /data directory removes only the mount point directory from the filesystem, but the /dev/sdb1 entry remains active in /etc/fstab. Since fstab still lists the device, systemd will still attempt the mount at boot, and the absence of the /data directory will cause the mount operation to fail (or behave unpredictably), so the system will not boot normally as required.
D
Use a text editor to insert '#' at the beginning of the /dev/sdb1 line in /etc/fstab, then reboot.
Inserting a '#' at the start of the /dev/sdb1 line comments out the entire fstab entry, causing systemd-fstab-generator to skip generating a mount unit for that device. With the entry now inactive, no mount attempt is made at boot, and the system avoids the failure while retaining the rest of the fstab configuration for maintenance or later re-enablement.

Question 6hardmultiple choice

A system administrator is troubleshooting a custom service called 'database.service' that fails intermittently. The service is a proprietary database that requires large amounts of memory. The administrator runs systemctl status database and sees 'Active: failed (Result: core-dump)' and the journal shows 'Out of memory: Killed process (database) total-vm:...' The server has 8GB RAM and 2 CPU cores. The service unit file does not contain any memory limits. The application is configured to use up to 4GB. The administrator suspects the systemd service is being killed by the OOM killer. Which action should the administrator take to prevent this issue?

Trap 1: Set MemoryMax=6G in the service unit file.

MemoryMax=6G limits the service's cgroup memory usage to 6 GiB, but it does not alter the process's oom_score or OOM-killer eligibility. If the service exceeds that cgroup limit, the kernel may kill it deliberately for violating memory.max, and under global memory pressure the OOM killer can still select it based on its oom_score. Therefore, capping memory can even increase the chance of a kill rather than prevent one.

Trap 2: Modify kernel parameters to disable the OOM killer.

There is no common kernel parameter that cleanly disables the OOM killer; sysctl vm.panic_on_oom=1 causes the kernel to panic instead of killing a process, which typically crashes the entire host. Even if one tried to make allocations fail rather than kill, the kernel depends on the OOM killer to maintain system stability, and removing it would lead to unresponsive processes or hangs. Thus this approach is either unsupported or more harmful than the original problem.

Trap 3: Increase swap space to 16GB.

Increasing swap to 16GB merely gives the kernel more anonymous pages it can move out of physical memory, but it does not change the OOM killer's calculations. Under sustained memory pressure, the kernel may still invoke the OOM killer because swap is full or because processes are continuously faulting; the custom service could be selected based on its oom_score. It also risks severe swap thrashing that degrades performance without protecting the service.

A
Set MemoryMax=6G in the service unit file.
Why it fails: MemoryMax=6G limits the service's cgroup memory usage to 6 GiB, but it does not alter the process's oom_score or OOM-killer eligibility. If the service exceeds that cgroup limit, the kernel may kill it deliberately for violating memory.max, and under global memory pressure the OOM killer can still select it based on its oom_score. Therefore, capping memory can even increase the chance of a kill rather than prevent one.
B
Set OOMScoreAdjust=-1000 in the service unit file.
Setting OOMScoreAdjust=-1000 in the service unit file writes -1000 to /proc/<pid>/oom_score_adj, making the process's OOM score effectively zero and marking it as the kernel's last choice for OOM victim selection. This is a direct, unit-level mitigation that applies only to the custom service, so it doesn't weaken system-wide OOM behavior. It is the recommended way to protect a critical service from being killed.
C
Modify kernel parameters to disable the OOM killer.
Why it fails: There is no common kernel parameter that cleanly disables the OOM killer; sysctl vm.panic_on_oom=1 causes the kernel to panic instead of killing a process, which typically crashes the entire host. Even if one tried to make allocations fail rather than kill, the kernel depends on the OOM killer to maintain system stability, and removing it would lead to unresponsive processes or hangs. Thus this approach is either unsupported or more harmful than the original problem.
D
Increase swap space to 16GB.
Why it fails: Increasing swap to 16GB merely gives the kernel more anonymous pages it can move out of physical memory, but it does not change the OOM killer's calculations. Under sustained memory pressure, the kernel may still invoke the OOM killer because swap is full or because processes are continuously faulting; the custom service could be selected based on its oom_score. It also risks severe swap thrashing that degrades performance without protecting the service.

Question 7hardmultiple choice

A system administrator is troubleshooting a cron job that runs a script as root. The script is located at /root/scripts/backup.sh and has permissions 755. The cron job is defined in /etc/crontab with the line: 0 2 * * * root /root/scripts/backup.sh. However, the script does not run at the scheduled time. The administrator checks the cron logs and finds no errors. The administrator then runs the script manually as root and it executes successfully. What is the most likely cause of the cron job not running?

Trap 1: The script is not executable by the root user.

If the script lacked execute permission for the owner, attempting to run it manually would produce a 'Permission denied' error, but the administrator reported that manual execution succeeds. The file permissions of 755 grant the owner (root) read and execute access, so the script is clearly executable by root. Since cron runs the script with the same kernel-level permission checks as a manual run, an invalid permission flag cannot be the cause of the cron-specific failure.

Trap 2: The cron daemon is not running.

If the cron daemon (crond) were not running, no cron jobs at all would fire, including any other scheduled maintenance tasks, and the system would show cron service errors or empty cron logs. The fact that only this particular job fails while others presumably run, and that the script executes successfully when invoked by hand, strongly indicates the daemon is operational. A stopped daemon would also prevent the cron job from even being attempted, rather than producing a script-level runtime failure.

Trap 3: The /etc/crontab file does not allow running scripts from /root.

There is no inherent restriction in /etc/crontab or in the cron subsystem that forbids executing scripts from /root. Cron simply runs the command as the specified user, and as long as that user (here root) has read and execute permissions on the script, the file location is irrelevant. The script's location in /root may affect environment variables that rely on PATH, but since the cron line uses an absolute path, that is a non-issue. If /root were the problem, the script would also fail when run manually from another directory, which is not the case.

A
The cron job line uses absolute path to the script but the script requires an environment variable that is not set in cron's minimal environment.
Cron runs with a sparse environment that includes only a few default variables such as HOME, LOGIN, and SHELL, with PATH typically set to /usr/bin:/bin. It does not source /etc/profile or a user's ~/.bash_profile, so any custom environment variable that the script expects—for example, a database password or an application home—will be unset. Even though the script's absolute path is correct and it runs manually, the missing variable causes the script to fail under cron. The solution is to export the variable inside the script or define it explicitly in the crontab.
B
The script is not executable by the root user.
Why it fails: If the script lacked execute permission for the owner, attempting to run it manually would produce a 'Permission denied' error, but the administrator reported that manual execution succeeds. The file permissions of 755 grant the owner (root) read and execute access, so the script is clearly executable by root. Since cron runs the script with the same kernel-level permission checks as a manual run, an invalid permission flag cannot be the cause of the cron-specific failure.
C
The cron daemon is not running.
Why it fails: If the cron daemon (crond) were not running, no cron jobs at all would fire, including any other scheduled maintenance tasks, and the system would show cron service errors or empty cron logs. The fact that only this particular job fails while others presumably run, and that the script executes successfully when invoked by hand, strongly indicates the daemon is operational. A stopped daemon would also prevent the cron job from even being attempted, rather than producing a script-level runtime failure.
D
The /etc/crontab file does not allow running scripts from /root.
Why it fails: There is no inherent restriction in /etc/crontab or in the cron subsystem that forbids executing scripts from /root. Cron simply runs the command as the specified user, and as long as that user (here root) has read and execute permissions on the script, the file location is irrelevant. The script's location in /root may affect environment variables that rely on PATH, but since the cron line uses an absolute path, that is a non-issue. If /root were the problem, the script would also fail when run manually from another directory, which is not the case.