Automation and Scheduling (cron, at, systemd timers)

Automation and scheduling are the techniques that let you tell your Linux system to perform tasks at specific times or when certain conditions are met, without you needing to be logged in. Think of it as setting a digital calendar for your computer. The three main tools for this are cron, at, and systemd timers.

First, let's talk about 'cron'. Cron is a daemon—a background process that runs constantly—that reads configuration files called 'crontabs'. A crontab, short for 'cron table', is a file that lists jobs (commands to run) and their schedule. The schedule is defined by five time-and-date fields: minute (0-59), hour (0-23), day of month (1-31), month (1-12), and day of week (0-7, where 0 and 7 are Sunday). For example, '30 2 * * 1' means 'run every Monday at 2:30 AM'. The asterisks mean 'every'. So, you could have a cron job that backs up a database every day at midnight, or a job that sends a log report every Friday at 5 PM. Users can manage their own crontabs with the 'crontab -e' command. The system also has system-wide crontabs stored in directories like /etc/cron.hourly/, /etc/cron.daily/, etc.

Now, what about 'at'? The 'at' command is for one-time scheduled tasks. If you need to run a command at a specific time in the future, just once, you use 'at'. For instance, you might say 'at 3:00 PM' and then type the command you want to run. The system will execute it at exactly 3:00 PM, and then forget about it. This is ideal for deferred maintenance, like scheduling a reboot for 2 AM during a low-usage period. The 'atq' command lists pending 'at' jobs, and 'atrm' cancels them. The 'at' daemon (atd) must be running for this to work.

Finally, there are 'systemd timers'. systemd is the modern init system and service manager used by most Linux distributions today. A systemd timer is a unit file that controls when a corresponding service unit (a command or script) should run. Timers are more powerful than cron. They can run jobs at specific calendar times (like cron), but they can also run jobs relative to system events, such as '15 minutes after boot' or 'one hour after the last job finished'. Timers also have built-in features like randomised delays (to avoid load spikes from many jobs starting at once), persistent timers that catch up on missed runs after a system outage, and accurate monotonic timers for real-time latency. You enable a timer with 'systemctl enable timerName.timer' and start it with 'systemctl start timerName.timer'.

Why do we have three different tools? Each solves a different problem. Cron is excellent for predictable, recurring tasks like nightly maintenance. 'at' is perfect for one-off tasks that you don't want to repeat. systemd timers are the most modern and flexible, offering finer control and integration with the systemd ecosystem (like dependency management and logging). On the LPIC-2 exam, you must understand the syntax of each, how to create and manage them, and the common pitfalls such as environment variables (cron jobs run with a minimal shell environment, not your user's full profile) and output handling (by default, cron mails any output to the job owner).

Let's look at a concrete example of each. For a cron job, you'd edit your crontab: 'crontab -e'. You then add a line: '0 3 * * 1 /usr/bin/backup-script.sh'. This runs backup-script.sh every Monday at 3:00 AM. For an 'at' job, you'd type 'at 5:00 PM' then press Enter, type 'shutdown -h now' and press Ctrl+D. For a systemd timer, you create two files in /etc/systemd/system/: a service file (e.g., mybackup.service) and a timer file (e.g., mybackup.timer). The timer file contains the schedule, like 'OnCalendar=daily' or 'OnBootSec=10min'. You then enable and start the timer.

Common gotchas include remembering that the cron daemon must be running ('systemctl status cron' or 'ps aux | grep cron'), that 'at' jobs are stored in /var/spool/at/, and that systemd timers are separate unit files. Also, the environment for cron jobs is sparse—PATH is often just /usr/bin:/bin, so you should always use full paths to commands.

To manage these tools, you use standard Linux commands: 'crontab' (to edit, list, remove user crontabs), 'at', 'atq', 'atrm', 'systemctl', 'systemd-run' (to run a command as a transient timer). Understanding the logging for each is also crucial for exam troubleshooting. Cron logs are typically in /var/log/syslog or /var/log/cron. 'at' logs are often in /var/log/daemon.log. systemd timers log through the journal, accessible with 'journalctl -u mybackup.timer'. For the LPIC-2 exam, you need to be able to read and interpret these logs to diagnose why a job didn't run.

In summary, automation and scheduling are essential for efficient system administration. They free you from manual, repetitive tasks and ensure reliability. As you prepare for the LPIC-2 exam, focus on the syntax of each tool, the management commands, the environment differences, and the debugging techniques.

An IT professional, often called a system administrator or site reliability engineer, uses automation and scheduling to keep systems running smoothly without manual oversight. Consider a typical scenario at a mid-sized e-commerce company. The company has an online store, a customer database, and a server that processes orders. The IT team needs to perform several recurring tasks.

First, every night at 2:00 AM, the database must be backed up. The system administrator writes a cron job: '0 2 * * * /usr/local/bin/backup-db.sh'. This script connects to the database, dumps the data to a compressed file, and copies it to a remote storage location. Using cron ensures this happens every night reliably, even if the admin is asleep or on holiday.

Second, the company often releases software updates. The admin might schedule a one-off system reboot for next Tuesday at 4:00 AM using the 'at' command. They run 'at 4:00 AM Tue', then type 'shutdown -r now'. This ensures the reboot happens during off-peak hours and the admin doesn't have to remember to do it manually.

Third, the company has a monitoring script that checks if the main web server is running. If the server crashes, it needs to be restarted immediately, but only after the server has been down for at least 30 seconds to avoid a restart loop. The admin creates a systemd timer that runs a health check script every minute. The timer is configured with 'OnCalendar=*:0/1' and the associated service script checks the server, and if it's down, waits 30 seconds and then restarts it. The timer also has 'RandomizedDelaySec=5' to avoid all cron jobs hitting the system at the exact same second.

Fourth, the company has a compliance requirement to rotate log files every week. The admin adds a cron job that runs a log rotation script every Sunday at 3:00 AM. The script compresses old logs and deletes logs older than 30 days.

Finally, the admin needs to test a new backup script without waiting for the nightly cron job. They use 'systemd-run' to create a transient timer: 'systemd-run --on-active=10 /usr/local/bin/test-backup.sh'. This runs the test script after 10 seconds, without creating a permanent timer file.

The admin also regularly monitors the logs: they check /var/log/syslog for cron job failures, use 'journalctl -u backupdb.service' for systemd timer service logs, and 'atq' to see pending 'at' jobs. When a job fails, they check the environment. For example, a cron job might fail because the PATH environment variable doesn't include the directory where a command lives. The admin fixes this by specifying the full path in the script or by setting PATH in the crontab file.

In this real-world context, the IT professional doesn't just set and forget. They also manage user permissions (who can use 'at' or 'cron'), ensure the daemons are running ('systemctl status cron atd'), and handle exceptions like daylight saving time changes (cron usually handles this fine, but 'at' jobs scheduled during a skipped hour can cause issues). This practical, hands-on knowledge is exactly what the LPIC-2 exam evaluates. You must be able to create, manage, and troubleshoot these scheduling tools in real server environments.