DB-유

Scheduling Tasks with cron and Understanding Crontab Syntax

· linux, cron, automation

Every server I look after has a handful of cron jobs: database dumps, cert renewal, a script that checks disk space. cron is old and simple, and I still trip over the five-field syntax and the different crontab locations. Mostly I trip over jobs that fail quietly at three in the morning, so this is the syntax plus the habits that prevent that.

The five time fields

A crontab line is five time fields followed by the command:

# ┌ minute (0-59)
# │ ┌ hour (0-23)
# │ │ ┌ day of month (1-31)
# │ │ │ ┌ month (1-12)
# │ │ │ │ ┌ day of week (0-7, 0 and 7 are Sunday)
# │ │ │ │ │
  30 2 * * *  /usr/local/bin/backup-db.sh

Each field accepts a single value, a list (1,15), a range (1-5), a step (*/10) or * for every value. Some examples I use regularly:

*/5 * * * *      # every five minutes
0 * * * *        # every hour, on the hour
30 2 * * *       # daily at 02:30
0 3 * * 0        # Sundays at 03:00
0 6 1 * *        # first day of every month at 06:00
0 9-18 * * 1-5   # every hour from 09:00 to 18:00, Monday to Friday

Vixie cron (the one on CentOS and Ubuntu) also accepts the shortcuts @reboot, @hourly, @daily, @weekly and @monthly.

One subtlety: when both day-of-month and day-of-week are restricted, the job runs when either matches, not both. 0 0 13 * 5 runs on the 13th of every month and on every Friday, not only on Friday the 13th.

Where a job lives

Three places, each slightly different.

User crontabs are edited with crontab -e and stored under /var/spool/cron/ (CentOS) or /var/spool/cron/crontabs/ (Ubuntu). They run as that user and have no user field:

crontab -e
crontab -l          # list
sudo crontab -u postgres -l

/etc/crontab and /etc/cron.d/ are system crontabs with an extra field for the user to run as. I put one file per application in /etc/cron.d/. It's easy to copy around and easy to delete, and that's why I prefer it over crontab -e:

# /etc/cron.d/backup-db
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=ops@example.com

30 2 * * * root /usr/local/bin/backup-db.sh

Files in cron.d must be owned by root, not writable by others, and must not have a dot in the name or cron ignores them.

/etc/cron.daily/, cron.hourly/, cron.weekly/, cron.monthly/ hold executable scripts run by run-parts. Drop a script in and it runs; you have no control over the exact time, which is set by anacron on CentOS 7 (/etc/anacrontab) and by /etc/crontab on Ubuntu.

The environment is tiny

cron runs with a minimal environment: PATH is /usr/bin:/bin, no aliases, shell is /bin/sh. The most common reason a script "works from the command line but not from cron" is a program in /usr/local/bin or /usr/sbin called without its full path. I've lost an evening to that.

Set PATH at the top of the crontab as above, or use absolute paths. Locale or other variables go in the crontab too:

LANG=en_US.UTF-8
MYSQL_PWD=secret     # only if you cannot use ~/.my.cnf

Output and the mail flood

By default anything a job prints is emailed to the crontab owner, and without a configured mail it piles up in /var/spool/mail/root. I redirect output to a log file:

30 2 * * * root /usr/local/bin/backup-db.sh >> /var/log/backup-db.log 2>&1

Add a logrotate rule for the log, or use logger inside the script so output ends up in syslog. Set MAILTO="" to suppress mail entirely for a file.

Overlapping runs

A job slower than its interval will pile up. Wrap it in flock (part of util-linux on both distributions):

*/5 * * * * root flock -n /var/lock/sync-files.lock /usr/local/bin/sync-files.sh

-n makes the second instance exit immediately instead of waiting.

Test before you trust it

Add a throwaway job that fires every minute:

* * * * * date >> /tmp/cron-test.log

Then check that cron is running and what it executed:

systemctl status crond      # 'cron' on Ubuntu
sudo grep CRON /var/log/cron       # CentOS
sudo grep CRON /var/log/syslog     # Ubuntu

Remove the test job when you see entries.

Why isn't it running?

  • /bin/sh: backup-db.sh: command not found: missing PATH or a relative path. Use /usr/local/bin/backup-db.sh.
  • (root) BAD FILE MODE (/etc/cron.d/backup-db) in the cron log: file is group- or world-writable. chmod 644.
  • A cron.d job never runs and there's no error: the filename has a dot (backup.sh) or an editor's tilde. Rename it.
  • Percent signs in a command are treated as newlines, so date +%Y%m%d fails. Escape them (date +\%Y\%m\%d) or move the command into a script.
  • You (deploy) are not allowed to use this program (crontab): the user is in /etc/cron.deny or missing from /etc/cron.allow when that file exists.

For jobs needing dependencies or retries you may want something smarter. For "run this at this time and tell me if it fails", a line in /etc/cron.d/ with an explicit PATH, a log redirect and flock is enough for me.