If you need to run a certain task at given regular intervals automatically, you may use our cron service available on the hosts hpc-cron for HPC users and ecs-cron for those with no access to the HPCF.

Use hpc-cron or ecs-cron

Do not run your crontabs on any host other than "hpc-cron" or "ecs-cron", as they may disappear at any point after a reboot or maintenance session. The only guaranteed nodes are hpc-cron and ecs-cron.

How to check my scheduled cron jobs

You can see your cron jobs by logging into hpc-cron or ecs-cron via ssh:

ssh hpc-cron
# or for users with no HPCF access:
ssh ecs-cron

Once logged in, run:

crontab -l

How to add a new cron job

You can edit your crontab by using the following command on the hpc-cron or ecs-cron special nodes:

crontab -e

And the editor specified by the EDITOR environment variable will open it for you. Each line on the file will be a new cron job, with typically five time-and-date fields, followed by the command to execute.  For example, the following line:

03 10,22 1 * * /path/to/my_script.sh

would execute /path/to/my_script.sh on the first day of every month, at 10:03 and 22:03. You may check man 5 crontab for all the details on the syntax and possible options to customise your cron jobs.

Limited environment

Cron tasks run with a very limited user environment. No user profile will be sourced by cron to run before running your task. Unless the task is very lightweight, please consider running your cron tasks through batch as described below

In particular, the $SCRATCH environment variable IS NOT SET.  If you need to use $SCRATCH in your cron job then please specify the actual path to your $SCRATCH (i.e., /scratch/...) and DO NOT USE the environment variable !

Running your cron tasks through batch

The easiest and safest approach is to run your regular automated tasks is to use cron to submit a job to the batch system with sbatch. Using the same example as above, it would be as easy as:

03 10,22 1 * * sbatch -Q /path/to/my_script.sh

Quiet cron

Note the -Q option in sbatch to suppress normal output and avoid getting an email for every job submitted through cron.

This avoids loading the cron node itself with heavy tasks, distributing the load to the any available node on the entire HPCF. It also has the added advantage of giving you a full environment within the batch job.

Use case example: ensuring your ecflow logserver is always up

If you need to run a logserver on the Atos HPCF, it is a good idea to have a cron job that checks if the server is up and restarts it automatically otherwise. Because the log server runs on a different node, you would need to run it remotely. The following example runs every 15 minutes to make a connection to hpc-log to start the logserver:

*/15 * * * * ssh hpc-log "bash -lc 'ml ecflow; ecflow_logserver.sh <logserver options>'"

3 Comments

  1. Luke Jones

    cron on my workstation used to automatically send me any standard error by email, but this doesn't seem to happen on the Atos. It was useful when I had a typo in my crontab. Could this be restored?

    1. Cristian Simarro

      MAILTO="luke.jones@ecmwf.int"

      on the first line should work

      1. Luke Jones

        Great, that works. Thanks!