The Web Toolkit, previously known as ECaccess shell commands or ECtools, is a set of Perl scripts for the management of files, file transfers, jobs, ectrans associations and events at ECMWF. They can be run by any user and on any Member State host. The Web Toolkit is using SOAP to access the ECaccess web server. Running these commands requires a valid certificate. The command ecaccess-certificate-create will create a certificate in the user's home directory ($HOME/.eccert.crt) from an ECMWF user identifier and a passcode (generated by a security token). You need to ensure the following environment parameters are set with the correct values:

http_ecaccess=gateway.meteo.ms:9080 
https_ecaccess=gateway.meteo.ms:9443

(e.g. if your local ECaccess Gateway name is "gateway.meteo.ms" and you are using the default ECaccess http/s ports 9080/9443) The default values are pointing to the boaccess.ecmwf.int server. Your gateway administrator can provide other default values for these parameters. However, your environment variables take precedence over these default values. If the directory containing the Web Toolkit commands is not in your command path or you do not know the directory in which the commands are installed, try running the "ecaccess-gateway-name" command. If the command is not available, you will need to contact your Computing Representative, your local ECaccess administrator - if known - or User Support at ECMWF. Alternatively, you may wish to install the Web Toolkit yourself (see Releases - Web Toolkit). The ECaccess Tools are organized in sets covering access to the whole computing and archiving facilities of ECMWF and are described in the following sections. Each command is documented with its own man page which provide explanation as well as examples on how to use it. The following options are common to all the Web Toolkit commands:

-helpPrint a brief help message and exits.
-manualPrints the manual page and exits.
-debugDisplay the SOAP messages exchanged.
-versionPrint the ECaccess version number.

Certificate management

The ECaccess certificate can be created using the command ecaccess-certificate-create. From an ECMWF user identifier and a PASSCODE (using a security token), it generates a certificate in $HOME/.eccert.crt. Alternatively, a certificate can be created using the ECacess Web interface, see The Web server. The Web Toolkit is also available at ECMWF. In contrast to using the Web Toolkit installed at your site you will not need a certificate when using it at ECMWF as you have already been validated at login. To display a help screen describing the ecaccess-certificate-create usage:

> ecaccess-certificate-create -help
Usage:
    ecaccess-certificate-create [options] [user-id]

    Options: -help brief help message -manual full documentation -debug
    enable messages output

Options:
    -help   Print a brief help message and exits.

    -manual Prints the manual page and exits.

    -debug  Display the SOAP messages exchanged.

Certificates are PEM/Base64 encoded ASCII files. OpenSSL can be used to decode and display certificate components. The expiry of the various ECaccess services can be displayed with the command ecaccess-certificate-list:

uid@ad6-101{/home/uid}:1 --> ecaccess-certificate-list
submitJob            168h     Jan 18 11:56         job submission
getJobList           168h     Jan 18 11:56         job list
deleteJob            168h     Jan 18 11:56         delete a job
getJobResult         168h     Jan 18 11:56         job result
deleteFile           168h     Jan 18 11:56         delete file
getFileList          168h     Jan 18 11:56         get file list
mkdir                168h     Jan 18 11:56         make directory
getFileSize          168h     Jan 18 11:56         get file size
readFile             168h     Jan 18 11:56         read file
writeFile            168h     Jan 18 11:56         write file
moveFile             168h     Jan 18 11:56         move file
rmdir                168h     Jan 18 11:56         remove directory
chmod                168h     Jan 18 11:56         change file mode
getTempFile          168h     Jan 18 11:56         create temporary file
getTransferList      168h     Jan 18 11:56         get transfer list

As can be seen from the output, for a normal user-id the validity is 168 hours (7 days) for all services. The date/time shown refers to the expiration of the certificate.

General information

To display the "Message of the day" (/etc/motd on the Atos HPC (ecs-login or hpc-login)) containing announcements of upcoming system sessions etc., use the command ecaccess-cosinfo:

-> ecaccess-cosinfo
ECMWF SYSTEM SESSIONS
--------------------

WEDNESDAY the 25th of JANUARY 2023:
----------------------------------

08:30-10:30 UTC Mars and ECFS System Session:
impact: Mars and ECFS will be unavailable

To reread the message please use: more /etc/motd or cat /etc/motd

==============================================================================

Gateway information

Commands named ecaccess-gateway-* (table 2) provide information about ECaccess gateways (see ECaccess Gateway), see following table:   

CommandPurpose
ecaccess-gateway-listList of ECaccess Gateways
ecaccess-gateway-nameDisplay the name of your default ECaccess Gateway

The command ecaccess-gateway-name is the only one which can be used without authentication/certificate.

File management

Files at ECMWF can be managed through the Web Toolkit commands named ecaccess-file-*. The file location is specified with the following syntax [domain:][/user-id/]path where domain: can take the values as listed in table 3. The user-id refers to an ECMWF computer user-id. Only if the domain is specified as an ECFS domain, then the user-id could also be a common pool, e.g. demeter. Table 3: Domain values.

Domain valuePurpose
home:the Atos HPC $HOME file system
scratch:the Atos HPC $SCRATCH file system
ec:the ECFS domain ec:
ectmp:the ECFS domain ectmp:
host-name:any server at ECMWF, e.g. ecs-login

If no domain is specified then an absolute path will translate to an absolute path on the Atos HPC and a relative path will translate to a path in the HOME directory of the current user. If no user-id is specified then the current user-id is selected by default. When you specify a host-name you are by default under the root directory; the user-id parameter can not be used with a host-name. File location examples:

bin/a.outfile a.out in directory $HOME/bin of the current user
home:bin/a.outfile a.out in directory $HOME/bin of the current user
/tmp/a.outfile a.out in directory /tmp on ecs-login
home:/xzy/bin/a.outfile a.out in directory $HOME/bin of user xzy
ec:bin/a.outfile a.out in directory bin in the current user's ECFS domain ec:
ec:/xzy/bin/a.outfile a.out in directory bin in user xzy's ECFS domain ec:
hpc-login:/ec/vol/xzy/a.outfile a.out in directory /ec/vol/xzy/ on hpc-login

Table 4: Web Toolkit commands for file management (ecaccess-file-*).

CommandPurpose
ecaccess-file-chmodChange ECaccess File Mode Bits
ecaccess-file-copyCopy an ECaccess File
ecaccess-file-deleteRemove an ECaccess File
ecaccess-file-dirList ECaccess Directory Contents
ecaccess-file-getDownload an ECaccess File
ecaccess-file-mgetDownload Multiple ECaccess Files at once
ecaccess-file-mkdirMake a Directory on the ECaccess File System
ecaccess-file-modtimeShow the Last Modification Time of an ECaccess File
ecaccess-file-moveMove or Rename ECaccess Files
ecaccess-file-mputUpload Multiple Local Files on the ECaccess File System at once
ecaccess-file-putUpload a File on the ECaccess File System
ecaccess-file-rmdirRemove a Directory on the ECaccess File System
ecaccess-file-sizeShow the Size of an ECaccess File

Batch job management

Batch jobs at ECMWF can be managed through the Web Toolkit commands named ecaccess-job/queue-* in the table below.

CommandPurpose
ecaccess-job-deleteDelete an ECaccess Job
ecaccess-job-getDownload a Job Output/Input/Error File
ecaccess-job-listList all ECaccess Jobs
ecaccess-job-restartRestart an ECaccess Job
ecaccess-job-submitSubmit a new ECaccess Job
ecaccess-queue-listList available queues

Possible values of the job status, which can be checked with the command ecaccess-job-list or via the Web interface, are listed in table below.

StatusMeaning
DONEJobs have successfully completed
EXECJobs are running
INITJobs are being initialised
RETRJobs will be resubmitted
STDBYJobs are waiting for an event
STOPJobs have NOT completed (error)
WAITJobs have been queued to the scheduler (e.g. LoadLeveler)

A special service (see option -eventIds) allows to automatically submit jobs to be run when certain points in the daily ECMWF operational forecast suite have been reached. The main purpose is to ensure that certain data are available before e.g. submitting a MARS request. These events correspond to the different stages when the ECMWF operational activity has produced certain data or products. The list of events can be retrieved with the "ecaccess-event-list" command.  

Management of events

ECMWF maintains some notifications (events) which are linked to ECMWF's operational activity and offers the service for time-critical jobs (see also separate documentation). This service is also available to MS users who maintain their own notifications and can therefore create simple dependencies between different activities, at ECMWF and remote sites. The shell commands to managed events are listed in table 7. Table 7: Web Toolkit commands for management of events at ECMWF (ecaccess-event-*).

CommandPurpose
ecaccess-event-clearClear an ECaccess Event
ecaccess-event-createCreate an ECaccess Event
ecaccess-event-deleteDelete an ECaccess Event
ecaccess-event-grantGrant usage of an ECaccess Event
ecaccess-event-listList available events
ecaccess-event-sendTrigger an ECaccess Event

Management of ECtrans transfers

The commands for the management of ECMWF-initiated transfers (ectrans, see section 4.2) are listed in the table below. They can only be used for the management of transfers, which have used the ECaccess gateway as shown with the ecaccess-gateway-name command.  

CommandPurpose
ecaccess-ectrans-deleteDelete ECtrans
ecaccess-ectrans-listList all ectrans transfers
ecaccess-ectrans-requestRequest a new ECtrans transfer
ecaccess-ectrans-restartRestart an existing ECtrans transfer

Management of ECtrans associations

Before making use of ectrans, users will need to declare an ectrans association (previously referred to as 'remote Member State user (msuser)') for the storage/retrieval of the remote file. The management of these associations can be done through the Web Toolkit commands named ecaccess-association-* listed in the table below. 

CommandPurpose
ecaccess-association-deleteDelete Association
ecaccess-association-getGet the Association Descriptive File
ecaccess-association-listList your ECtrans associations
ecaccess-association-protocolList the supported ECtrans Protocol
ecaccess-association-putUpdate/Create an Association

Execution return codes

The option '-debug' can be used with any Web Toolkit command to display information concerning the SOAP messages exchanged. If a Web Toolkit command doesn't work correctly, please run it with the -debug option and send the output to ECMWF for further investigation. Web Toolkit commands return 0 if successful, and a non-zero code otherwise. Each time an error occurs, a message indicating the error is displayed to the user.