The Web Toolkit, previously known as ECaccess shell commands or ECtools, is a set of Perl scripts for the management of files, file transfers, jobs, ectrans associations and events at ECMWF. They can be run by any user and on any Member State host. The Web Toolkit is using SOAP to access the ECaccess web server. Running these commands requires a valid certificate. The command ecaccess-certificate-create will create a certificate in the user's home directory ($HOME/.eccert.crt) from an ECMWF user identifier and a passcode (generated by a security token). You need to ensure the following environment parameters are set with the correct values:
http_ecaccess=gateway.meteo.ms:9080 https_ecaccess=gateway.meteo.ms:9443
(e.g. if your local ECaccess Gateway name is "gateway.meteo.ms" and you are using the default ECaccess http/s ports 9080/9443) The default values are pointing to the boaccess.ecmwf.int server. Your gateway administrator can provide other default values for these parameters. However, your environment variables take precedence over these default values. If the directory containing the Web Toolkit commands is not in your command path or you do not know the directory in which the commands are installed, try running the "ecaccess-gateway-name" command. If the command is not available, you will need to contact your Computing Representative, your local ECaccess administrator - if known - or User Support at ECMWF. Alternatively, you may wish to install the Web Toolkit yourself (see Releases - Web Toolkit). The ECaccess Tools are organized in sets covering access to the whole computing and archiving facilities of ECMWF and are described in the following sections. Each command is documented with its own man page which provide explanation as well as examples on how to use it. The following options are common to all the Web Toolkit commands:
| -help | Print a brief help message and exits. |
| -manual | Prints the manual page and exits. |
| -debug | Display the SOAP messages exchanged. |
| -version | Print the ECaccess version number. |
Certificate management
The ECaccess certificate can be created using the command ecaccess-certificate-create. From an ECMWF user identifier and a PASSCODE (using a security token), it generates a certificate in $HOME/.eccert.crt. Alternatively, a certificate can be created using the ECacess Web interface, see The Web server. The Web Toolkit is also available at ECMWF. In contrast to using the Web Toolkit installed at your site you will not need a certificate when using it at ECMWF as you have already been validated at login. To display a help screen describing the ecaccess-certificate-create usage:
> ecaccess-certificate-create -help Usage: ecaccess-certificate-create [options] [user-id] Options: -help brief help message -manual full documentation -debug enable messages output Options: -help Print a brief help message and exits. -manual Prints the manual page and exits. -debug Display the SOAP messages exchanged.
Certificates are PEM/Base64 encoded ASCII files. OpenSSL can be used to decode and display certificate components. The expiry of the various ECaccess services can be displayed with the command ecaccess-certificate-list:
uid@ad6-101{/home/uid}:1 --> ecaccess-certificate-list
submitJob 168h Jan 18 11:56 job submission
getJobList 168h Jan 18 11:56 job list
deleteJob 168h Jan 18 11:56 delete a job
getJobResult 168h Jan 18 11:56 job result
deleteFile 168h Jan 18 11:56 delete file
getFileList 168h Jan 18 11:56 get file list
mkdir 168h Jan 18 11:56 make directory
getFileSize 168h Jan 18 11:56 get file size
readFile 168h Jan 18 11:56 read file
writeFile 168h Jan 18 11:56 write file
moveFile 168h Jan 18 11:56 move file
rmdir 168h Jan 18 11:56 remove directory
chmod 168h Jan 18 11:56 change file mode
getTempFile 168h Jan 18 11:56 create temporary file
getTransferList 168h Jan 18 11:56 get transfer list
As can be seen from the output, for a normal user-id the validity is 168 hours (7 days) for all services. The date/time shown refers to the expiration of the certificate.
General information
To display the "Message of the day" (/etc/motd on the Atos HPC (ecs-login or hpc-login)) containing announcements of upcoming system sessions etc., use the command ecaccess-cosinfo:
-> ecaccess-cosinfo ECMWF SYSTEM SESSIONS -------------------- WEDNESDAY the 25th of JANUARY 2023: ---------------------------------- 08:30-10:30 UTC Mars and ECFS System Session: impact: Mars and ECFS will be unavailable To reread the message please use: more /etc/motd or cat /etc/motd ==============================================================================
Gateway information
Commands named ecaccess-gateway-* (table 2) provide information about ECaccess gateways (see ECaccess Gateway), see following table:
| Command | Purpose |
| ecaccess-gateway-list | List of ECaccess Gateways |
| ecaccess-gateway-name | Display the name of your default ECaccess Gateway |
The command ecaccess-gateway-name is the only one which can be used without authentication/certificate.
File management
Files at ECMWF can be managed through the Web Toolkit commands named ecaccess-file-*. The file location is specified with the following syntax [domain:][/user-id/]path where domain: can take the values as listed in table 3. The user-id refers to an ECMWF computer user-id. Only if the domain is specified as an ECFS domain, then the user-id could also be a common pool, e.g. demeter. Table 3: Domain values.
| Domain value | Purpose |
| home: | the Atos HPC $HOME file system |
| scratch: | the Atos HPC $SCRATCH file system |
| ec: | the ECFS domain ec: |
| ectmp: | the ECFS domain ectmp: |
| host-name: | any server at ECMWF, e.g. ecs-login |
If no domain is specified then an absolute path will translate to an absolute path on the Atos HPC and a relative path will translate to a path in the HOME directory of the current user. If no user-id is specified then the current user-id is selected by default. When you specify a host-name you are by default under the root directory; the user-id parameter can not be used with a host-name. File location examples:
| bin/a.out | file a.out in directory $HOME/bin of the current user |
| home:bin/a.out | file a.out in directory $HOME/bin of the current user |
| /tmp/a.out | file a.out in directory /tmp on ecs-login |
| home:/xzy/bin/a.out | file a.out in directory $HOME/bin of user xzy |
| ec:bin/a.out | file a.out in directory bin in the current user's ECFS domain ec: |
| ec:/xzy/bin/a.out | file a.out in directory bin in user xzy's ECFS domain ec: |
| hpc-login:/ec/vol/xzy/a.out | file a.out in directory /ec/vol/xzy/ on hpc-login |
Table 4: Web Toolkit commands for file management (ecaccess-file-*).
| Command | Purpose |
| ecaccess-file-chmod | Change ECaccess File Mode Bits |
| ecaccess-file-copy | Copy an ECaccess File |
| ecaccess-file-delete | Remove an ECaccess File |
| ecaccess-file-dir | List ECaccess Directory Contents |
| ecaccess-file-get | Download an ECaccess File |
| ecaccess-file-mget | Download Multiple ECaccess Files at once |
| ecaccess-file-mkdir | Make a Directory on the ECaccess File System |
| ecaccess-file-modtime | Show the Last Modification Time of an ECaccess File |
| ecaccess-file-move | Move or Rename ECaccess Files |
| ecaccess-file-mput | Upload Multiple Local Files on the ECaccess File System at once |
| ecaccess-file-put | Upload a File on the ECaccess File System |
| ecaccess-file-rmdir | Remove a Directory on the ECaccess File System |
| ecaccess-file-size | Show the Size of an ECaccess File |
Batch job management
Batch jobs at ECMWF can be managed through the Web Toolkit commands named ecaccess-job/queue-* in the table below.
| Command | Purpose |
| ecaccess-job-delete | Delete an ECaccess Job |
| ecaccess-job-get | Download a Job Output/Input/Error File |
| ecaccess-job-list | List all ECaccess Jobs |
| ecaccess-job-restart | Restart an ECaccess Job |
| ecaccess-job-submit | Submit a new ECaccess Job |
| ecaccess-queue-list | List available queues |
Possible values of the job status, which can be checked with the command ecaccess-job-list or via the Web interface, are listed in table below.
| Status | Meaning |
| DONE | Jobs have successfully completed |
| EXEC | Jobs are running |
| INIT | Jobs are being initialised |
| RETR | Jobs will be resubmitted |
| STDBY | Jobs are waiting for an event |
| STOP | Jobs have NOT completed (error) |
| WAIT | Jobs have been queued to the scheduler (e.g. LoadLeveler) |
A special service (see option -eventIds) allows to automatically submit jobs to be run when certain points in the daily ECMWF operational forecast suite have been reached. The main purpose is to ensure that certain data are available before e.g. submitting a MARS request. These events correspond to the different stages when the ECMWF operational activity has produced certain data or products. The list of events can be retrieved with the "ecaccess-event-list" command.
Management of events
ECMWF maintains some notifications (events) which are linked to ECMWF's operational activity and offers the service for time-critical jobs (see also separate documentation). This service is also available to MS users who maintain their own notifications and can therefore create simple dependencies between different activities, at ECMWF and remote sites. The shell commands to managed events are listed in table 7. Table 7: Web Toolkit commands for management of events at ECMWF (ecaccess-event-*).
| Command | Purpose |
| ecaccess-event-clear | Clear an ECaccess Event |
| ecaccess-event-create | Create an ECaccess Event |
| ecaccess-event-delete | Delete an ECaccess Event |
| ecaccess-event-grant | Grant usage of an ECaccess Event |
| ecaccess-event-list | List available events |
| ecaccess-event-send | Trigger an ECaccess Event |
Management of ECtrans transfers
The commands for the management of ECMWF-initiated transfers (ectrans, see section 4.2) are listed in the table below. They can only be used for the management of transfers, which have used the ECaccess gateway as shown with the ecaccess-gateway-name command.
| Command | Purpose |
| ecaccess-ectrans-delete | Delete ECtrans |
| ecaccess-ectrans-list | List all ectrans transfers |
| ecaccess-ectrans-request | Request a new ECtrans transfer |
| ecaccess-ectrans-restart | Restart an existing ECtrans transfer |
Management of ECtrans associations
Before making use of ectrans, users will need to declare an ectrans association (previously referred to as 'remote Member State user (msuser)') for the storage/retrieval of the remote file. The management of these associations can be done through the Web Toolkit commands named ecaccess-association-* listed in the table below.
| Command | Purpose |
| ecaccess-association-delete | Delete Association |
| ecaccess-association-get | Get the Association Descriptive File |
| ecaccess-association-list | List your ECtrans associations |
| ecaccess-association-protocol | List the supported ECtrans Protocol |
| ecaccess-association-put | Update/Create an Association |
Execution return codes
The option '-debug' can be used with any Web Toolkit command to display information concerning the SOAP messages exchanged. If a Web Toolkit command doesn't work correctly, please run it with the -debug option and send the output to ECMWF for further investigation. Web Toolkit commands return 0 if successful, and a non-zero code otherwise. Each time an error occurs, a message indicating the error is displayed to the user.