Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Users who wish to transfer files between ECMWF and Member State servers using ectrans need to declare one or more ectrans associations for the storage/retrieval of the remote file. This can be done either through the Web Toolkit command ecaccess-association-put (see Web Toolkit - The full featured client) or through the ECaccess Web interface of the target gateway (see The Web server). For every association (previously known as "msuser"), the hostname, login username and password, target path and transfer protocol need to be specified. Target directories can be located on:

...

NameDefaultType/UnitComment
sftp.usetmp"true"booleanForce using a temporary name when transmitting a file to the remote SFTP server. Once the transmission has completed successfully, the file is renamed with its final target name. The temporary file name is by default the file name with the ".tmp" suffix concatenated to it, however this behaviour can be customised with the "mksuffix", "prefix" and "suffix" options.
sftp.mkdirs"true"booleanAllow creating the directory named by the target pathname, including any necessary but non-existent parent directories.
sftp.prefix-stringWhen using the "usetmp" option, this option allow setting the prefix to use for the temporary file name (e.g. ".").
sftp.suffix-stringWhen using the "usetmp" option, this option allow setting the suffix to use for the temporary file name (e.g. ".tmp").
sftp.chmod-integerIf set, the file mode will be changed according to the value of this option once it has uploaded a file successfully (e.g. "640").
sftp.ignoreCheck"false"booleanIf set, the remote size of the file is not checked after an upload has been completed. If not set, the remote size of the uploaded file is checked and compared against the size of the source file.
sftp.execCode"0"integerAllow specifying the expected exit code of the command (or list of commands) started with the "execCmd" option. If the exit code is different from the one specified in this parameter then the transmission is tagged as failed.
sftp.execCmd-stringAllow specifying a command (or list of commands) to execute on the SSH server once a file has been transmitted (multiple commands can be specified using the semi columns separator). The parameter "$filename" can be introduced in the commands and it gives the name of the file transmitted (source or target depending if the transmission was a upload or a download).
sftp.kex-stringAllow specifying the key exchange algorithms to allow (e.g. "ecdh-sha2-nistp256,ecdh-sha2-nistp384"). The list of valid algorithms can be found at http://www.jcraft.com/jsch/README (multiple algorithms can be specified using the columns separator). By default all supported algorithms are allowed.
sftp.serverHostKey-stringAllow specifying the host key type to allow (e.g. "ssh-dss,ssh-rsa,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521"). The list of valid key types can be found at http://www.jcraft.com/jsch/README (multiple algorithms can be specified using the columns separator). By default all supported key types are allowed.
sftp.cipher-stringAllow specifying the cipher algorithms to allow (e.g. "aes128-cbc,3des-cbc,blowfish-cbc"). The list of valid algorithms can be found at http://www.jcraft.com/jsch/README (multiple algorithms can be specified using the columns separator). By default all supported algorithms are allowed.
sftp.mac-string

Allow specifying the Message Authentication Code (MAC) algorithms to allow (e.g. "hmac-md5,hmac-md5-96,hmac-sha1,hmac-sha1-96"). The list of valid algorithms can be found at http://www.jcraft.com/jsch/README (multiple algorithms can be specified using the columns separator). By default all supported algorithms are allowed.

sftp.compression-stringAllow specifying the compression algorithms to allow (e.g. "zlib,none"). The list of valid algorithms can be found at http://www.jcraft.com/jsch/README (multiple algorithms can be specified using the columns separator). By default all supported algorithms are allowed.
sftp.fingerPrint-stringAllow specifying a key's fingerprint. It is a lower-case hexadecimal representation of the MD5 of a key (e.g. "22:fb:ee:fe:18:cd:aa:9a:9c:78:89:9f:b4:78:75:b4"). If a key's fingerprint is defined then it is compared against the key's fingerprint of the remote SFTP server when connecting. If it does not match then the connection is rejected.
sftp.preferredAuthentications-stringAllow specifying the list of supported authentication methods (e.g. password,publickey,keyboard-interactive).
sftp.passPhrase-stringThe passphrase necessary to access the private key when the "privateKey" or "privateKeyFile" option is used. This option is only required if the private key is protected.
sftp.privateKeyFile-string

Adds This option adds a private key to be used for public-key authentication. It is only supported for associations hosted on remote ECaccess Gateways that have been properly configured by the gateway administrator.

The private key should must be in the PEM format and stored in a file accessible from the ECaccess Gateway (the , with appropriate file permissions should be set accordingly).

This option contains the name of the should contain the full path to the key file (e.g. "/home/uid/.ssh/mykey.pem").

If both options, "privateKeyFile" and "privateKey" are defined, then the latest will be ignored.

sftp.privateKey-string

Adds a private key to be used for public-key authentication. The private key should be in the PEM format.

e.g. sftp.privateKey="
-----BEGIN DSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,2AEF2CF4618CA866

HFYVuVUSTdJacwFiaTcY3I5jUHaVIYdDQHIZarzox/ikWC6rN7uJbwPrZZPH+WV1
PfwX9yC7jVlUSgPu7o8fSdEhBaPZQ0NZ7Gq3cOpsRYicRa3F4GebK748m8VZLXLe
(...)
b4DxuN8evYgV/M9V76JoW5TDiizgst86w915bjM/gl+mVPPH/XitSOpaADtHSpDv
ph1kgY1fjwt5le+i66AnLmbJ4cLLAMz3
-----END DSA PRIVATE KEY-----
"

As shown in the example above, the first line should always be sftp.privateKey=" and the last line should always be " with the content of the private key in between.

sftp.listenAddress-stringAllow specifying the source address used by the SFTP client when connecting to the remote SFTP server. By default, the system will pick up a valid local address.
sftp.sessionTimeOut"60000"millisecondsThis value is used as the socket timeout parameter, and also as the default connection timeout. The value must be > 0. A timeout of zero is interpreted as an infinite timeout.
sftp.connectTimeOut-millisecondsThis value is used as the connection timeout. The value must be > 0. A timeout of zero is interpreted as an infinite timeout. The default value is the same as the "sessionTimeOut" value.
sftp.mksuffix"false"booleanIf set, the SFTP client will generate a random suffix for the "usetmp" option with 3 characters (e.g. ".3te"). If this option is used then the "prefix" and "suffix" options are ignored.
sftp.port"22"shortAllow specifying the port number of the remote SFTP server to connect to.

...

> ectrans -help
usage: ectrans [-gateway name] -remote association@[protocol] \
          [-get|-put] -source [ec:|ectmp:]filename [args ...] (*)
        ectrans -check requestID (*)

 -gateway  {arg} - access gateway name (default (**): ecaccess boaccess.ecmwf.int)
 -remote   {arg} - association and (optional) protocol (default (**): *none*)
 -source   {arg} - source file name
 -target   {arg} - target file name (default: same as -source)
 -mailto   {arg} - target email address (default: current user)
 -lifetime {arg} - lifetime of the file in the spool (default: 1w) (***) (****)
 -delay    {arg} - transmission delay (default: immediate transfer) (***) (****)
 -at       {arg} - transmission date (default: immediate transfer) (****)
 -format   {arg} - define the date format as used with -at (default: yyyyMMddHHmmss)
 -retryCnt {arg} - define the number of retries (default: async=144, sync=0)
 -retryFrq {arg} - define the frequency of retries (default: async=10m, sync=1m) (***)
 -maxTime  {arg} - define the maximum transfer duration (default: 12h) (***)
 -priority {arg} - transmission priority 0-99 (default: 99) (****)
 -put            - interactive/synchronous transfer (no spool)
 -get            - interactive/synchronous pull (rather than push) file
 -onsuccess      - mail sent on successful transfer
 -onfailure      - mail sent when transfer has failed
 -onretry        - mail sent when transfer is retried
 -keep           - keep the request in the spool till expiration (****) (*****)
 -remove         - always remove the request from the spool (****) (*****)
 -reject         - if existing target file (default)
 -append         - if existing target file
 -resume         - if existing target file
 -overwrite      - if existing target file
 -verbose        - verbose mode on
 -version        - print version number
 -help           - this message

    (*) If successful, a requestID is returned, which can be used in
          check requests. Exit code is 0 on success and >0 otherwise.
   (**) The default values depend on the GATEWAY or REMOTE environment
          variables.
  (***) Duration in weeks, days, hours, minutes or seconds (e.g. 1w|2d).
 (****) These options are only relevant when the spool is used. The spool
          is no used during interactive transfers (-get and -put options).
(*****) By default, successful requests are removed from the spool and
          failed requests are kept in the spool till expiration.

...

> ectrans gateway ecaccess.meteo.ms \
          -remote myUser@genericFtp \
          -get -source fff \
          -verbose
gateway: ecaccess boaccess.ecmwf.int
echost: ecgate.ecmwf.int
ecport: 644
action: get
ecuser: uid
target: fff
source: fff
keep  : false
option: reject
File to download (0 bytes)
5140480 bytes to download

...