ECFS
Section: User Contributed Perl Documentation (1)Updated: 2015-10-26
Index Return to Main Contents
NAME
ecfs - ECMWF file storageDESCRIPTION
ECFS is designed and run by the DHS group at ECMWF. It adds additional storage domains to a UNIX-like filesystem, and is modelled on the UNIX file interface, mapping files to a Unix-compatible directory tree.In each domain it supports absolute and relative pathnames, a current working directory and home directory (analogous to those in UNIX) and some wildcard support.
Each ECFS command, say ecp, tries to be closely modelled on its corresponding UNIX command, in this case cp, and generally implements a subset of its options.
ECFS domains at ECMWF
An ECFS domain is specified by prefixing a pathname with a domain name thus: domain:path/to/file.At ECMWF there are two storage domains:
- *
- ec: - Permanent; files are kept indefinitely, although future developments might offer or apply an expiry facility.
- *
- ectmp: - Temporary; files are stored for a limited time (currently 90 days) after which they are automatically deleted.
In addition the local UNIX filesystem can also be regarded as a domain with an optional prefix local.
Specifying domains
In most commands, there is a default domain, which is not the local domain; it is usually ec:. So, for example, the commands:- *
- els -l ec:foo
- *
- els -l foo
are equivalent; in the latter case the default ECFS domain is implied.
But for commands that involve two domains, the ECFS domain must be explicitly specified. For example, copying local file foo into domain ec: as filename bar is:
- *
- ecp foo ec:bar
Note (1) the default domain is now local, so foo is local; (2) the ec: specification for the ECFS domain is mandatory.
If in doubt, state the domain.
Specifying paths
A pathname in ECFS resembles that in UNIX, but there may be restrictions on the character set supported. The currently accepted set is:
-
alphanumeric: A-Z, a-z, 0-9
other: '_' ',' '.' '+' '-'
directory separator: '/'
There is limited wildcard support of '*', '?' and '[...]' as per the simple UNIX Bourne shell. But wildcards are only available in the final component of a path. So some/path/to/*something* is supported but some/*guess*/to/something is not.
Some differences from UNIX commands
Where reasonably possible an ECFS command mimics the corresponding UNIX command. But there are some inevitable restrictions, a few additions and some differences.This section headlines a few of those, mainly by way of example and to give a feel for for principles at work. It is not a complete list; the relevant man pages provide the detail.
- *
-
emove: UNIX offers cp and mv, so ECFS offers ecp and emv.
But ECFS additionally offers emove.
The UNIX mv actually does two different things:
-
- *
- Across filesystems it is a copy of the contents from an old location to a new location and a deletion of the old location; this is a big operation for big files.
- *
- Within a filesystem it simply renames a file; this is a very fast and efficient operation irrespective of file size
-
In ECFS emv is for the copy-new-then-delete-old operation across different domains. An emove command is introduced for the swift rename case; it requires that both the old and new locations are within the same domain.
-
- *
- ecp, emv, emove: Additional option flags control whether or not to overwrite an existing destination file.
- *
- ecp, emv, emove: When a destination path includes non-existent directories, they are automatically created. This can be envisaged as similar to UNIX mkdir -p and being enabled by default.
- *
- els: Whereas the UNIX ls command has a wide range of option flags els has a small subset.
- *
- Wildcards in pathnames are restricted to the final component.
- *
- Associated with each domain are its own current working directory and home directory: see epwd and ecd.
- *
- There is a separate 'umask' specification for ECFS domains, controlled by eumask.
- *
- ecat always requires two arguments to delineate the source and destination; one of these is always '-' for the corresponding standard input (STDIN) or standard output (STDOUT). Further, when the destination is an ECFS pathname, an estimated size must be supplied as '-s bytes' command line arguments.
- *
- etest implements only those options which apply to files and directories.
RETURN VALUE
Commands return 0 if successful, and a non-zero code otherwise.SEE ALSO
epwd(1), ecd(1), ecp(1), emv(1), emove(1), ecat(1), etouch(1), erm(1), emkdir(1), ermdir(1), echgrp(1), echmod(1), eumask(1), els(1), etest(1)
Index
This document was created by man2html, using the manual pages.
Time: 14:51:57 GMT, July 23, 2018