Versions Compared

Old Version 14

changes.mady.by.user Xavier Abellan

Saved on

compared with

New Version 15

changes.mady.by.user Xavier Abellan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Explicitly discouraging the blind copy of HOME which may include dot files

...

Note

All examples are done with the generic HPCF login node hpc-login, but you should use ecs-login if you don't have access to the full HPCF service.

Example: Transferring a directory tree between

...

Reading Platforms onto the Atos HPCF

Warning
titleHOME dot files

If you are transferring your HOME from Reading platforms into Atos HPCF, make sure you do not transfer your dot files and directories in your home top level directory, such as your ~/.profile, ~/.bashrc, ~/.config or ~/.local. They may interfere with the local configuration on the Atos HPCF. Instead, we suggest you carefully select and transfer your files and directories in your home that you really need to keep.

You may initiate the transfer from ECGATE for the standard set of filesystems:

No Format
user@ecgb11:~> rsync -avz $SCRATCH/mydataset hpc-login:/scratch/user/

...

Tip

You may add the --delete option if you also wish to delete files on the destination that have been removed from the source.

The same idea can be applied to transfer files from your Cray HPCF filesystems.

Warning

You can use the $HOME environment variable to refer to your files in the rsync command provided this is protected by single quotes to ensure the variable is expanded on the remote host.

For example,  to copy a directory from $HOME on ecgate to $HOME on Atos HPCF, initiating the transfer from ecgate, use:

No Format
rsync -avz $HOME/mydataset 'hpc-login:$HOME/'


However, the $SCRATCH or $PERM environment variables can only be used to refer to your files on the local host from where you initiate the rsync command as these are not set for use by the rsync command on the remote host.

An alternative option is to use the $HOME variable with pattern substitution.  For example, to copy a directory tree from $SCRATCH on ecgate to $SCRATCH on Atos HPCF running the command on ecgate use:

No Format
user@ecgb11:~> rsync -avz $SCRATCH/mydataset 'hpc-login:${HOME/home/scratch}/'

or from $PERM on ecgate to $PERM on Atos HPCF running the command on the latter use:

No Format
user@aa6-100:~> rsync -avz 'ecgate:${HOME/home/perm}/mydataset' $PERM/

The single quotes are needed here to ensure the $HOME variable takes its value on the remote rather than the local host.

...