• Created by Unknown User (nagc), last modified on May 25, 2017

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

This tutorial explains how to create a new experiment from an existing one by copying the initial files and setting up the experiment directory.

On this page...

 

Login to ECMWF Cray High Performance Computing Facility

  •  login in to ecaccess.ecmwf.int using the training id and securID password you have been given.

  • To access the Cray HPCF:

    ssh ccb

Creating the experiment initial files

This section will explain how to copy and change the experiment id.

 Initial files for the observed SST experiment, 'ob00', and the climatological experiment 'clim', have been created and are made available in the directory : /perm/rd/openifs/oifs_workshop_2017/expts-inidata/

This directory contains multiple dates for each experiment:

ob00/2015110100 to ob00/2015110100 etc.

Copy previous experiment

Before creating the experiment, we first need to create the initial data. If you plan to rerun the ob00 or clim experiments and use the initial data 'as-is' you can skip this step.

In this example, we will create a new experiment id from the existing ob00 experiment, without changing any of the data itself. You can of course, use your own previous experiments, the only difference is the location of the starting initial data.

In this example, the new experiment is intended for a 50 member ensemble run. We want to use a new experiment id, ob50, to distinguish it from ob00 which only used 10 ensemble members.

First step is to make a copy of the ob00 data to your /scratch directory:

Copy previous initial data from previous experiment...
cd scratch				# starting from your home directory
mkdir -p inidir/ob50 	# our new experiment id
cd inidir/ob50
cp -rL /perm/rd/openifs/oifs_workshop_2017/expts-inidata/ob00/2015110100 .

Make sure you use the -L option!  This ensures any 'linked' files are copied as actual files and not symbolic links (true for the fort.4 file). Not doing so will make the fort.4 uneditable.

Ignore any errors about files not being copies because of permission problems.

To copy multiple dates. Example 1: first 5 days
cd inidir/ob50
cp -rL /perm/rd/openifs/oifs_workshop_2017/expts-inidata/ob00/2015110[1-5]* .

Please only copy the initial dates you intend to use. Each date uses 0.5 Gbyte of file storage.

To copy multiple dates. Example 2: copy all dates
cd inidir/ob50
cp -rL /perm/rd/openifs/oifs_workshop_2017/expts-inidata/ob00/201511* .   # this will be slow!

Useful Unix commands

 

Remove a directory and ALL it's contents.. (note this is recursive delete!)
rm -rf 2015110100
rm -rf 2015110[1-5]00
Check size of all files in directory...
du -hs 2015110100
Check filesystem quota (on Cray)..
quota

whicih will produce output similar to the following. The '$SCRATCH' quota is important as this is the total limit for the filesystem /scratch where your experiments are stored.

Quota for $HOME and $PERM:
Disk quotas for user troifs0 (uid 16144): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
cnasa1:/vol/home
                   226M    480M    500M             132   20000   22000        
Disk quotas for user troifs0 (uid 16144): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
cnasa2:/vol/perm
                      0  26624M  27648M               1    200k    210k        

Quota for $SCRATCH ($TEMP) including $SCRATCHDIR ($TMPDIR):
Disk quotas for user troifs0 (uid 16144):
     Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
  /lus/snx11062 767099824  32212254720 32212254720       -  116570  5000000 5000000       -
Disk quotas for group ectrain (gid 1400):
     Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
  /lus/snx11062 767105640       0       0       -  118078       0       0       -

 

 

 

  • No labels