To do once, at the initial time setting up the environment:
1. Get access to IFS group on ecgate, if not already joined, check
--> groups
2. Unpack the tarball sms.tar.gz and place it to your ecgate user, ideally directly in your home directory.
Standard setup is for operational analysis, copy respective ERA-I files to same name as for
OPERA setup to use this option.
3. In your ~/sms directory edit "cray.def" to (use either OPERA or ERA-I cray depending on
operational forecasting system or ERA-Interim extraction):
i/ replace all "/home/ms/se/usr" by your home address where “usr” is your username
ii/ replace "acnt" by your billing units account
find accound by entering
--> account -s your_user_name
iii/ replace “usr” in OWNER and USER to your user name
iv/ set LOGDIR to some directories on cca where you want to store your log files.
4. Edit sms/cray/getdata_scm.sms and sms/cray/getmars_scm.sms files to have paths accessible
by your user (both files setting should be consistent, copy either from ERA-I or from OPERA
depending on choice of data, copy to sms/cray to be used by extraction routine):
export SCRATCH=/ws/scratch/rd/rdx # only useful for $SCMDATA definition
export SCMDATA=$SCRATCH/pafv/data # will define the place your resulting profiles are stored
TMPDIR=/scratch/rd/pafv/scm_cray # working directory on cca
5. Edit sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray*
replace “usr” in EXPVER=usr to username
6. Make sure you have set up your ssh_keys in the way that you don't need to use password to go from ecgate to cca and from cca to ecgate.
--> ssh-keygen (no passphrase) and then
--> ssh-copy-id -i ~/.ssh/id_rsa.pub name_of_remote_host
7. Invoke the sms server to find your sms identifier, which is number after PROGNUM parameter, needed to log in to ecgate and set up server
--> /usr/local/apps/sms/current/bin/sms_start
8. Set up server in xcdp viewer
--> xcdp &
Add your server by: Edit → Preferences → Servers
Choose ”Name”, state “Host server” (e.g. ecgate) and set PROGNUM value (step 7) as “Number”
Steps to set up an experiment
1. Define the starting day(s) of your experiment(s) by adapting the YMD variable in sms/cray.def:
The current setting uses the starting date of 1st of April 2016:
define rcmd { repeat string YMD 20160401 }
If you want to do multiple experiments simply add other dates in the following way:
define rcmd { repeat string YMD 20160401 20160402 20160403 }
2. Edit following two files:
sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray
sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray.dataonly
between the lines labelled as "definition area" to reflect your desirable setting for points.
Note that when you ask for 1 hour frequency, some fields won't be available from Mars. So better to stick with 3 hour interval for the operational forecast.
If using only one time step keep slash afterwards for correct call of sed command.
Note that the start time has to be either 00 or 12 (for the operational forecast). So while you can have multiple dates at one go where is no way to have multiple starting time in the similar way.
getini1c_mars_run_validation_SCM_FC has information on which variables are read from MARS archive
Steps to run an experiment:
1. Invoke the sms server.
(Most likely it is already running. But then the invocation will just do nothing.):
--> /usr/local/apps/sms/current/bin/sms_start
2. Load your suite under the SMS server and start running it :
--> cd ~/sms
--> cdp
> set SMS_PROG your_sms_identifier
where your_sms_identifier is given in the the output after invoking the sms server, look for variable called PROGNUM
> login -t 60 ecgate your_user_name 314159
password 314159, can also be 1 instead of 314159
if errors here: likely need to invoke sms server
> play -r /cray<cray.def
> begin /cray<cray.def
> ctrl-D
to cancel a running application put
> /cancel -y -f /cray
before entering the play command
3. Invoke the xcdp viewer:
--> xcdp &
4. Expand branch of jobs on host by middle click on symbol to right. Get further information on job by right clicking.
Color coding explanation for xcdp:
grey/light blue: queuing job
green: job is running
yellow: job has successfully completed
red: job was aborted with errors
more information on errors can be found in either getmars_scm.* or getmars_scm.* in folder LOGDIR on cca (cray) depending on which process crashes
before restarting job check via
--> qstat -u your_user_name
if job is still running, if yes, kill by $ qdel process_id
violet: right click and select ”Restart”
all scripts in tree below will also be restarted and exectuables again copied to cca (changes included on ecgate will be activated)
however: no restart of cray.def file, need to login to cdp to change date
After all your boxes turns to yellow your task is completed and you should find your results in the directory specified in $SCMDATA
Download data (more information https://software.ecmwf.int/wiki/display/UDOC/File+Transfer)
e.g. http://ecaccess.ecmwf.int
Browsing → ECscratch files → data → click on symbol to left (page with red arrow) to download
Adapting extraction to other operational systems than current one (from 8 March 2016)
change scripts:
sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray
sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray.dataonly
in the way to modify there (for period June 2013 – March 2016)
CYCLE=40r3
dataid2="t1279_2010"
CLIMVERSION=climate.v011
GTYPE=l_2
and substitute there:
$TMPDIR/run_parallel /home/rd/nag/bin/getini1c_TCo1279
by older script (as used for ERA-I):
$TMPDIR/run_parallel /home/rd/nag/ bin/getini1c
Extracting operational forecasts or analyses further back in time needs different adaptions to the two include_cray/getini1c_setup_cycle* scripts.
Horizontal resolution updates:
21 Nov 2000: T511
1 Feb 2006: T799
26 Jan 2010: T1279
8 March 2916: T2047
Vertical resolution updates:
21 Nov 2000: L60
1 Feb 2006: L91
June 2013: L137
If changing horizontal resolution:
Also change ZDELTA with coarser grid so that point is found in circle around desired location.
Choose large enough but not to large so that point further away is detected.
TSTEP is not used in getini1c script, not as important to set.
If changing vertical resolution:
In addition to LEVELS change option “-l 137” in getini1c_mars_run_validation_SCM_FC call in include_cray/getini1c_setup_cycle_validation_SCM_FC_cray.dataonly according to new number of levels (default 137)
Extraction from ERA-I:
use scripts in directory include_cray/ERA-I instead and also ERAI-cray.def
Data availability:
00: forecast step 0, 3, 6, 9, 12
06: analysis step 0
12: forecast step 0, 3, 6, 9, 12
18: analysis step 0
Acknowledgements
Filip Váña, Nils Wedi, Paul Dando, Kerstin Hartung and Glenn Carver