What's a model restart?
A restart allows the model to continue a forecast with a succession of individual runs, each one starting where the other one left off. e.g. running an extended forecast of 100 days with 5 separate runs of 20 days each. A restart will always be exact, that is, the results of the restarted forecast should always match a single run of the model for the same total length (assuming all settings remain the same).
This page describes how to configure and use the model restart facility.
n.b. the restarted run must always have the same parallel decomposition (TASKS x THREADS)
How to configure restarts
The namelist NAMRES controls model restarts (see ifs/namelist/namres.h). The most useful variables in this namelist are: NFFRES and NRESTS.
NFRRES : frequency of restart file writes (restart file interval). Use this to set a regular restart frequency. e.g. restart files created every 24hrs.
This can be either positive or negative.
If positive it represents a frequency in model timesteps. e.g. NFRRES=144 means a restart file will be created every 144 model timesteps. With a timestep of 10mins this would mean restart files are written every day.
If negative it represents a frequency in hours. e.g. NFRRES=-48, would mean the model creates restart files every 2 days regardless of the model timestep.
NRESTS : list of restart times. Use this to set an irregular restart. e.g. restart file created at 24hrs, 72hrs, 144hrs.
Values for NRESTS can be either positive in which case they are interpreted as the model timesteps at which a restart file is made, or negative in which case they specify hours.
The first value for NRESTS must indicate the number of valid entries in the
Examples
&NAMRES
NFRRES=-24,
/
In this trivial example, the model will write restart files every 24hrs (as NFRRES is negative). NRESTS does not need to be specified (defaults to zero).
&NAMRES
NFRRES=1,
NRESTS=-3,-48,-120,-192,
/
In this example, the first entry (-3) is negative so all times are in hours (which are also negative). It also indicates 3 restart write times are requested. The restart files will be created at 48hrs, 120hrs and 192hrs. Note the restart files do not need to be created at equally spaced intervals if using a list of restart times in NRESTS. The value of NFRRES is normally set to 1 in this case. If NFRRES is set > 1 it's assumed to .
Files created
Restart files
The output files all begin with the name 'srf'. One file will be created per MPI task. The files are written as unformatted binary (not GRIB) in order to preserve precision.
The file name includes a date as : srfddddhhmm, where dddd is the day number of the run, hh is the hour and mm is the minutes. e.g. srf00000120.0002 would be for day 1 and 20mins into the run written from MPI task 2.
By default, old restart files are automatically deleted. This is advisable because the files can be very large. One restart file at T1279 is approx 500Mb when using 128 MPI tasks, giving a total restart file requirement of approx 64Gb. Changing this behaviour is not possible via NAMRES. Instead, edit the file ifs/setup/sures.F90 and change the default value of LDELRES to TRUE.
Restart namelist
A file, 'rcf' will be created by the model at the point when the restart files are written. This file contains the NAMELIST that informs the model what it needs to know to restart the model.
Do not delete or edit this file, otherwise the model will be unable to restart - regardless of whether the actual restart files (those beginning with srf) are present.
Conversely, if you don't want to run a restart but want to repeat the run, do delete the rcf file (and the srf files). If you don't the model will attempt to continue the run according to the namelist in the rcf file.
Model code
The key subroutines for restarts are:
monio.F90 - sets up the internal arrays to determine write times.
wrresf.F90 - calls the I/O subsystem to write out the restart files. If you add more arrays to the model and want them to appear in restart files, change this routine.
reresf.F90 - calls the I/O subsystem to read the restart files. Counterpart to wrresf.F90. Any changes to wrresf must be mirrored by changes to reresf.F90.