Over climatological runtimes OIFS 43r1v1 has stack memory and runtime growth problems. Shown is a 120 year long EC-Earth 4 simulation. The issue was reproduced with standalone OIFS.
A solution targeting the root problem is in the workings. Until then I present a workaround to modify the rcf and waminfo restart timesteps such that the model always sees itself being close to the inital date. To move the model forward in time with regards to forcing and output dates we then change the inital date from leg to leg.
RCF
The restart control file for OIFS contains the timestep and hour of the restart time.
&NAMRCF CSTEP = " 8760", CTIME = "000003650000 ",
To reset the timestep back down starting from the third restart we use:
CSTEP=$(printf $(($(($(($DAYS_SINCE_INITIAL_LAST_RUN_oifs+1))*86400))/$TIME_STEP_oifs)))
CTIME=$(printf '%08d' $(($DAYS_SINCE_INITIAL_LAST_RUN_oifs+1)))0000
sed -i `grep -n "CSTEP" ${WORK_DIR}/rcf|cut -d ':' -f 1`"c \ CSTEP = \" $CSTEP\"\," ${WORK_DIR}/rcf
sed -i `grep -n "CTIME" ${WORK_DIR}/rcf|cut -d ':' -f 1`"c \ CTIME = \"$(printf '%-9s' $CTIME) \"\," ${WORK_DIR}/rcf
waminfo
The restart control file for the WAM submodel contains two lines that need to be modified:
ANALYSIS FROM 19900101000000 TO 19900101000000 BEGIN DATE FOR USING SURFACE CURRENT = 19900101000000
and we do so with
sed -i `grep -n "ANALYSIS FROM" ${WORK_DIR}/waminfo|cut -d ':' -f 1`"c ANALYSIS FROM ${INI_DATE_oifs}000000 TO ${INI_DATE_oifs}000000" ${WORK_DIR}/waminfo
sed -i `grep -n "BEGIN DATE" ${WORK_DIR}/waminfo|cut -d ':' -f 1`"c BEGIN DATE FOR USING SURFACE CURRENT = ${INI_DATE_oifs}000000" ${WORK_DIR}/waminfo
updating inital date
Now that we are resetting the restart timesteps in the restart control file we can go ahead and change the inital date of the run
if [[ "x${RUN_NUMBER_oifs}" = "x1" ]]; then
continue
else
# We modify the input files to alter the initial date after a restart. This way we can stay away from large fields allocated with the restart timestep.
# Inital time in wam_namelist
sed -i `grep -n "CBPLTDT" wam_namelist|cut -d ':' -f 1`"c \ CBPLTDT = \"${INI_DATE_oifs}000000\"\," wam_namelist
sed -i `grep -n "CDATECURA" wam_namelist|cut -d ':' -f 1`"c \ CDATECURA = \"${INI_DATE_oifs}000000\"\," wam_namelist
sed -i `grep -n "CDATEF" wam_namelist|cut -d ':' -f 1`"c \ CDATEF = \"${INI_DATE_oifs}000000\"\," wam_namelist
# Initial time in OIFS INIT files
grib_set -s dataDate=${INI_DATE_oifs} ICMGG${EXP_ID}INIT ICMGG${EXP_ID}INIT_o &
grib_set -s dataDate=${INI_DATE_oifs} ICMSH${EXP_ID}INIT ICMSH${EXP_ID}INIT_o &
grib_set -s dataDate=${INI_DATE_oifs} ICMGG${EXP_ID}INIUA ICMGG${EXP_ID}INIUA_o &
wait
mv ICMGG${EXP_ID}INIT_o ICMGG${EXP_ID}INIT &
mv ICMSH${EXP_ID}INIT_o ICMSH${EXP_ID}INIT &
mv ICMGG${EXP_ID}INIUA_o ICMGG${EXP_ID}INIUA &
wait
# Inital time in runscript
sed -i `grep -n "INITIAL_DATE_oifs" ../scripts/${scriptname}|cut -d ':' -f 1`"c INITIAL_DATE_oifs=${END_DATE_oifs}" ../scripts/$scriptname
fi
Renaming restart files
To complete the set of changes we rename the restart files while we link them into the work folder such that their names match up with the new restart timesteps.
if files=$(ls ${INI_RESTART_DIR_oifs_protect}/srf????????????.???? 2>/dev/null); then
for f in $files; do
file=$(basename $f)
echo " Link file $file "
out=${file:16}
in="${WORK_DIR}/srf$(printf "%08d" $(($DAYS_SINCE_INITIAL_LAST_RUN_oifs+1)))0000."
out2=$in$out
ln -s $f $out2
done
else
I should do the same for the wam LAW and BLS files, but chronologically I changed those first and when with an inferior approach whereby I simply removed the timestamp from their names:
diff --git a/src/wam/Wam_oper/grstname.F b/src/wam/Wam_oper/grstname.F
index ea1f243..abe282d 100644
--- a/src/wam/Wam_oper/grstname.F
+++ b/src/wam/Wam_oper/grstname.F
@@ -70,7 +70,7 @@
INTEGER :: IDDDDDD, IHH, IMIMI, ISS
CHARACTER(LEN=14) :: CDATEH, CDT
- CHARACTER(LEN=30) :: FILENA
+ CHARACTER(LEN=10) :: FILENA
!* 1.0 CONSTRUCT FILE NAME ALONE
! -------------------------
@@ -81,41 +81,6 @@
IMAXYEAR=(2**(8*IREP-1)-1)/IYEARSEC
FILENA = FILEID
- IF (CDATE.LE.CDATEF) THEN
- CDATEH = CDATE
- ISHIFT = 0
- ISHIFTDAY=0
- ELSE
- CDATEH = CDATEF
-
- READ(CDATEF,'(I4, 5I2)')IYEAR1,IMON1,IDAY1,IHOUR1,IMIN1,ISEC1
- READ(CDATE ,'(I4, 5I2)')IYEAR2,IMON2,IDAY2,IHOUR2,IMIN2,ISEC2
- ISHIFTDAY=0
- CDT=CDATE
-! IN CASE THE DATE DIFFERENCE IS LARGER THAN WHAT DIFDATE CAN DO.
- IYRDIF=IYEAR2-IYEAR1
- IF(ABS(IYRDIF).GE.IMAXYEAR) THEN
- NCHUNCK=ABS(IYRDIF)/IMAXYEAR
- DO IC=1,NCHUNCK
- ISFT=-SIGN(1,IYRDIF)*IMAXYEAR*IYEARSEC
- CALL INCDATE(CDT,ISFT)
- ISHIFTDAY=ISHIFTDAY-ISFT/86400
- ENDDO
- ENDIF
-
- CALL DIFDATE (CDATEF, CDT, ISHIFT)
- ENDIF
- FILENA(4:17) = CDATEH
- FILENA(18:18) = "_"
-
- IDDDDDD=ISHIFT/86400
- IHH=(ISHIFT-IDDDDDD*86400)/3600
- IMIMI=(ISHIFT-IDDDDDD*86400-IHH*3600)/60
- ISS=ISHIFT-IDDDDDD*86400-IHH*3600-IMIMI*60
-
- IDDDDDD=IDDDDDD+ISHIFTDAY
-
- WRITE (FILENA(19:30),'(I6.6,3I2.2)')IDDDDDD,IHH,IMIMI,ISS
LNAME = LEN_TRIM(FILENA)
If you are doing something similar I suggest not to do it this way, since the restart files loose all indication of which timestamp they are from beside the name of the folder you might copy them into.
Although not elegant the changes made resolved the problem shown in the plots above. Them model continues to be in leg two with the inital time moving forward. All forcing and output operations proceed as normal.
Cheers, Jan
3 Comments
Unknown User (nagc)
Thanks Jan, it's a good example of some of the inner workings.
Jan Streffing
Since there is some interest in this from EC-Earth side: Note that the WAM section is out of date. AWI and GEOMAR now treat the WAM restart files in a very similar fashion to IFS restart files.
A more refined processing of the files can be found here:
https://github.com/esm-tools/esm_tools/blob/feature/awicm3_prep_merge_geomar/configs/components/oifs/change_icm_date.sh
and here:
https://github.com/esm-tools/esm_tools/blob/feature/awicm3_prep_merge_geomar/configs/components/oifs/change_rcf_date.sh
Etienne Tourigny
Thanks Jan. I think other OpenIFS users (e.g. OpenIFS/CC) will be interested in this topic.