OpenIFS user workshop 2015

 

Preface

In these exercises we will look at a case study using a forecast ensemble. You will start by studying the evolution of the ECMWF HIRES forecast and the ECMWF ensemble forecast for this event. Then you will run your own OpenIFS forecast for a single ensemble member at lower resolutions and work in groups to study the OpenIFS ensemble forecasts.

Caveat: In practise many more cases need to be run in order to establish correct statistical behaviour of the ensemble.

 

Starting up metview

  • Type the following command in a terminal window:
metview

Recap

Case study

St. Judes storm..... (see separate sheet?)

Key points

  • sources of uncertainty: initial analysis and model error.
  • Initial analysis uncertainty: accounted for by use of Singular Vectors (SV) and Ensemble Data Assimilation (EDA).
  • Model uncertainty: accounted for by use of stochastic processes. In IFS this means Stochastically Perturbed Physical Tendencies (SPPT) and the spectral backscatter scheme (SKEB)

ECMWF operational forecasts consist of:

  • HRES : T1279 (16km grid) highest resolution 10 day forecast
  • ENS : Ensemble (50 members), T639 for days 1-10, T319 days 11-15.

 

Exercise 1. Evaluating the ECMWF forecast

(following approach in metview training course ensemble forecast)

Task 1: Visualise operational forecast

Dates 24th - 29th.

  1. How does HIRES forecast compare to analysis?
  2. How does HIRES forecast compare to observations?

Task 2 : Visualize the ensemble

  1. Visualize ensemble mean
  2. How does the ensemble mean compare to HIRES & analyses?

Task 3 : Visualize ensemble spread

Ensemble spread is defnied as..

  1. Visualize stamp map - are there any members that provide a better forecast?
  2. Visualize spaghetti map - see how members spread over the duration of the forecast.

Exercise 2. Creating an ensemble forecast using OpenIFS

(see separate handout?)

OpenIFS running at T319 (resolution of second leg of ECMWF's forecast ensemble).

Each participant runs one ensemble.

(possibly including Filip's coding exercise here).

At the end of this, participants will have a single member ensemble run with SPPT+SKEB enabled (model error only).

Need steps to process the data for metview - macro or grib tools?

Aim is to understand the impact of these different methods on the ensemble

(point out this is a case study and the correct approach would be to use more cases to get better statistics)

Exercise 3. Verifying / Quantifying OpenIFS forecasts

Experiments available:

  • EDA+SV+SPPT+SKEB : nagc/gbzl in MARS
  • EDA+SV only              : nagc/gc11 in MARS
  • SPPT+SKEB only       : run by participants (available in MARS as gc1d)

These are at T319 with start dates: 24/25/26/27 Oct 00Z for 5 days with 3hrly output.

Tasks

  • Look at ensemble mean and spread for all 3 cases.
  • How does it vary? Which gives the better spread? How does the forecast change with reducing lead time?
  • For this case only, does the forecast improve by including model uncertainty?
  • Find an ensemble member that gives the best forecast and take the difference from the control. Step back to the beginning of the forecast and look to see where the difference originates from.  How does this differ between the 3 OpenIFS runs? (with model uncertainty only, each initial state is identical so differences will develop from 

Ensemble perturbations are applied in positive and negative pairs. For each perturbation computed, the initial fields are CNTL +/- PERT. (need a diagram here)

  • Choose an odd & even ensemble member from one of the 3 OpenIFS forecasts (e.g. members 9 and 10). For different forecast steps, compute difference of each member from the control forecast and then subtract those differences.  (i.e. centre the differences about the control forecast).
  • What is the result? Do you get zero? If not why not? Use Z200 & Z500? MSLP?
  • Repeat looking at one of the other forecasts. How does it vary between the different forecasts?

If time:

  • Plot PV at 330K. What are the differences between the forecast? Upper tropospheric differences played a role in the development of this shallow fast moving cyclone.

Exercise 4. CDF/RMSE at different locations

Concepts to introduce

RMSE & CDF (needs explanation)

Discuss concept of ensemble reliability.

  • Choose 3 locations.e.g. Reading, Amsterdam, Copenhagen.
  • Using 10m wind and/or wind gust data plot CDF & RMSE curves using one of the OpenIFS forecasts and the ECMWF analysis data.
  • What is the difference and why?
  • Repeat for one of the other OpenIFS runs. Are there any differences, if so why?

 

NOTES

These will disappear in the final handout.

 

Need to think how to group some of these activities so that people on a row are working together.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  • MSLP
  • 10m winds
  • T2m
  • Z500, Z200
  • wind gust : model & obs (nb. model wind gust data is accumulated)(Linus has windgust obs in geopoint format)
  • PV (decide which levels)

  • layout
  • contour / colourfill (overlay)
  • wind vector
  • spaghetti plots
  • stamp maps
  • rmse & cdf at several locations (user chooses)(Linus has macros for these)
  • Difference maps : to plot fc - an and ens member - control ( or ens member(i) - ensemble member(j) )
  • step animation of spaghetti plots etc to see spread developing.
  • Linus' vortex centre & tracking plot

Retrieve data from MARS for all apart from the OpenIFS experiment the participants will run themselves.

Note that operational ensemble runs at T319 are also available if we want to use them to compare 40r1 with OpenIFS? (see Linux for MARS script)

wind gust data is not simply max windspeed over 6hr period, it includes convective component (as well as conversion from windspeed to gust speed).

 

 

 

 

 

 

Question. How best to organise the experiments?  Each user has an account or use one account with multiple directories?

Linus suggested running a script that reorders the data to have 1 file with all ensemble members for each field of interest. Do this after all members have run?

Suggests doing 2 more runs: EDA only and SV only. The perturbations from these have different characteristics and as we don't yet know what the results look like it might be useful to have these runs.

(I may not have all the details right - check with Linus who apparently did a PhD thesis on this)

Relate the perturbations to the baroclinic zones (ie. large scale flow). While the cyclone is developing the baroclinicity will be high and the EDA/SV perturbations will be more significant than when the storm is more developed with fronts when it will be more stable.

Should see that SV are quite artificial, some pertubations will grow rapidly and then die out.  By contrast the perturbations in the EDA will be larger in the initial conditions.

Maybe we should plot the Eady index?

Erland says very important to plot these and see what we get in order to refine the questions and direction we want participants to take.

 

 

 

Linus explained that with the OpenIFS runs will have differing amounts of uncertainty, so the spread should noticeably change for points near the track in the analysis. This is particularly because of (a) timing error between the analysis & fc, (b) the ensemble tracks being more to the north of the analysis track. So Amsterdam for instance should see much less spread as the uncertainty in the ensemble is reduced.

Parameters that do not have a Gaussian like distribution in the ensemble can be problematic.