Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Section
Column

Introduction

In these exercises we will look at a case study of a severe storm using a forecast ensemble. During the course of the exercise, we will explore the scientific rationale for using ensembles, how they are constructed and how ensemble forecasts can be visualised. A key question is how uncertainty from the initial data and the model parametrizations impact the forecast.

You will start by studying the evolution of the ECMWF analyses and forecasts for this event. You will then run your own OpenIFS forecast for a single ensemble member at lower resolutions and work in group to study the OpenIFS ensemble forecasts.

 

Note
titleCaveat

In practise many cases are aggregated in order to evaluate the forecast behaviour of the ensemble. However, it is always useful to complement such assessments with case studies of extreme events, like the one in this exercise, to get a more complete picture of IFS performance and identify weaker aspects that need further exploration.

Column
width25%
Panel
titleTable of contents

Table of Contents
maxLevel1

Recap

ECMWF operational forecasts consist of:

...

Right-click the mouse button on the 'an_1x1.mv' icon and select the 'Visualise' menu item (see figure right)

...

Right-click the mouse button on the 'an_1x1.mv' icon and select the 'Edit' menu item (see figure right).

...

If you prefer to see multiple plots per page rather than overlay them, please use the an_2x2.mv macro.

Info

That completes the first exercise.

...

Additional plotting tips.

For the an_2x2.mv icon the number of maps appearing in the plot layout can be 1, 2, 3 or 4.  This is true of all the icons labelled '2x2'.

an_2x2.mv demonstrates how to plot a four-map layout in a similar fashion to the one-map layout. The only difference here is that you need to define four plots instead of one.

...

Panel

For this exercise, you will use the metview icons in the row labelled 'HRES forecast' as shown above.

hres_rmse.mv                                mv                                : this plots the root-mean-square-error growth curves for the operational HRES forecast for the different lead times.

hres_1x1.mv & hres_2x2.mv    : these work in a similar way to the same icons used in the previous task where parameters from a single lead time can be plotted.

hres_to_an_runs.mv                     mv                     : this plots the same parameter from the different forecasts for the same verifying time. Use this to understand how the forecasts differed, particularly for the later forecasts closer to the event.

hres_to_an_diff.mv                       mv                       : this plots a single parameter as a difference between the operational HRES forecast and the ECMWF analysis. Use this to understand the forecast errors from the different lead times.

 

Parameters & map appearance. These macros have the same choice of parameters to plot and same choice of mapType, either the Atlantic sector or over Europe.

...

In this task, all 4 forecast dates will be used.

Using the hres_rmse.mv icon, right-click, select 'Edit' and plot the RMSE curves for MSLP (mean-sea-level pressure). Repeat for the 10m wind-gust parameter wgust10.

...

Task 2: Compare forecast to analysis

a) Use the hres_to_an_runs.mv icon (right-click -> Edit) and plot the MSLP and wind fields. This shows a comparison of 3 of the 4 forecasts to the analysis (the macro can be edited to change the choice of forecasts).

b) Use the hres_to_an_diff.mv icon and plot the difference map between a forecast date and the analysis.

...

Panel

For these exercises please use the Metview icons in the row labelled 'ENS'.

ens_rmse.mv : this is similar to the oper_rmse.mv in the previous exercise. It will plot the root-mean-square-error growth for the ensemble forecasts.

ens_to_an.mv : this will plot (a) the mean of the ensemble forecast, (b) the ensemble spread, (c) the HRES deterministic forecast and (d) the analysis for the same date.

ens_to_an_runs_spag.mv : this plots a 'spaghetti map' for a given parameter for the ensemble forecasts compared to the analysis. Another way of visualizing ensemble spread.

stamp.mv : this plots all of the ensemble forecasts for a particular field and lead time. Each forecast is shown in a stamp sized map. Very useful for a quick visual inspection of each ensemble forecast.

stamp_diff.mv : similar to stamp.mv except that for each forecast it plots a difference map from the analysis. Very useful for quick visual inspection of the forecast differences of each ensemble forecast.

 

Additional plots for further analysis:

pf_to_cf_diff.mv : this useful macro allows two individual ensemble forecasts to be compared to the control forecast. As well as plotting the forecasts from the members, it also shows a difference map for each.

ens_to_an_diff.mv : this will plot the difference between an ensemble forecast member and the analysis for a given parameter.

...

Panel
borderStyle
bgColorwhite
titleBGColorwhite
borderStyledotted
titleStorm track printed handout: ens_operdotted

Please refer to the handout showing the storm tracks labelled 'ens_oper' during this exercise. It is provided for reference and may assist interpreting the plots.

Each page shows 4 plots, one for each starting forecast lead time. The position of the symbols represents the centre of the storm valid 28th Oct 2013 12UTC. The colour of the symbols is the central pressure.

The actual track of the storm from the analysis is shown as the red curve with the position at 28th 12Z highlighted as the hour glass symbol. The HRES forecast for the ensemble is shown as the green curve and square symbol. The lines show the 12hr track of the storm; 6hrs either side of the symbol.

Note the propagation speed and direction of the storm tracks.  The plot also shows the centres of the barotropic low to the North.

Q. What can be deduced about the forecast from these plots?

Info

The plots in the handout can also be found in the 'pics' folder.

...

This is similar to task 1 in exercise 2, except now the RMSE curves for all the ensemble members from a particular forecast will be plotted. All 4 forecast dates are shown.

Using the ens_rmse.mv icon, right-click, select 'Edit' and plot the curves for 'mslp' and 'wgust10'. Note this is only for the European region. The option to plot over the larger geographical region is not available.

...

Task 5:  Cumulative distribution function at different locations

Recap

The probability distribution function of the normal distribution
or Gaussian distribution. The probabilities expressed as a
percentage for various widths of standard deviations (σ)
represent the area under the curve.

Image Modified

Figure from Wikipedia.

Cumulative distribution function for a normal
distribution with varying standard deviation (σ)

Image Modified

Figure from Wikipedia.

Cumulative distribution function (CDF)

...

Panel
titleForecast for the Queen
Her Majesty The Queen has invited Royals from all over Europe to a garden party at Windsor castle on 28th October 2013 (~20 miles west of London). Your team is responsible for the weather prediction and decision to have an outdoor party for this event. You have 3-4 days to plan for the event. You can use the data and macros provided to you to first derive probabilities of severe weather at this location (this doesn't need to be exact so use the information for Reading). 

i) What would your prediction for the probability of winds > 10m/s (~20 mph) ; > 30m25m/s (~45 mph) ; > 25m30m/s (~67 mph)?
ii) The price of ordering the marquees and outdoor catering for the event is £100,000. Chances of the marquees falling apart when winds > 10m/s = 20% probability ; winds > 20m/s = 40% ; winds > 30m/s = 80%. Now what would the probabilities of the marquees failing be, given this new information from the rental service and the weather prediction you made?

This type of problem is often discussed in terms of risk, or the idea of the cost/loss ratio of a user. Here the loss (L) would be some financial value that would be at loss if the bad weather forecast event happens and no precautionary actions had been taken. The costs (C) would be the financial value associated with precautionary actions in case the event happens. The ratio of the costs to the loss, often called C/L ratio, could then be used for decision making by users. If the costs are substantially smaller than the potential loss, then already a relatively small forecast probability for the event would indicate to take precautionary actions. Whereas for large C/L this is less, or not at all, the case.

...

Panel
bgColorwhite
titleBGColorlightlightgrey
titleAvailable plot types

 The different macros available for this exercise are very similar to those in previous exercises.

Image Modified

For this exercise, use the icons in the row labelled 'Experiments'. These work in a similar way to the previous exercises.

ens_exps_rmse.mv     : this will produce RMSE plumes for all the above experiments and the operational ensemble.

ens_exps_to_an.mv   : this produces 4 plots showing the ensemble spread from the OpenIFS experiments compared to the analysis.

ens_exps_to_an_spag.mv : this will produce spaghetti maps for a particular parameter contour value compared to the analysis.

ens_part_to_all.mv     : this allows the spread & mean of a subset of the ensemble members to be compared to the whole ensemble.

Note that the larger geographical area, mapType=1, will only work for MSLP due to data volume restrictions.

...

We gratefully acknowledge the following for their contributions in preparing these exercises, in particular from ECMWF: Glenn Carver, Sandor Kertesz, Linus Magnusson, Martin Leutbecher, Iain Russell, Filip Vana, Erland Kallen. From University of Oxford: Aneesh Subramanian, Peter Dueben, Peter Watson, Hannah Christensen, Antje Weisheimer.

  

...

Excerpt Include
Credits
Credits
nopaneltrue