...
Section | ||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Note | ||
---|---|---|
| ||
In practise many cases are aggregated in order to evaluate the forecast behaviour of the ensemble. However, it is always useful to complement such assessments with case studies of individual events, like the one in this exercise, to get a more complete picture of IFS performance and identify weaker aspects that need further exploration. |
...
- HRES : spectral T1279 (16km grid) highest resolution 10 day deterministic forecast.
- ENS : spectral T639 (34km 31km grid) resolution ensemble forecast (50 members) is run for days 1-10 of the forecast, T319 (70km) is run for days 11-15.
In 2016, the ECMWF operational forecasts has been was upgraded compared to 2012 and consisted of:
...
These exercises use a relatively large domain with high resolution data. Some of the plotting options can therefore require significant amounts of memory. If the virtual machine freezes when running metviewMetview, please try increasing the memory assigned to the VM.
...
Info |
---|
Please enter the folder 'openifs_2018' to begin working. |
Saving images and
...
animations
Info |
---|
The macros described in this tutorial can write PostScript and GIF image files to the To save the images, use the 'Execute' menu option on the icon, rather than 'Visualise'. The 'okular' command can be used to view the PDF & gif images. |
To save any other images during these exercises for discussion later, you can either use:
"
To save images during these exercises for discussion later, you can either use:
"Export" button in Metview's display window under the 'File' menu to save to PNG image format. This will also allow animations to be saved into postscript.
or use the following ksnapshot
command to take a 'snapshot' of the screen :
Code Block | ||||
---|---|---|---|---|
| ||||
ksnapshot |
Convert postscript to animated GIF
and save it to a file.
If you want to create animations from other imagesIf you want to create animations, save the figures as postscript and then use the convert
command:
...
Begin by entering the folder labelled 'Analysis':
Task 1: Mean-sea-level pressure and track
This task will look at the synoptic development of Hurricane Nadine and the cutoff low up to 00Z, 20th September 2012. The forecasts in the next exercises start from this time and date.
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
an_1x1.mv : this plots horizontal maps of parameters from the ECMWF analyses overlaid on one plot. an_2x2.mv : this plots horizontal maps of parameters from the ECMWF analyses four plots to a page (two by two). |
Right-click on the an_
...
1x1.mv
...
Task 1: Mean-sea-level pressure and track
Right-click on the 'an_1x1.mv' icon and select the 'Visualise' menu item (see figure icon and select the 'Visualise' menu item (see figure)
After a pause, this will generate a map showing mean-sea-level pressure (MSLP).
Drag and drop the mv_track.mv
icon onto the map to add the track of Hurricane Nadine.
In the plot window, use the play button in the animation controls
to animate the map and follow the development and track of Hurricane Nadine....
Info |
---|
If the contour lines appear jagged, in the plot window, select the menu item 'Tools -> Antialias'. |
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Q. What is unusual about Hurricane Nadine? |
Warning | ||
---|---|---|
| ||
Please close any unused plot windows if using a virtual machine. This case study uses high resolution data over a relatively large domain. Multiple plot windows can therefore require significant amounts of computer memory which can be a problem for virtual machines with restricted memory. |
Task 2: MSLP and 500hPa geopotential height
...
Right-click the mouse button on the 'an_1x1.mv
' icon and select the 'Edit' menu item.
...
The surface fields (single level) are: mslp (mean-sea-level-pressure), t2 (2-metre temperature), wind10 (10-metre wind arrows), speed10 (wind-speed at 10m : sqrt(u^2+v^2)), sst (sea-surface temperature).
The upper level fields are: t (temperature), z (geopotential), pt (potential temperature), eqpt (equivalent potential temperature), wind (wind arrows), speed (wind-speed as contours), r (relative humidity).
These The upper level fields have a list of available pressure levels in square brackets.
...
Panel | |||||||
---|---|---|---|---|---|---|---|
| |||||||
With the edit window open for an_1x1.mv, find the line that defines 'plot1':
Change this line to:
The '.s' means plot the 500hPa geopotential as a shaded plot instead of using contours (this style is not available for all fields). Make sure Click the play button Click the play button and then animate the map that appears |
Use the an_1x1.mv
(or the an_2x2.mv
) to plot fields of your choice.
Panel | ||||
---|---|---|---|---|
| ||||
Change the value of
You might add the mslp or z500 fields to this plot e.g.
Note that the fields are plotted in the order specified in the list! |
Panel | ||||
---|---|---|---|---|
| ||||
Q. When does the cut-off low form (see z500)? |
Task 3: Changing the map geographical area
Right-click on 'an_1x1.mv
' icon and select 'Edit'.
In the edit window that appears
...
Task 4: Wind fields, sea-surface temperature (SST)
The 'an_2x2.mv
' icon plots up to 4 separate figures on a single frame.
Right-click on the 'an_2x2.mv
' icon and select the 'Edit' menu item.
...
Panel | ||
---|---|---|
| ||
Q. What do you notice about the SST field? |
Task 5: Satellite images
Open the folder 'satellite' (back in the top level folder 'openifs_2018') (scroll the window if it is not visible).
This folder contains satellite images (water vapour, infra-red, false colour) for 00Z on 20-09-2012 and animations of the infra-red and water vapour images. The animations go beyond the 20-09-2012.
Double click the images to display them.
Use the an_1x1.mv and/or the an_2x2.mv macros to compare the ECMWF analyses with the satellite images.
Task 6: Cross-sections
I think this task should move to the second look at analyses after 20/9?
The last task in this exercise is to look at cross-sections through Hurricane Nadine and the cut-off low.
Right click on the icon 'an_xs.mv', select 'Edit' and push the play button.
The plot shows potential vorticity (PV), wind vectors and potential temperature roughly through the centre of the Hurricane and the cut-off low. The red line on the map of MSLP shows the location of the cross-section.
Panel | ||
---|---|---|
| ||
Q. Look at the PV field, how do the vertical structures of Nadine and the cut-off low differ? |
Changing forecast time
Cross-section data is only available every 24hrs.
This means the 'steps' value in the macros is only valid for the times: [2012-09-20 00:00], [2012-09-21 00:00], [2012-09-22 00:00], [2012-09-23 00:00], [2012-09-24 00:00], [2012-09-25 00:00]
To change the date/time of the plot, edit the macro and change the line:
Code Block | ||
---|---|---|
| ||
steps=[2012-09-22 00:00] |
Changing fields
A reduced number of fields is available for cross-sections: temperature (t), potential temperature (pt), relative humidity (r), potential vorticity (pv), vertical velocity (w), wind-speed (speed; sqrt(u*u+v*v)) and wind vectors (wind3).
Changing cross-section location
Code Block | ||
---|---|---|
| ||
#Cross section line [ South, West, North, East ]
line = [30,-29,45,-15] |
The cross-section location (red line) can be changed in this macro by defining the end points of the line as shown above.
Remember that if the forecast time is changed, the storm centres will move and the cross-section line will need to be repositioned to follow specific features. This is not computed automatically, but must be changed by altering the coordinates above.
Exercise 2: The operational ECMWF HRES forecast
Forecast performance
Exercise 1 looked at the synoptic development up to the 20-Sept-2012. This exercise looks at the ECMWF HRES forecast from this date and how the IFS model developed the interaction between Hurricane Nadine and the cut-off low.
Enter the folder 'HRES_forecast'
in the 'openifs_2018'
folder to begin.
Recap
The ECMWF operational deterministic forecast is called HRES. At the time of this case study, the model ran with a spectral resolution of T1279, equivalent to 16km grid spacing.
Only a single forecast is run at this resolution as the computational resources required are demanding. The ensemble forecasts are run at a lower resolution.
Before looking at the ensemble forecasts, first understand the performance of the operational HRES forecast of the time.
Available forecast
Data is provided for a single 5 day forecast starting from 20th Sept 2012, as used in the paper by Pantillon et al. referred to at the start of this tutorial.
Data is provided at the same resolution as the operational model, in order to give the best representation of the Hurricane and cut-off low iterations. This may mean that some plotting will be slow.
Available parameters
A new parameter is total precipitation : tp.
The parameters available in the analyses are also available in the forecast data.
Available plot types
Panel |
---|
For this exercise, you will use the metview icons in the folder ' |
Task 1: Synoptic development
Task 3: Precipitation over France
This task produces a plot similar to Figure 2 in Pantillon et al.
Choose a hres macro to use, plot the total precipitation (parameter: tp), near surface wind field, relative humidity (and any other parameters of interest).
As for the analyses, the macros hres_1x1.mv, hres_2x2.mv and hres_xs.mv can be used to plot and animate fields or overlays of fields from the HRES forecast.
Panel | ||
---|---|---|
| ||
Q. Was it a good or bad forecast? Why? |
Suggested plots for discussion
The following is a list of parameters and plots that might be useful to produce for later group discussion. Choose a few plots and use both the HRES forecast and the analyses.
For help on how to save images, see the beginning of this tutorial.
Panel | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
|
Exercise 2: Operational ECMWF HRES forecast
HRES performance
Exercise 1 looked at the synoptic development up to 20-Sept-2012. This exercise looks at the ECMWF HRES forecast from this date and how the IFS model developed the interaction between Hurricane Nadine and the cut-off low.
Enter the folder 'HRES_forecast'
in the 'openifs_2018'
folder to begin.
Recap
The ECMWF operational deterministic forecast is called HRES. At the time of this case study, the model ran with a spectral resolution of T1279, equivalent to 16km grid spacing.
Only a single forecast is run at this resolution as the computational resources required are demanding. The ensemble forecasts are run at a lower resolution.
Before looking at the ensemble forecasts, first understand the behaviour of the operational HRES forecast of the time.
Available forecast
Data is provided for a single 10 day forecast starting from 20th Sept 2012.
Data is provided at the same resolution as the operational model, in order to give the best representation of the Hurricane and cut-off low iterations. This may mean that some plotting will be slow.
Available parameters
A new parameter is total precipitation : tp.
The parameters available in the analyses are also available in the forecast data.
Available plot types
Panel |
---|
For this exercise, you will use the Metview icons in the folder ' hres_1x1.mv & : for this exercise, this icon can be used to overlay the forecast track of Nadine (and not the track from the analyses as in Exercise 1) |
Task 1: Synoptic development (day 0-5)
Study the forecast scenario to day+5, focus on:
- the evolution of Nadine,
- the fate of the cutoff low,
- the precipitation over France.
You can use the hres_1x1.mv
and hres_2x2.mv
icons in the same way as Exercise 1 when looking at the analyses.
Although the macros will animate the data for the whole forecast, for this task concentrate on the forecast for the first 5 days.
For example, to plot geopotential at 500hPa and MSLP using the hres_1x1.mv
macro, right click, select Edit and put:
Code Block |
---|
plot1=["z500.s","mslp"] |
To add the forecast track of Hurricane Nadine drag and drop the mv_track.mv
icon onto any map.
Precipitation over France
Choose a hres macro (hres_1x1
or hres_2x2
) and plot the total precipitation (parameter: tp), near surface wind field (parameter: wind10), relative humidity (parameter: r).
Change the area to France by setting 'maptype=2'
in the macro script.
Other suggested isobaric maps
Using either the hres_1x1.mv
or hres_2x2.mv
macro plot some of these other maps to study the synoptic development.
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
|
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Q. How strongly does Nadine appear to interact with the cutoff? |
Task 2: Vertical structure and forecast evolution to day 10
This task focuses on the fate of Nadine and examines vertical PV cross-sections of Nadine and the cutoff at different forecast times.
Right-click on the icon 'hres_xs.mv'
icon, select 'Edit' and push the play button.
The plot shows the cross-section for the 22nd Sept., (day 2 of the forecast), for potential vorticity (PV), wind vectors projected onto the plane of the cross-section and potential temperature drawn approximately through the centre of the Hurricane and the cut-off low. The red line on the map of MSLP shows the location of the cross-section.
Panel | ||||
---|---|---|---|---|
| ||||
Q. Look at the PV field, how do the vertical structures of Nadine and the cut-off low differ? |
Changing forecast time
Cross-section data is only available every 24hrs until the 30th Sept 00Z (step 240).
This means the 'steps' value in the macros is only valid for the times: [2012-09-20 00:00], [2012-09-21 00:00], .... and so on to [2012-09-30 00:00].
Change the forecast time to day+6 (26th Sept). Nadine has now intensified as it approaches the coast.
Code Block |
---|
steps=[2012-09-26 00:00] |
To change the forecast length for hres_1x1.mv
and hres_2x2.mv
, right-click, select Edit and change:
Code Block |
---|
fclen=5 |
to
Code Block |
---|
fclen=10 |
Changing cross-section location
Code Block |
---|
#Cross section line [ South, West, North, East ]
line = [30,-29,45,-15] |
The cross-section location (red line) can be changed by editing the end points of the line as shown above.
If the forecast time is changed, the storm centres will move and the cross-section line will need to be repositioned to follow specific features. This is not computed automatically, but must be changed by altering the coordinates above. Use the cursor data icon to find the new position of the line.
Change the forecast time again to day+8 (28th Sept), or a different date if you are interested, relocate and plot the cross-section of Nadine and the low pressure system. Use the hres_1x1.mv
icon from task 1 if you need to follow location of Nadine.
If time, try some of the other vertical cross-sections below.
Panel | ||||
---|---|---|---|---|
| ||||
Q. What changes are there to the vertical structure of Nadine during the forecast? |
Cyclone phase space (CPS) diagrams
An objectively defined cyclone phase space (CPS) is described using the storm-motion-relative thickness asymmetry (symmetric/non-frontal versus asymmetric/frontal) and vertical derivative of horizontal height gradient (cold- versus warm-core structure via the thermal wind relationship). A cyclone's life cycle can then be analyzed within this phase space, providing insight into the cyclone structural evolution.
This allows a classification of cyclone phase, unifying the basic structural description of tropical, extratropical, and hybrid cyclones into a continuum.
Suggestions for other vertical cross-sections
A reduced number of fields is available for cross-sections compared to the isobaric maps: temperature (t), potential temperature (pt), relative humidity (r), potential vorticity (pv), vertical velocity (w), wind-speed (speed; sqrt(u*u+v*v)) and wind vectors (wind3).
Choose from the following (note the cross-section macro hres_xs.mv
uses slightly different names for the parameters).
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Panel | ||||||
| ||||||
|
Exercise 3:
...
Operational ensemble forecasts
Recap
- ECMWF operational ensemble forecasts treat uncertainty in both the initial data and the model.
- Initial analysis uncertainty: sampled by use of Singular Vectors (SV) and Ensemble Data Assimilation (EDA) methods. Singular Vectors are a way of representing the fastest growing modes in the initial state.
- Model uncertainty: sampled by use of stochastic parametrizations. In IFS this means the 'stochastically perturbed physical tendencies' (SPPT) and the 'spectral backscatter scheme' (SKEB)
- Ensemble mean : the average of all the ensemble members. Where the spread is high, small scale features can be smoothed out in the ensemble mean.
- Ensemble spread : the standard deviation of the ensemble members, represents how different the members are from the ensemble mean.
...
In this case study, there are two operational ensemble datasets and additional datasets created with the OpenIFS model, running at lower resolution, where the initial and model uncertainty are switched off in turn. The OpenIFS ensembles are discussed in more detail in later exercises, one from the original 2012 operational forecast, the other from a reforecast of the event using the 2016 operational ensemble.
An ensemble forecast consists of:
...
ens_oper: This dataset is the operational ensemble from 2012 and was used in the Pantillon et al. publication. A key feature of this ensemble is use of a climatological SST field (you should have seen this in the earlier tasks!).
2016 Operational ensemble
ens_2016: This dataset is a reforecast of the 2012 event using the ECMWF operational ensemble of March 2016.
Two key differences between the 2016 and 2012 operational ensembles are: higher horizontal resolution, and coupling of NEMO ocean model to provide forecast fields of SST (sea-surface temperature) from the start of the forecast.
The analysis was not rerun for 20-Sept-2012. This means the reforecast using the 2016 ensemble will be using the original 2012 analyses. Also only 10 ensemble data assimilation (EDA) members were used in 2012, whereas 25 are in use for 2016 operational ensembles, so each EDA member will be used multiple times for this reforecast. This will impact on the spread and clustering seen in the tasks in this exercise.
Ensemble exercise tasks
Visualising ensemble forecasts can be done in various ways. During this exercise we will use a number of visualisation techniques in order to understand the errors and uncertainties in the forecast,
Key parameters: MSLP and z500. Key parameters: MSLP and z500. We suggest concentrating on viewing these fields. If time, visualize other parameters (e.g. PV320K).
Available plot types
Panel | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
For these exercises please use the Metview icons in the row labelled 'ENS'. ens_rmse.mv : this is similar to the hres_rmse.mv in the previous exercise. It will plot the root-mean-square-error growth for the ensemble forecasts. ens_to_an.mv :
stamp_diff.mv : similar to stamp.mv except that for each forecast it plots a difference map from the analysis. Very useful for quick visual inspection of the forecast differences of each ensemble forecast.
Additional plots for further analysis: pf_to_cf_diff.mv : this useful macro allows two individual ensemble forecasts to be compared to the control forecast. As well as plotting the forecasts from the members, it also shows a difference map for each. Additional plots for further study
|
Group working
If working in groups, each group could follow the tasks below with a different ensemble forecast. e.g. one group uses the 'ens_oper', another group uses 'ens_2016' and so on.
Choose your ensemble dataset by setting the value of 'expId', either 'ens_oper' or 'ens_2016' for this exercise.
One of the OpenIFS ensembles could also be used but it's recommended one of the operational ensembles is studied first.
|
Group working
If working in groups, each group could follow the tasks below with a different ensemble forecast. e.g. one group uses the 'ens_oper', another group uses 'ens_2016'.
Choose your ensemble dataset by setting the value of 'expId', either 'ens_oper' or 'ens_2016' for this exercise.
Code Block | ||||
---|---|---|---|---|
| ||||
#The | ||||
Code Block | ||||
| ||||
#The experiment. Possible values are:
# ens_oper = operational ENS
# ens_2016 = 2016 operational ENS
expId="ens_oper" |
Ensemble forecast
...
uncertainty
In these tasks, the performance of the ensemble forecast is studied.
Panel | ||||
---|---|---|---|---|
| ||||
Q. How does the ensemble mean MSLP and Z500 fields compare to the HRES forecast and analysis? |
Task 1: RMSE "plumes"
This is similar to task 1 in exercise 2, except the RMSE curves for all the ensemble members from a particular forecast will be plotted.
Right-click the ens_rmse.mv icon, select 'Edit' and plot the curves for 'mslp' and 'z500'.
Task 1: Ensemble spread
Use the ens_maps.mv
icon and plot the MSLP and z500. This will produce plots showing: the mean of all the ensemble forecasts, the spread of the ensemble forecasts and the operational HRES deterministic forecast.
Change 'expId'
if required to select either the 2012 ensemble expId="ens_oper"
or the reforecast ensemble expId="ens_2016"
.
Animate this plot to see how the spread grows.
This macro can also be used to look at clusters of ensemble members. It will be used later in the clustering tasks. For this task, make sure all the members of the ensemble are usedChange 'expID' for your choice of ensemble.
Code Block | ||||
---|---|---|---|---|
| ||||
clustersId="off" |
...
| |
#ENS members (use ["all"] or a list of members like [1,2,3]
members=["all"] #[1,2,3,4,5] or ["all"] or ["cl.example.1"] |
Panel | ||
---|---|---|
| ||
Q. How do When does the HRES, ensemble control forecast and ensemble mean compare? |
There might be some evidence of clustering in the ensemble plumes.
There might be some individual forecasts that give a lower RMS error than the control forecast.
If time:
- Explore the plumes from other variables.
- Do you see the same amount of spread in RMSE from other pressure levels in the atmosphere?
Task 2: Ensemble spread
In the previous task, uncertainty in the forecast by starting from different initial conditions and the stochastic parameterizations can result in significant differences in the RMSE (for this particular case and geographical region).
This task will explore the difference in another way by looking at the 'ensemble spread'.
Use the ens_to_an.mv icon and plot the MSLP and z500. This will produce plots showing: the mean of all the ensemble forecasts, the spread of the ensemble forecasts, the operational HRES deterministic forecast and the analysis.
Change 'expId' if required.
Animate this plot to see how the spread grows.
ensemble spread grow the fastest during the forecast? |
Spaghetti plots - another way to visualise spread
A "spaghetti" plot is where a single contour of a parameter is plotted for all ensemble members. It is another way of visualizing the differences between the ensemble members and focussing on features.
Use the ens_to_ref_spag.mv
icon. Plot and animate the MSLP and z500 fields using your suitable choice for the contour level. Find a value that highlights the low pressure centres. Note that not all members may reach the low pressure set by the contour.
The red contour line shows the control forecast of the ensemble.
Note that this may animate slowly because of the computations required.
Experiment with changing the contour value and (if time) plotting other fields.
Task 2: Visualise ensemble members
Stamp maps are used to visualise all the ensemble members as normal maps. These are small, stamp sized contour maps plotted for each ensemble member using a small set of contours.
Use stamp.mv to plot the MSLP and z500 fields in the ensemble.
The stamp map is slow to plot as it reads a lot of data. Rather than animate each forecast step, a particular date can be set by changing the 'steps' variableThis macro can also be used to look at clusters of ensemble members. It will be used later in the clustering tasks. For this task, make sure all the members of the ensemble are used.
Code Block | ||||
---|---|---|---|---|
| ||||
#ENS members (use ["all"] or a list of members like [1,2,3]
members=["all"] #[1,2,3,4,5] or ["all"] or ["cl.example.1"] |
Panel | ||
---|---|---|
| ||
Q. How does the mean of the ensemble forecasts compare to the HRES & analysis? |
Task 3: Spaghetti plots - another way to visualise spread
| |
#Define forecast steps
steps=[2012-09-24 00:00,"to",2012-09-24 00:00,"by",6] |
Make sure clustersId="off
" for this task. Clustering will be used later.
Precipitation over France
Use stamp.mv and plot total precipitation ('tp') over France (mapType=2) for 00Z 24-09-2012.
Panel | ||
---|---|---|
| ||
Q. How much uncertainty is there in the precipitation forecast over southern France? |
Compare ensemble members to the deterministic and control forecast
After visualizing the stamp maps, it can be useful to animate a comparison of individual ensemble members to the HRES and ensemble control deterministic forecasts.
This can help in identifying individual ensemble members that produce a different forecast than the control or HRES forecastA "spaghetti" plot is where a single contour of a parameter is plotted for all ensemble members. It is another way of visualizing the differences between the ensemble members and focussing on features.
Use the ens_to_ref_diff.mv
icon to compare an _runs_spag.mv icon. Plot and animate the MSLP and z500 fields using your suitable choice for the contour level. Find a value that highlights the low pressure centres. Note that not all members may reach the low pressure set by the contour.
The red contour line shows the control forecast of the ensemble.
Note that this macro may animate slowly because of the computations required.
Experiment with changing the contour value and (if time) plotting other fields.
Task 4: Visualise ensemble members and differences
So far we have been looking at reducing the information in some way to visualise the ensemble.
Stamp maps are used to visualise all the ensemble members as normal maps. These are small, stamp sized contour maps plotted for each ensemble member using a small set of contours.
There are two icons to use, stamp.mv and stamp_diff.mv.
Use stamp.mv to plot the MSLP and z500 fields in the ensemble.
The stamp map is slow to plot as it reads a lot of data. Rather than animate each forecast step, a particular date can be set by changing the 'steps' variable.
Code Block | ||||
---|---|---|---|---|
| ||||
#Define forecast steps
steps=[2012-09-24 00:00,"to",2012-09-24 00:00,"by",6] |
Make sure clustersId="off
" for this task.
Precipitation over France
Use stamp.mv and plot total precipitation ('tp') over France (mapType=2) for 00Z 24-09-2012 (compare with Figure 2 in Pantillon).
Note, stamp_diff.mv cannot be used for 'tp' as there is no precipitation data in the analyses.
Difference stamp maps
Use the stamp_diff.mv plot to look at the differences between the ensemble members and the analysis. It can be easier to understand the difference in the ensembles by using difference stamp maps.
Panel | ||
---|---|---|
| ||
Q. Using the stamp and stamp difference maps, study the ensemble. Identify which ensembles produce "better" forecasts. |
Compare ensemble members to analysis
After visualizing the stamp maps, it can be useful to animate a comparison of individual ensemble members to the analyses.
ens_to_an_diff.mv and pf_to_cf_diff.mv can be used to compare ensemble members.
...
title | Use ens_to_an_diff to compare an ensemble member to the analysis |
---|
To animate the difference in MSLP of an individual ensemble member 30 to the analysis, edit the lines:
Code Block |
---|
param="mslp"
ensType="pf30" |
To compare the control forecast:
Code Block |
---|
ensType="cf" |
Further analysis using ensembles
...
title | Use pf_to_cf_diff.mv to compare two ensemble members to the control forecast |
---|
This will show the forecasts from the ensemble members and also their difference with the ensemble control forecast.
To animate the difference in MSLP with ensemble members '30' and '50', set:
Code Block |
---|
param="mslp"
pf=[30,50] |
Panel | ||
---|---|---|
| ||
Compare the SST parameter used for the ens_oper and ens_2016 ensemble forecasts. The 2016 reforecast of this case study used a coupled ocean model unlike the 2012 ensemble and HRES forecast that used climatology for the first 5 days. |
Panel | ||
---|---|---|
| ||
To show a cross-section of a particular ensemble member, use the macro 'ens_xs.mv'. This works in the same way as the an_xs.mv and hres_xs.mv macros. |
...
title | Identifying sensitive region for better forecasts |
---|
Find ensemble members that appear to produce a better forecast and look to see how the initial development in these members differs.
- Select 'better' forecasts using the stamp plots and use ens_to_an.mv to modify the list of ensemble plots.
- Use pf_to_cf_diff and ens_to_an_diff to take the difference between these perturbed ensemble member forecasts from the control and analyses to also look at this.
Panel | ||
---|---|---|
| ||
Q. Can you tell which area is more sensitive for the forecast? |
ensemble member to the HRES forecast. Use pf_to_cf_diff.mv
to compare ensemble members to the control forecast.
Panel | ||||
---|---|---|---|---|
| ||||
To animate the difference in MSLP of an individual ensemble member 30 to the HRES forecast, edit the lines:
and visualise the plot. To compare the control forecast, change:
|
Panel | ||
---|---|---|
| ||
This will show the forecasts from the ensemble members and also their difference with the ensemble control forecast. To animate the difference in MSLP with ensemble members '30' and '50', set:
|
Panel | ||||
---|---|---|---|---|
| ||||
Compare the control forecast scenario to the HRES: Q. Try to identify ensemble members which are the closest and furthest to the HRES forecast. |
Sea-surface temperature
Compare the SST parameter used for the ens_oper and ens_2016 ensemble forecasts. The 2016 reforecast of this case study used a coupled ocean model unlike the 2012 ensemble and HRES forecast that used climatology for the first 5 days.
Panel | ||
---|---|---|
| ||
Q. What is different about SST between the two ensemble forecasts? |
Cross-sections of an ensemble member
To show a cross-section of a particular ensemble member, use the macro ens_xs.mv.
This works in the same way as the hres_xs.mv
macros.
Exercise 4: CDF, percentiles and probabilities
Recap
Figure 1. The probability distribution function of the normal distribution or Gaussian distribution. The probabilities expressed as a percentage for various widths of standard deviations (σ) represent the area under the curve. |
---|
Figure from Wikipedia. |
Figure 2. Cumulative distribution function for a normal |
---|
Figure from Wikipedia. |
Cumulative distribution function
The figures above illustrate the relationship between a normal distribution and its associated cumulative distribution function. The CDF is constructed from the area under the probability density function.
The CDF gives the probability that a value on the curve will be found to have a value less than or equal to the corresponding value on the x-axis. For example, in the figure, the probability for values less than or equal to X=0 is 50%.
The shape of the CDF curve is related to the shape of the normal distribution. The width of the CDF curve is directly related to the value of the standard deviation of the probability distribution function.
For an ensemble, the width is therefore related to the 'ensemble spread'. For a forecast ensemble where all values were the same, the CDF would be a vertical straight line.
Not all parameters will have a Gaussian distribution in values from the ensemble. This will be apparent in the exercises below.
Percentiles and probabilities
For a specified location, the CDF gives the probability that the parameter (for example, total precipitation) is below or equal to the percentile value, p, from the ensemble forecast. This means that the probability of the precipitation being above the value is 1-p.
A probability map then shows the spatial distribution of the precipitation exceeding a specific threshold, p, for example, a map showing the probability that the precipitation will exceed a threshold of 20mm in a 6hr period. A percentile map is very similar and shows the spatial distribution of the given percentile, for example, a map of total precipitation (in mm) for a percentile of 95%.
In the next tasks, we will look at probabilities of the total precipitation in different ways, highlighting the differences between the two forecast ensembles.
Task 1: Plot probabilities and percentiles of total precipitation
Enter the folder Probabilities in the openifs_2018 folder.
The prob_tp_compare.mv
icon will produce maps over France showing the probability that total 6-hourly precipitation exceeds a threshold expressed in mm, for both the 2012 and 2016 forecast ensembles.
Maps are produced for 3 forecast times: +90, +96 and +102 hours.
Edit prob_tp_compare.mv
and set the probability to 10mm:
Code Block | ||
---|---|---|
| ||
#The probability of precipitation greater than (mm)
prob=10 |
Leave the location
as an empty string for now:
Code Block |
---|
location="" |
Run the macro and view the map.
Panel | ||||
---|---|---|---|---|
| ||||
Q. Where are the highest rainfall areas? |
Location for CDF
Using the probability map, click the cursor data icon and move the pointer over the map for +96h and choose a location in the region of highest rainfall. Do this for both the 2012 and 2016 ensemble map.
Make a note of the latitude and longitude coordinates. The highest rainfall area was approximately over the Cévennes mountains ( 44°25′34″N 03°44′21″E ).
Edit prob_tp_compare.mv
and set the location:
Code Block |
---|
location=[44.0,4.1] # [ lat, lon ] -- use your own values! |
and replot the map. A small purple dot will appear at the location specified. If the dot is not in the right location, change it and replot.
Probabilities
Using the plotted probability map for 10mm precipitation threshold, use the cursor data icon to read the probability at the chosen location for +96 hours. Make a note of this value.
Edit prob_tp_compare.mv
, and change the threshold value to 20mm:
Code Block |
---|
prob=20 |
Replot the map and make a note of the probability at your chosen location.
Finally change the threshold probability to 30mm and replot:
Code Block |
---|
prob=30 |
At your chosen location, using the cursor data icon, make a note of the probability for the 30mm threshold values.
You should now have the probability values that total precipitation will exceed 10mm, 20mm and 30mm, for both the 2012 and 2016 ensembles, for forecast time +96 hours for your chosen location.
Task 2: Plot the CDF
This exercise uses the cdf.mv icon.
Right-click, select 'Edit' and then plot a CDF for your location chosen in Task 1 for the 2012 ensemble forecast:
Code Block | ||
---|---|---|
| ||
param="tp"
station=[44.0,4.0] # !use your own values!
expID="ens_oper" |
Make sure the steps value is set correctly to +96 hours (00Z 24th Sept):
Code Block |
---|
steps=[2012-09-24 00:00,"to",2012-09-24 00:00,"by",6] |
Make sure useClusters='off'.
Do the same for the 2016 operational ensemble reforecast:
Code Block |
---|
expID="ens_2016" |
Compare the CDF from the different forecast ensembles.
Panel | ||
---|---|---|
| ||
Q. What can you say about the spread? |
Compare with probability map values
Using the CDF graph for the 2012 ensemble, read the probability that total precipitation will exceed 10mm. For example, see what percentile value, p, is indicated on the y-axis for x=10mm. The probability that total precipitation exceeds this value is then 100-p.
The value read from the CDF graph in this way should agree with the value you obtained by reading the probability value from the map in Task 1.
Check your probabilities for 20mm and 30mm total precipitation.
Panel | ||||
---|---|---|---|---|
| ||||
Q. Do your probabilities read from the 2012 and 2016 maps of total precipitation in Task 1, agree with values from the CDF curves? |
The values may not match exactly as the number of samples (ensembles forecasts in this case) is limited.
Task 3. Plot percentiles of total precipitation
To further compare the 2012 and 2016 ensemble forecasts, plots showing the percentile amount above a threshold can be made for total precipitation.
These can also be compared to the CDF curves from Task 2.
As before, this will use the 6-hourly total precipitation for forecast steps at 90, 96 and 102 hours, plotted over France.
Edit the percentile_tp_compare.mv
icon.
Set the percentile for the total precipitation to 70% and specify the location as in Tasks 1 & 2:
Code Block | ||
---|---|---|
| ||
#The percentile of ENS precipitation forecast
perc=70
location=[44.0,4.1] # [ lat, lon ] -- use your own values! |
Plot the map. It is very similar to the probability map but now shows precipitation values (in mm) for the specified percentile.
From the CDF graph, read the percentile value of 70% on the y-axis and find the total precipitation value indicated on the x-axis.
Use the cursor data icon on the map, as before, and confirm the CDF value agrees with the value at the location on the map (shown by the purple dot).
Repeat this by setting the percentile to 80% and 95%
Panel | ||||
---|---|---|---|---|
| ||||
Q. From the CDF and probabilities maps, which ensemble forecast shows increased probability of precipitation higher than 10mm? |
Exercise 5: Cluster analysis
The paper by Pantillon et al, describes the use of clustering to identify the main scenarios among the ensemble members.
Using clustering will highlight the ensemble members in each cluster in the plots.
In this exercise you will:
- Construct your own qualitative clusters by choosing members for two clusters.
- Generate clusters using principal component analysis.
Enter the folder 'Clusters
' in the openifs_2018
folder to begin working.
Task 1: Create your own clusters
Clusters can be created manually from lists of the ensemble members.
Choose members for two clusters. The stamp maps are useful for this task.
From the stamp map of z500 at 24/9/2012 (t+96), identify ensemble members that represent the two most likely forecast scenarios.
It is usual to create clusters from z500 as it represents the large-scale flow and is not a noisy field. However, for this particular case study, the stamp map of 'tp' (total precipitation) over France is also very indicative of the distinct forecast scenarios.
You can choose any parameter to construct the clusters from, if you think another parameter shows a clear clustering behaviour.
How to create your own cluster
Right-click 'ens_oper_cluster.example.txt' and select Edit (or make a duplicate)
The file contains two example lines:
Code Block |
---|
1# 2 3 4 9 22 33 40
2# 10 11 12 31 49 |
The first line defines the list of members for 'Cluster 1': in this example, members 2, 3, 4, 9, 22, 33, 40.
The second line defines the list of members for 'Cluster 2': in this example, members 10, 11, 12, 31, 49.
Change these two lines!.
Put your choice of ensemble member numbers for cluster 1 and 2 (lines 1 and 2 respectively).
You can create multiple cluster definitions by using the 'Duplicate' menu option to make copies of the file for use in the plotting macros..
The filename is important!
The first part of the name 'ens_oper' refers to the ensemble dataset and must match the name used in the plotting macro.
The 'example' part of the filename can be changed to your choice and should match the 'clustersId'.
As an example a filename of: ens_both_cluster.fred.txt would require 'expId=ens_both
', 'clustersId=fred
' in the macro
Plot maps of parameters as clusters
The macro cluster_to_ref.mv
can be used to plot maps of parameters as clusters and compared to the ensemble control forecast and the HRES forecast.
Use cluster_to_ref.mv
to plot z500 maps of your two clusters.
If your cluster definition file is called 'ens_oper_cluster.example.txt', then Edit cluster_to_ref.mv
and set:
Code Block | ||
---|---|---|
| ||
#ENS members (use ["all"] or a list of members like [1,2,3]
members_1=["cl.example.1"]
members_2=["cl.example.2"] |
If your cluster definition file has another name, e.g. ens_oper_cluster.fred.txt, then members_1=["cl.fred.1"].
Plot ensembles with clusters
In this part of the task, redo the plots from the previous exercise which looked at ways of plotting ensemble data, but this time with clustering enabled.
Stamp maps: the stamp maps will be reordered such at the ensemble members will be grouped according to their cluster. This will make it easier to see the forecast scenarios according to your clustering. | |
Spaghetti maps: with clusters enabled, two additional maps are produced which show the contour lines for each cluster. |
Use the clusters of ensemble members you have created in ens_oper_cluster.example.txt
.
Set clustersId='example'
in each of the stamp.mv and ens_to_ref_spag.mv to enable cluster highlighting.
If time, also try the ens_part_to_all.mv
icon. This compares the spread and mean of part of the ensemble to the full ensemble.
Plot other parameters
Use the stamp.mv
icon and change it to plot the total precipitation over France with clusters enabled.e.g.
Code Block |
---|
param="tp"
expId="ens_oper"
mapType=2
clustersId="example" |
If you choice of clustering is accurate, you should see a clear separation of precipitation over France between the two clusters.
Panel | ||
---|---|---|
| ||
Q. Are two clusters enough? Do all of the ensemble members fit well into two clusters? |
Task 2: Empirical orthogonal functions / Principal component analysis
A quantitative way of clustering an ensemble uses empirical orthogonal functions from the differences between the ensemble members and the control forecast and then using an algorithm to determine the clusters from each ensemble as projected in EOF space (mathematically).
As a smooth dynamical field, geopotential height at 500hPa at 00Z 24/9/2012 is recommend (it used in the paper by Pantillon et al.), but the steps described below can be used for any parameter at any step.
The eof.mv
macro computes the EOFs and the clustering.
Warning |
---|
Always use the Otherwise If you change step or ensemble, recompute the EOFS and cluster definitions using Note that the EOF analyses is run over the smaller domain over France. This may produce a different clustering to your manual cluster if you used a larger domain. |
Panel | ||
---|---|---|
| ||
Edit Set the parameter to use, choice of ensemble and forecast step required for the EOF computation:
Run the macro. The above example will compute the EOFs of geopotential height anomaly at 500hPa using the 2012 operational ensemble at forecast step 00Z on 24/09/2012. A plot will appear showing the first two EOFs. The geographical area for the EOF computation is: 35-55N, 10W-20E. If desired it can be changed in |
Panel | |||||||
---|---|---|---|---|---|---|---|
| |||||||
The The filename will be different, it will have 'eof' in the filename to indicate it was created by using empirical orthogonal functions.
If a different ensemble forecast is used, for example This cluster definition file can then be used to plot any variable at all steps (as for task 1). |
Panel | ||
---|---|---|
| ||
Q. What do the EOFs plotted by |
Plot ensemble and cluster maps
Use the cluster definition file computed by eof.mv
to the plot ensembles and maps with clusters enabled (as above, but this time with the 'eof' cluster file).
The macro cluster_to_ref.mv
can be used to plot maps of parameters as clusters and compared to the HRES forecast.
Use cluster_to_ref.mv
to plot z500 and MSLP maps of the two clusters created by the EOF analysis.
Edit cluster_to_ref.mv
and set:
Code Block | ||
---|---|---|
| ||
#ENS members (use ["all"] or a list of members like [1,2,3]
members_1=["cl.eof.1"]
members_2=["cl.eof.2"] |
Run the macro.
If time also look at other parameters such as PV/320K.
Panel | ||
---|---|---|
| ||
Q. What are the two scenarios proposed by the two clusters? |
If time, change the date/time used to compute the clusters. How does the variance explained by the first two clusters change? Is geopotential the best parameter to use?
Changing the number of clusters
To change the number of clusters created by the EOF analysis, edit eof.mv.
Change:
Code Block |
---|
clusterNum=2 |
to
Code Block |
---|
clusterNum=3 |
Now if you run the eof.mv
macro, it will generate a text file, such as ens_oper.eof.txt
with 3 lines, one for each cluster. It will also show the 3 clusters as different colours.
You can use the 3 clusters in the cluster_to_ref.mv
macro, for example:
Code Block |
---|
param="z500.s"
expId="ens_oper"
members_1=["cl.eof.1"]
members_2=["cl.eof.3"] |
would plot the mean of the members in the first and the third clusters (it's not possible to plot all three clusters together).
You can have as many clusters as you like but it does not make sense to go beyond 3 or 4 clusters.
Panel | ||
---|---|---|
| ||
For those interested: The code that computes the clusters can be found in the Python script: This uses the 'ward' cluster method from SciPy. Other cluster algorithms are available. See http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html#scipy.cluster.hierarchy.linkage The python code can be changed to a different algorithm or the more adventurous can write their own cluster algorithm! |
Exercise 6. Assessment of forecast errors
In this exercise, the analyses covering the forecast period are now available to see how Nadine and the cut-off low actually behaved.
Various methods for presenting the forecast error are used in the tasks below. The clusters created in the previous exercise can also be used.
Enter the 'Forecast errors
' folder in the openifs_2018
folder to start work on this exercise.
Task 1: Satellite images
Open the folder 'satellite' (scroll the window if it is not visible).
This folder contains satellite images (water vapour, infra-red, false colour) for 00Z on 20-09-2012 and animations of the infra-red and water vapour images.
Double click the images to display them and watch the observed behaviour of Nadine and the cut-off low.
Task 2: Analyses from 20th Sept.
Now look at the analyses from 20th Sept to observe what actually happened.
Right-click an_1x1.mv
, Edit and set the plot to show MSLP and geopotential at 500hPa:
Code Block |
---|
plot1=["z500.s","mslp"] |
Click the play button and animate the plot to watch how Nadine and the cut-off low behave.
Drop the mv_track.mv
icon to overlay the track of Nadine onto the map.
If time, use the other icons such as an_2x2.mv
and an_xs.mv
to look at the cross-section through the analyses and compare to the forecast cross-sections from the previous exercises.
Task 3: Compare forecast to analysis
Plot forecast difference maps to see how and when the forecast differed from the analyses.
Panel |
---|
: this plots a single parameter as a difference map between the operational HRES forecast and the ECMWF analysis. Use this to understand the forecast errors. |
Use the hres_to_an_diff.mv
icon and plot the differences between the z500, MSLP and other fields to how the forecast differences evolve.
Also try the ctrl_to_an_diff.mv
icon which plots the difference but this time using the ensemble control forecast.
Panel |
---|
Q. How does the behaviour of Nadine and the cut-off low differ from the HRES deterministic forecast and the ensemble control forecast? |
Task 4: Forecast error curve
In this task, we'll look at the difference between the forecast and the analysis by using "root-mean-square error" (RMSE) curves as a way of summarising the performance of the forecast.
Root-mean square error curves are a standard measure to determine forecast error compared to the analysis and several of the exercises will use them. The RMSE is computed by taking the square-root of the mean of the forecast difference between the HRES and analyses. RMSE of the 500hPa geopotential is a standard measure for assessing forecast model performance at ECMWF (for more information see: http://www.ecmwf.int/en/forecasts/quality-our-forecasts).
Panel |
---|
: this plots the root-mean-square-error growth curves for the operational HRES forecast compared to the ECMWF analyses. |
Right-click the hres_rmse.mv icon, select 'Edit' and plot the RMSE curve for z500.
Repeat for the mean-sea-level pressure mslp.
Repeat for both geographical regions: mapType=1
(Atlantic) and mapType=2
(France).
Panel | ||
---|---|---|
| ||
Q. What do the RMSE curves show? |
Task 5: RMSE "plumes" for the ensemble
This is similar to the previous exercise, except the RMSE curves for all the ensemble members from a particular forecast will be plotted.
Right-click the ens_rmse.mv icon, select 'Edit' and plot the curves for 'mslp' and 'z500'.
Change 'expID' for your choice of ensemble (either ens_oper or ens_2016).
Clusters
First plot the plumes with clustering off:
Code Block | ||||
---|---|---|---|---|
| ||||
clustersId="off" |
There might be some evidence of clustering in the ensemble plumes.
There might be some individual forecasts that give a lower RMS error than the control forecast.
Next, use the cluster files created from the earlier exercise. You can use either your own created cluster file as before, or use the EOF generated file.
For example:
Code Block |
---|
clustersId="eof" |
would use the cluster definitions in the file: ens_oper_cluster.eof.txt (for the 2012 operational ensemble).
The cluster files are 'linked' from the Cluster folder, but if they do not work, just copy the cluster file (e.g. ens_oper_cluster.eof.txt) to the Forecast_errors folder.
Panel | ||
---|---|---|
| ||
Q. How do the HRES, ensemble control forecast and ensemble mean compare? |
Task 6: Difference stamp maps
Use the stamp_diff.mv plot to look at the differences between the ensemble members and the analysis. It can be easier to understand the difference in the ensembles by using difference stamp maps.
Note, stamp_diff.mv
cannot be used for 'tp' as there is no precipitation data in the analyses.
Clustering can also be enabled for this task.
Panel | ||
---|---|---|
| ||
Q. Using the stamp and stamp difference maps, study the ensemble. Identify which ensembles produce "better" forecasts. |
Task 5: Cumulative distribution function
Recap
The probability distribution function of the normal distribution or Gaussian distribution. The probabilities expressed as a percentage for various widths of standard deviations (σ) represent the area under the curve. |
---|
Figure from Wikipedia. |
Cumulative distribution function for a normal |
---|
Figure from Wikipedia. |
Cumulative distribution function (CDF)
The figures above illustrate the relationship between a normal distribution and its associated cumulative distribution function. The CDF is constructed from the area under the probability density function.
The CDF gives the probability that a value on the curve will be found to have a value less than or equal to the corresponding value on the x-axis. For example, in the figure, the probability for values less than or equal to X=0 is 50%.
The shape of the CDF curve is related to the shape of the normal distribution. The width of the CDF curve is directly related to the value of the standard deviation of the probability distribution function.
For an ensemble, the width is therefore related to the 'ensemble spread'.
For a forecast ensemble where all values were the same, the CDF would be a vertical straight line.
Plot the CDFs
This exercise uses the cdf.mv icon. Right-click, select 'Edit' and then:
- Plot the CDF of MSLP for Toulouse for your choice of ensemble
- Find a latitude/longitude point in the area of intense precipitation on 12Z 24/9/2012 (see Figure 2(c) Pantillon et al) and plot the CDF for MSLP (set station=[lat,lon] in the macro cdf.mv)
Note that only MSLP, 2m temperature (t2) and 10m wind-speed (speed10) are available for the CDF.
Make sure useClusters='off'.
Panel | ||
---|---|---|
| ||
Q. Compare the CDF from the different forecast ensembles; what can you say about the spread? |
Exercise 4: Cluster analysis
The paper by Pantillon et al, describes the use of clustering to identify the main scenarios among the ensemble members.
This exercise repeats some of the plots from the previous one but this time with clustering enabled.
Using clustering will highlight the ensemble members in each cluster in the plots.
In this exercise you will:
- Construct your own qualitative clusters by choosing members for two clusters
- Generate clusters using principal component analysis (similar to Pantillon et al).
Task 1: Create your own clusters
Clusters can be created manually from lists of the ensemble members.
Choose members for two clusters. The stamp maps are useful for this task.
From the stamp map of z500 at 24/9/2012 (t+96), identify ensemble members that represent the two most likely forecast scenarios.
It is usual to create clusters from z500 as it represents the large-scale flow and is not a noisy field. However, for this particular case study, the stamp map of 'tp' (total precipitation) over France is also very indicative of the distinct forecast scenarios.
Panel | ||
---|---|---|
| ||
Right-click 'ens_oper_cluster.example.txt' and select Edit (or make a duplicate) The file contains two example lines:
The first line defines the list of members for 'Cluster 1': in this example, members 2, 3, 4, 9, 22, 33, 40. The second line defines the list of members for 'Cluster 2': in this example, members 10, 11, 12, 31, 49. Change these two lines!. You can create multiple cluster definitions by using the 'Duplicate' menu option to make copies of the file for use in the plotting macros.. The filename is important! |
Panel | ||
---|---|---|
| ||
Use the clusters of ensemble members you have created in Set Replot ensembles:RMSE: plot the RMSE curves using Stamp maps: the stamp maps will be reordered such at the ensemble members will be groups according to their cluster. Applies to Spaghetti maps: with clusters enabled, two additional maps are produced which show the contour lines for each cluster. The spaghetti maps are similar to Figure 10. in Pantillon et al. |
Panel | |||||
---|---|---|---|---|---|
| |||||
The macro Use If your cluster definition file is called 'ens_oper_cluster.example.txt', then Edit
If your cluster definition file is has another name, e.g. ens_oper_cluster.fred.txt, then members_1=["cl.fred.1"]. Plot other parameters:Plot total precipitation for France ( |
Panel | ||
---|---|---|
| ||
Q. Experiment with the choice of members in each clusters and plot z500 at t+96 (Figure 7 in Pantillon et al.). How similar are your cluster maps? |
Task 2: Empirical orthogonal functions / Principal component analysis
A quantitative way of clustering an ensemble is by computing empirical orthogonal functions from the differences between the ensemble members and the control forecast.
Although geopotential height at 500hPa at 00 24/9/2012 is used in the paper by Pantillon et al., the steps described below can be used for any parameter at any step.
The eof.mv
macro computes the EOFs and the clustering.
Warning |
---|
Always use the Otherwise cluster_to_an.mv and other plots with clustering enabled will fail or plot with the wrong clustering of ensemble members. If you change step or ensemble, recompute the EOFS and cluster definitions using eof.mv. Note however, that once a cluster has been computed, it can be used for all steps with any parameter. |
Panel | ||
---|---|---|
| ||
Edit 'eof.mv' Set the parameter to use, choice of ensemble and forecast step required for the EOF computation:
Run the macro. The above example will compute the EOFs of geopotential height anomaly at 500hPa using the 2012 operational ensemble at forecast step 00Z on 24/09/2012. A plot will appear showing the first two EOFs (similar to Figure 5 in Pantillon et al.) The geographical area for the EOF computation is: 35-55N, 10W-20E (same as in Pantillon et al). If desired it can be changed in |
Panel | |||||||
---|---|---|---|---|---|---|---|
| |||||||
The eof.mv macro will create a text file with the cluster definitions, in the same format as described above in the previous task. The filename will be different, it will have 'eof' in the filename to indicate it was created by using empirical orthogonal functions.
If a different ensemble forecast is used, for example This cluster definition file can then be used to plot any variable at all steps (as for task 1). |
Panel | ||
---|---|---|
| ||
Q. What do the EOFs plotted by eof.mv show? |
Panel | |||||
---|---|---|---|---|---|
| |||||
Use the cluster definition file computed by eof.mv to the plot ensembles and maps with clusters enabled (as described for task 1, but this time with the 'eof' cluster file). The macro Use Edit
Run the macro. If time also look at the total precipitation (tp) over France and PV/320K. |
Panel | ||
---|---|---|
| ||
Q. How similar is the PCA computed clusters to your manual clustering? |
Panel | ||
---|---|---|
| ||
For those interested: The code that computes the clusters can be found in the Python script: This uses the 'ward' cluster method from SciPy. Other cluster algorithms are available. See http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html#scipy.cluster.hierarchy.linkage The python code can be changed to a different algorithm or the more adventurous can write their own cluster algorithm! |
Exercise 5. Percentiles and probabilities
To further compare the 2012 and 2016 ensemble forecasts, plots showing the percentile amount and probabilities above a threshold can be made for total precipitation.
Use these icons:
Both these macros will use the 6-hourly total precipitation for forecast steps at 90, 96 and 102 hours, plotted over France.
Task 1. Plot percentiles of total precipitation
Edit the percentile_tp_compare.mv
icon.
Set the percentile for the total precipitation to 75%:
Code Block | ||
---|---|---|
| ||
#The percentile of ENS precipitation forecast
perc=75 |
Run the macro and compare the percentiles from both the forecasts. Change the percentiles to see how the forecasts differ.
Task 2: Plot probabilities of total precipitation
This macro will produce maps showing the probability of 6-hourly precipitation for the same area as in Task 1.
In this case, the maps show the probability that total precipitation exceeds a threshold expressed in mm.
Edit the prob_tp_compare.mv
and set the probability to 20mm:
Code Block | ||
---|---|---|
| ||
#The probability of precipitation greater than
prob=20 |
Run the macro and view the map. Try changing the threshold value and run.
Panel | ||
---|---|---|
| ||
Q. Using these two macros, compare the 2012 and 2016 forecast ensemble. Which was the better forecast for HyMEX flight planning? |
Exercise 6. Assessment of forecast errors
In this exercise, various methods for presenting the forecast error are presented.
Panel |
---|
hres_rmse.mv : this plots the root-mean-square-error growth curves for the operational HRES forecast compared to the ECMWF analyses. hres_to_an_diff.mv : this plots a single parameter as a difference map between the operational HRES forecast and the ECMWF analysis. Use this to understand the forecast errors. |
Task 1: Forecast error
In this task, we'll look at the difference between the forecast and the analysis by using "root-mean-square error" (RMSE) curves as a way of summarising the performance of the forecast.
Root-mean square error curves are a standard measure to determine forecast error compared to the analysis and several of the exercises will use them. The RMSE is computed by taking the square-root of the mean of the forecast difference between the HRES and analyses. RMSE of the 500hPa geopotential is a standard measure for assessing forecast model performance at ECMWF (for more information see: http://www.ecmwf.int/en/forecasts/quality-our-forecasts).
Right-click the hres_rmse.mv icon, select 'Edit' and plot the RMSE curve for z500.
Repeat for the mean-sea-level pressure mslp.
Repeat for both geographical regions: mapType=1 (Atlantic) and mapType=2 (France).
Panel | ||
---|---|---|
| ||
Q. What do the RMSE curves show? |
Task 2: Compare forecast to analysis
Use the hres_to_an_diff.mv icon and plot the difference map between the HRES forecast and the analysis for z500 and mslp.
Panel | ||
---|---|---|
| ||
Q. What differences can be seen? |
...
Appendix
Further reading
For more information on the stochastic physics scheme in (Open) IFS, see the article:
Shutts et al, 2011, ECMWF Newsletter 129.
Acknowledgements
We gratefully acknowledge the following for their contributions in preparing these exercises. From ECMWF: Glenn Carver, Gabriella Szepszo, Sandor Kertesz, Linus Magnusson, Iain Russell, Simon Lang, Filip Vana. From ENM/Meteo-France: Frédéric Ferry, Etienne Chabot, David Pollack and Thierry Barthet for IT support at ENM.
...
Excerpt Include | ||||||
---|---|---|---|---|---|---|
|