...
Section | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
- HRES : spectral T1279 (16km grid) highest resolution 10 day deterministic forecast.
- ENS : spectral T639 (34km 31km grid) resolution ensemble forecast (50 members) is run for days 1-10 of the forecast, T319 (70km) is run for days 11-15.
In 2016, the ECMWF operational forecasts has been was upgraded compared to 2012 and consisted of:
...
These exercises use a relatively large domain with high resolution data. Some of the plotting options can therefore require significant amounts of memory. If the virtual machine freezes when running metviewMetview, please try increasing the memory assigned to the VM.
...
Info |
---|
The macros described in this tutorial will can write PostScript , PDF and PNG output GIF image files to the To save the images, use the 'Execute' menu option on the icon, rather than 'Visualise'. The 'okular' command can be used to view the PDF & gif images. |
To save any other images during these exercises for discussion later, you can either use:
...
In this exercise, the development of Hurricane Nadine and the cut-off flow up to the 20th September 2012 is studied.
Begin by entering the folder labelled 'Analysis':
...
This task will look at the synoptic development of Hurricane Nadine and the cutoff low up to 00Z, 20th September 2012. The forecasts in the next exercises start from this time and date.
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
an_1x1.mv : this plots horizontal maps of parameters from the ECMWF analyses overlaid on one plot. an_2x2.mv : this plots horizontal maps of parameters from the ECMWF analyses four plots to a page (two by two). |
...
Info |
---|
If the contour lines appear jagged, in the plot window, select the menu item 'Tools -> Antialias'. |
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Q. What is unusual about Hurricane Nadine? |
Warning | ||
---|---|---|
| ||
Please close any unused plot windows if using a virtual machine. This case study uses high resolution data over a relatively large domain. Multiple plot windows can therefore require significant amounts of computer memory which can be a problem for virtual machines with restricted memory. |
...
Panel | ||||
---|---|---|---|---|
| ||||
Change the value of
You might add the mslp or z500 fields to this plot e.g.
Note that the fields are plotted in the order specified in the list! |
Panel | ||||
---|---|---|---|---|
| ||||
Q. When does the cut-off low form (see z500)? |
Task 3: Changing the map geographical area
Task 3: Changing the map geographical area
Right-click on Right-click on an_1x1.mv
icon and select 'Edit'.
...
Panel |
---|
For this exercise, you will use the metview Metview icons in the folder ' hres_1x1.mv & : for this exercise, this icon can be used to overlay the forecast track of Nadine (and not the track from the analyses as in Exercise 1) |
...
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
|
...
This task focuses on the fate of Nadine and examines vertical PV cross-sections of Nadine and the cutoff at different forecast times to characterize the diabatic warm core PV tower of Nadine compared to the upper level PV cold core of the cutoff.
Right-click on the icon 'hres_xs.mv'
icon, select 'Edit' and push the play button.
...
Panel | ||||
---|---|---|---|---|
| ||||
Q. Look at the PV field, how do the vertical structures of Nadine and the cut-off low differ? |
Changing forecast time
Cross-section data is only available every 24hrs until the 30th Sept 00Z (step 240).
...
Code Block |
---|
steps=[2012-09-26 00:00] |
Changing cross-section location
To change the forecast length for hres_1x1.mv
and hres_2x2.mv
, right-click, select Edit and change:
Code Block |
---|
fclen=5 |
to
Code Block |
---|
fclen=10 |
Changing cross-section location
Code Block |
---|
Code Block |
#Cross section line [ South, West, North, East ] line = [30,-29,45,-15] |
...
If the forecast time is changed, the storm centres will move and the cross-section line will need to be repositioned to follow specific features. This is not computed automatically, but must be changed by altering the coordinates above. Use the cursor data icon to find the new position of the line.
Change the forecast time again to day+8 (28th Sept), or a different date if you are interested, relocate and plot the cross-section of Nadine and the low pressure system. Use the hres_1x1.mv
icon from task 1 if you need to follow location of Nadine.
...
Panel | ||||
---|---|---|---|---|
| ||||
Q. What changes are there to the vertical structure of Nadine during the forecast? |
Cyclone phase space (CPS) diagrams
An objectively defined cyclone phase space (CPS) is described using the storm-motion-relative thickness asymmetry (symmetric/non-frontal versus asymmetric/frontal) and vertical derivative of horizontal height gradient (cold- versus warm-core structure via the thermal wind relationship). A cyclone's life cycle can then be analyzed within this phase space, providing insight into the cyclone structural evolution.
...
Panel | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Additional plots for further analysis:study
|
...
Panel | ||||
---|---|---|---|---|
| ||||
Q. How does the ensemble mean MSLP and Z500 fields compare to the HRES forecast? |
Task 1: Ensemble spread
Use the ens_maps.mv
icon and plot the MSLP and z500. This will produce plots showing: the mean of all the ensemble forecasts, the spread of the ensemble forecasts and the operational HRES deterministic forecast.
...
Panel | ||
---|---|---|
| ||
Q. How much uncertainty is there in the precipitation forecast over southern France? |
Compare ensemble members to the deterministic and control forecast
After visualizing the stamp maps, it can be useful to animate a comparison of individual ensemble members to the HRES and ensemble control deterministic forecastforecasts.
This can help in identifying individual ensemble members that produce a better different forecast than the control or HRES forecast.
Icons Use the ens_to_ref_diff.mv
and icon to compare an ensemble member to the HRES forecast. Use pf_to_cf_diff.mv
can be used to compare ensemble members to the control forecast.
Panel | ||||
---|---|---|---|---|
| ||||
To animate the difference in MSLP of an individual ensemble member 30 to the analysisHRES forecast, edit the lines:
and visualise the plot. To compare the control forecast, change:
|
Further analysis using ensembles
Panel | ||
---|---|---|
| ||
This will show the forecasts from the ensemble members and also their difference with the ensemble control forecast. To animate the difference in MSLP with ensemble members '30' and '50', set:
|
Panel | ||||
---|---|---|---|---|
| ||||
Compare the control forecast scenario to the HRES: Q. Try to identify ensemble members which are the closest and furthest to the HRES forecast. |
Sea-surface temperature
Compare the SST parameter used for the ens_oper and ens_2016 ensemble forecasts. The 2016 reforecast of this case study used a coupled ocean model unlike the 2012 ensemble and HRES forecast that used climatology for the first 5 days.
Panel | ||
---|---|---|
| ||
Q. What is different about SST between the two ensemble forecasts? |
Cross-sections of
...
an ensemble member
To show a cross-section of a particular ensemble member, use the macro ens_xs.mv.
This works in the same way as the hres_xs.mv
macros.
Identifying sensitive regions
Find ensemble members that appear to produce a better forecast and look to see how the initial development in these members differs.
...
Exercise 4: CDF, percentiles and probabilities
Recap
Figure 1. The probability distribution function of the normal distribution or Gaussian distribution. The probabilities expressed as a percentage for various widths of standard deviations (σ) represent the area under the curve. |
---|
Figure from Wikipedia. |
Figure 2. Cumulative distribution function for a normal |
---|
...
Panel | ||
---|---|---|
| ||
Q. Can you tell which area is more sensitive for the forecast? |
Exercise 4: CDF, percentiles and probabilities
Recap
Figure 1. The probability distribution function of the normal distribution or Gaussian distribution. The probabilities expressed as a percentage for various widths of standard deviations (σ) represent the area under the curve. |
---|
Figure from Wikipedia. | Figure 2.
Cumulative distribution function
...
The figures
...
Figure from Wikipedia.
Cumulative distribution function
The figures above illustrate the relationship between a normal distribution and its associated cumulative distribution function. The CDF is constructed from the area under the probability density function.
...
Using the probability map, click the cursor data icon select choose a location in the region of highest rainfall. Do this for both the 2012 or and 2016 ensemble map.
and move the pointer over the map for +96h andMake a note of the latitude and longitude coordinates. The highest rainfall area was approximately over the Cévennes mountains , approximately ( 44°25′34″N 03°44′21″E ).
Edit prob_tp_compare.mv
and set the location:
...
and replot the map. A small purple dot will appear at the location specified. If the dot is not in the right location, change it and replot.
...
You should now have the probability values that total precipitation will exceed 10mm, 20mm and 30mm, for both the 2012 and 2016 ensembles, for forecast time +96 hours for your chosen location.
Task 2: Plot the CDF
This exercise uses the cdf.mv icon.
...
Set the percentile for the total precipitation to 70% and specify the location as in Task Tasks 1 & 2:
Code Block | ||
---|---|---|
| ||
#The percentile of ENS precipitation forecast perc=70 location=[44.0,4.1] # [ lat, lon ] -- use your own values! |
...
Panel | ||||
---|---|---|---|---|
| ||||
Q. From the CDF and probabilities maps, which ensemble forecast shows increased probability of precipitation higher than 10mm? |
Exercise 5: Cluster analysis
...
- Construct your own qualitative clusters by choosing members for two clusters.
- Generate clusters using principal component analysis.
...
You can choose any parameter to construct the clusters from, if you think another parameter shows a clear clustering behaviour.
How to create your own cluster
...
The filename is important!
The first part of the name 'ens_oper' refers to the ensemble dataset and must match the name used in the plotting macro.
The 'example' part of the filename can be changed to your choice and should match the 'clustersId'.
As an example a filename of: ens_both_cluster.fred.txt would require 'expId=ens_both
', 'clustersId=fred
' in the macro.
Plot
...
maps of parameters as clusters
The macro cluster_to_ref.mv
can be used to plot maps of parameters as clusters and compared to the ensemble control forecast and the HRES forecast.
Use cluster_to_ref.mv
to plot z500 maps of your two clusters.
If your
Use the clusters of ensemble members you have created in ens_oper_cluster.example.txt
.
Set clustersId='example'
in each of the ensemble plotting macros to enable cluster highlighting.
Replot ensembles:
Stamp maps: the stamp maps will be reordered such at the ensemble members will be groups according to their cluster. Applies to stamp.mv
icon. This will make it easier to see the forecast scenarios according to your clustering.
Spaghetti maps: with clusters enabled, two additional maps are produced which show the contour lines for each cluster.
Plot maps of parameters as clusters
Warning |
---|
TODO: add image of cluster_hres.mv (renamed from cluster_to_an.mv) here. |
The macro cluster_to_an.mv
can be used to plot maps of parameters as clusters and compared to the HRES forecasts. TODO: rename cluster_to_an.mv to cluster_hres.mv and comment out plot of AN.
Use cluster_to_an.mv
to plot z500 maps of your two clusters.
If your cluster definition file is called 'ens_oper_cluster.example.txt', then Edit cluster_to_anref.mv
and set:
Code Block | ||
---|---|---|
| ||
#ENS members (use ["all"] or a list of members like [1,2,3] members_1=["cl.example.1"] members_2=["cl.example.2"] |
If your cluster definition file is has another name, e.g. ens_oper_cluster.fred.txt, then members_1=["cl.fred.1"].
Plot other parameters:
Plot total precipitation for France (mapType=2
). Compare with Figure 8. in Pantillon et al.
has another name, e.g. ens_oper_cluster.fred.txt, then members_1=["cl.fred.1"].
Plot ensembles with clusters
In this part of the task, redo the plots from the previous exercise which looked at ways of plotting ensemble data, but this time with clustering enabled.
Stamp maps: the stamp maps will be reordered such at the ensemble members will be grouped according to their cluster. This will make it easier to see the forecast scenarios according to your clustering. | |
Spaghetti maps: with clusters enabled, two additional maps are produced which show the contour lines for each cluster. |
Use the clusters of ensemble members you have created in ens_oper_cluster.example.txt
.
Set clustersId='example'
in each of the stamp.mv and ens_to_ref_spag.mv to enable cluster highlighting.
If time, also try the ens_part_to_all.mv
icon. This compares the spread and mean of part of the ensemble to the full ensemble.
Plot other parameters
Use the stamp.mv
icon and change it to plot the total precipitation over France with clusters enabled.e.g.
Code Block |
---|
param="tp"
expId="ens_oper"
mapType=2
clustersId="example" |
If you choice of clustering is accurate, you should see a clear separation of precipitation over France between the two clusters.
Panel | ||
---|---|---|
| ||
Panel | ||
borderColor | redQ. Are two clusters enough? Do all of the ensemble members fit well into two clusters? |
|
Task 2: Empirical orthogonal functions / Principal component analysis
A quantitative way of clustering an ensemble is by computing uses empirical orthogonal functions from the differences between the ensemble members and the control forecast. and then using an algorithm to determine the clusters from each ensemble as projected in EOF space (mathematically).
As a smooth dynamical field, Although geopotential height at 500hPa at 00 00Z 24/9/2012 is recommend (it used in the paper by Pantillon et al.), but the steps described below can be used for any parameter at any step.
Warning |
---|
TODO: update macro image below. |
The eof.mv
macro computes the EOFs and the clustering.
Warning |
---|
Always use the Otherwise If you change step or ensemble, recompute the EOFS and cluster definitions using Note that the EOF analyses is run over the smaller domain over France. This may produce a different clustering to your manual cluster if you used a larger domain. |
...
Panel | ||
---|---|---|
| ||
Edit ' Set the parameter to use, choice of ensemble and forecast step required for the EOF computation:
Run the macro. The above example will compute the EOFs of geopotential height anomaly at 500hPa using the 2012 operational ensemble at forecast step 00Z on 24/09/2012. A plot will appear showing the first two EOFs. The geographical area for the EOF computation is: 35-55N, 10W-20E (same as in Pantillon et al). If desired it can be changed in |
...
Panel | |||||||
---|---|---|---|---|---|---|---|
| |||||||
The The filename will be different, it will have 'eof' in the filename to indicate it was created by using empirical orthogonal functions.
If a different ensemble forecast is used, for example This cluster definition file can then be used to plot any variable at all steps (as for task 1). |
...
Panel | ||
---|---|---|
| ||
Q. What do the EOFs plotted by |
...
? |
Plot ensemble and cluster maps
Use the cluster definition file computed by eof.mv
to the plot ensembles and maps with clusters enabled (as above, but this time with the 'eof' cluster file).
The macro cluster_to_
ref.mv
can be used to plot maps of parameters as clusters and compared to the
analysis andHRES
forecastsforecast.
Use cluster_to_
ref.mv
to plot z500 and MSLP maps of the two clusters created by the EOF
analysis.
Edit cluster_to_
ref.mv
and set:
Code Block | ||
---|---|---|
| ||
#ENS members (use ["all"] or a list of members like [1,2,3]
members_1=["cl.eof.1"]
members_2=["cl.eof.2"] |
Run the macro.
If time also look at
the total precipitation (tp) over France andother parameters such as PV/320K.
Cluster 1 corresponds to a cutoff low moving eastward over Europe and cluster 2 to a weak ridge over western Europe. Cluster 1 exhibits a weak
Panel | ||
---|---|---|
| ||
Panel | ||
borderColor | redQ. What are the two scenarios proposed by the two clusters? |
...
in the two clusters? |
|
If time, change the date/time used to compute the clusters. How does the variance explained by the first two clusters change?
...
Is geopotential the best parameter to use?
...
Changing the number of clusters
To change the number of clusters created by the EOF analysis,
...
edit eof.mv.
...
Change:
Code Block |
---|
clusterNum=2 |
to
Code Block |
---|
clusterNum=3 then select 'File' and 'Save' to save the changes. |
Now if you run the eof.mv
macro, it will generate a text file, such as ens_oper.eof.txt
with 3 lines, one for each cluster. It will also show the 3 clusters as different colours.
You can use the 3 clusters in the cluster_to_
...
ref.mv
macro, for example:
Code Block |
---|
param="z500.s"
expId="ens_oper"
members_1=["cl.eof.1"]
members_2=["cl.eof.3"] |
would plot the mean of the members in the first and the third clusters (it's not possible to plot all three clusters together).
You can have as many clusters as you like but it does not make sense to go beyond 3 or 4 clusters.
Panel | ||
---|---|---|
| ||
For those interested: The code that computes the clusters can be found in the Python script: This uses the 'ward' cluster method from SciPy. Other cluster algorithms are available. See http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html#scipy.cluster.hierarchy.linkage The python code can be changed to a different algorithm or the more adventurous can write their own cluster algorithm! |
Exercise 6. Assessment of forecast errors
Exercise 6. Assessment of forecast errors
In this exercise, the analyses covering the forecast period are now available to see how Nadine and the cut-off low actually behaved.
Various In this exercise, various methods for presenting the forecast error are presentedused in the tasks below. The clusters created in the previous exercise above can also be used.
...
TODO:
...
Enter the 'Forecast errors
' folder in the openifs_2018
folder to start work on this exercise.
Task 1: Satellite images
Open the folder 'satellite'the folder 'satellite' (scroll the window if it is not visible).
This folder contains satellite images (water vapour, infra-red, false colour) for 00Z on 20-09-2012 and animations of the infra-red and water vapour images.
Double click the images to display them and watch the observed behaviour of Nadine and the cut-off low.
Task 2: Analyses from 20th Sept.
The first task is to now look at the analyses from 20th Sept to observe what actually happened.
Warning |
---|
TODO
|
Enter the folder 'Forecast_errors' (TODO: include icon graphic)
.
Task 2: Analyses from 20th Sept.
Now look at the analyses from 20th Sept to observe what actually happened.
As in Exercise 1, task 1. Right-click an_1x1.mv
, Edit and set the plot to show MSLP and geopotential at 500hPa:
Code Block |
---|
plot1=["z500.s","mslp"] |
Click the play button and animate the plot button and animate the plot to watch how Nadine and the cut-off low behave.
Drop the mv_track.mv
icon to overlay the track of Nadine onto the map.
If time, use the other icons such as an_2x2.mv
and an_xs.mv
to look at the cross-section through the analyses and compare to the forecast cross-sections from the previous exercises.
Task 3: Compare forecast to analysis
Plot forecast difference maps to see how and when the forecast differed from the analyses.
Panel |
---|
hres_to_an_diff.mv : this plots a single parameter as a difference map between the operational HRES forecast and the ECMWF analysis. Use this to understand the forecast errors. |
Use the hres_to_an_diff.mv
icon and plot the differences between the z500, MSLP and other fields to how the forecast differences evolve.
Also try the ctrl_to_an_diff.mv
icon which plots the difference but this time using the ensemble control forecast.
Panel |
---|
Q. How does the behaviour of Nadine and the cut-off low differ from the HRES deterministic forecast and the ensemble control forecast? |
Task 4: Forecast error curve
...
...
In this task, we'll look at the difference between the forecast and the analysis by using "root-mean-square error" (RMSE) curves as a way of summarising the performance of the forecast.
Root-mean square error curves are a standard measure to determine forecast error compared to the analysis and several of the exercises will use them. The RMSE is computed by taking the square-root of the mean of the forecast difference between the HRES and analyses. RMSE of the 500hPa geopotential is a standard measure for assessing forecast model performance at ECMWF (for more information see: http://www.ecmwf.int/en/forecasts/quality-our-forecasts)-our-forecasts).
Panel |
---|
: this plots the root-mean-square-error growth curves for the operational HRES forecast compared to the ECMWF analyses. |
Right-click the hres_rmse.mv icon, select 'Edit' and plot the RMSE curve for z500.
...
would use the cluster definitions in the file: ens_oper_cluster.eof.txt (for the 2012 operational ensemble).
The cluster files are 'linked' from the Cluster folder, but if they do not work, just copy the cluster file (e.g. ens_oper_cluster.eof.txt) to the Forecast_errors folder.
Panel | ||
---|---|---|
| ||
Q. How do the HRES, ensemble control forecast and ensemble mean compare? |
...
Panel | ||
---|---|---|
| ||
Q. Using the stamp and stamp difference maps, study the ensemble. Identify which ensembles produce "better" forecasts. |
Appendix
Further reading
For more information on the stochastic physics scheme in IFS, see the article:
Shutts et al, 2011, ECMWF Newsletter 129.
Acknowledgements
We gratefully acknowledge the following for their contributions in preparing these exercises. From ECMWF: Glenn Carver, Gabriella Szepszo, Sandor Kertesz, Linus Magnusson, Iain Russell, Simon Lang, Filip Vana. From ENM/Meteo-France: Frédéric Ferry, Etienne Chabot, David Pollack and Thierry Barthet for IT support at ENM.
...
Excerpt Include | ||||||
---|---|---|---|---|---|---|
|