Introduction
The ECMWF operational ensemble forecasts for the western Mediterranean region exhibited high uncertainty while Hurricane Nadine was slowly moving over the eastern N.Atlantic in Sept. 2012. Interaction with an Atlantic cut-off low produced a bifurcation in the ensemble and significant spread, which controls both the track of Hurricane Nadine and the synoptic conditions downstream.
The HyMEX (Hydrological cycle in Mediterranean eXperiment) field campaign was also underway and forecast uncertainty was a major issue for planning observations during the first special observations period of the campaign.
This interesting case study examines the forecasts in the context of the interaction between Nadine and the Atlantic cut-off low in the context of ensemble forecasting. It will explore the scientific rationale for using ensemble forecasts, why they are necessary and they can be interpreted, particularly in a "real world" situation of forecasting for a observational field campaign.
Pantillon, F., Chaboureau, J.-P. and Richard, E. (2015), 'Vortex-vortex interaction between Hurricane Nadine and an Atlantic cutoff dropping the predictability over the Mediterranean, http://onlinelibrary.wiley.com/doi/10.1002/qj.2635/abstract
In this case study
In the exercises for this interesting case study we will:
- study the development of Hurricane Nadine and the interaction with the Atlantic cut-off low using the ECMWF analyses.
- study the performance of the ECMWF high resolution (HRES) deterministic forecast of the time.
- use the operational ensemble forecast to look at the forecast spread and understand the uncertainty downstream of the interaction.
- compare a reforecast using the current (May/2016) ECMWF operational ensemble with the 2012 ensemble forecasts.
- use principal component analysis (PCA) with clustering techniques (see Pantillon et al) to characterize the behaviour of the ensembles.
- see how forecast products were used during the HyMEX field campaign.
Table of contents
Caveat on use of ensembles for case studies
In practise many cases are aggregated in order to evaluate the forecast behaviour of the ensemble. However, it is always useful to complement such assessments with case studies of individual events, like the one in this exercise, to get a more complete picture of IFS performance and identify weaker aspects that need further exploration.
Obtaining the exercises
The exercises described below are available as a set of Metview macros with the accompanying data. This is available as a downloadable tarfile for use with Metview (if installed). It is also available as part of the OpenIFS/Metview virtual machine, which can be run on different operating systems.
For more details of the OpenIFS virtual machine and how to get the workshop files, please contact: openifs-support@ecmwf.int.
ECMWF operational forecasts
At the time of this case study in 2012, ECMWF operational forecasts consisted of:
- HRES : spectral T1279 (16km grid) highest resolution 10 day deterministic forecast.
- ENS : spectral T639 (34km grid) resolution ensemble forecast (50 members) is run for days 1-10 of the forecast, T319 (70km) is run for days 11-15.
At the time of this workshop (2016), the ECMWF operational forecasts has been upgraded compared to 2012 and consisted of:
- HRES/2016 : spectral T1279 with an octahedral grid configuration providing highest resolution of 9km.
- ENS/2016 : spectral T639 with an octahedral grid configuration providing highest resolution of 18km for all 15 days of the forecast.
Please follow this link to see more details on changes to the ECMWF IFS forecast system (http://www.ecmwf.int/en/forecasts/documentation-and-support/changes-ecmwf-model)
Virtual machine
If using the OpenIFS/Metview virtual machine with these exercises the recommended minimum memory is 4Gb. These exercises use a relatively large domain with high resolution data. Some of the plotting options can therefore require significant amounts of memory. If the virtual machine freezes when running metview, please try increasing the memory assigned to the VM.
Starting up metview
To begin:
metview
Please enter the folder 'openifs_2016' to begin working.
Saving images and printing
To save images during these exercises for discussion later, you can either use:
"Export" button in Metview's display window under the 'File' menu to save to PNG image format. This will also allow animations to be saved into postscript.
or use the following command to take a 'snapshot' of the screen:
ksnapshot
Exercise 1. The ECMWF analysis
Learning objectives
- Study synoptic conditions over the N.Atlantic and Europe during the approach of Hurricane Nadine.
Hurricane Nadine and the cut-off low
For these tasks, use the metview icons in the row labelled 'Analysis'
an_1x1.mv : this plots horizontal maps of parameters from the ECMWF analyses overlaid on one plot.
an_2x2.mv : this plots horizontal maps of parameters from the ECMWF analyses four plots to a page (two by two).
an_xs.mv : this plots vertical cross-sections of parameters from the ECMWF analyses.
Task 1: Mean-sea-level pressure and track
Right-click the mouse button on the 'an_1x1.mv' icon and select the 'Visualise' menu item (see figure right)
After a pause, this will generate a map showing mean-sea-level pressure (MSLP).
Now drag the mv_track.mv icon onto the map. This will add the track of Nadine. Although the full track of the tropical storm is shown from the 10-09-2012 to 04-10-2016, the ECMWF analyses (for the purpose of this study) only show 15-09-2012 to 25-09-2012.
In the plot window, use the play button in the animation controls 
to animate the map and follow the development and track of Hurricane Nadine.
You can use the 'Speed' menu to change the animation speed (each frame is every 6 hours).
If the contour lines appear jagged, in the plot window, select the menu item 'Tools -> Antialias'.
Close unused plot windows!
Please close any unused plot windows if using a virtual machine. This case study uses high resolution data over a relatively large domain. Multiple plot windows can therefore require significant amounts of computer memory which can be a problem for virtual machines with restricted memory.
Task 2: MSLP and 500hPa geopotential height
This task creates Figure 2. from Pantillon et al.
Right-click the mouse button on the 'an_1x1.mv' icon and select the 'Edit' menu item.
An edit window appears that shows the Metview macro code used to generate the plot. During these exercises you can change the top lines of these macros to alter the choice of parameters and plot types.
#Available parameters: # mslp,t2,wind10,speed10,sst # t,z,pt,eqpt [850,700,500,200] # wind,speed,r[925,850,700,500,200] # w700, vo850, pv320K
The surface fields (single level) are: MSLP (mean-sea-level-pressure), 2-metre temperature (t2), 10-metre wind arrows (wind10), wind-speed at 10m (sqrt(u^2+v^2): speed10), sea-surface temperature (sst).
The upper level fields are: temperature (t), geopotential (z), potential temperature (pt), equivalent potential temperature (eqpt), wind arrows (wind), wind-speed (speed), relative humidity (r).
These fields have a list of available pressure levels in square brackets.
To plot upper level fields, specify the pressure level after the name. e.g. z500 would plot geopotential at 500hPa.
Some extra fields are also provided: vertical velocity at 700hPa (w700), relative vorticity at 850hPa (vo850) and potential vorticity at 320K.
Wind fields are normally plotted as coloured arrows. To plot them as wind barbs add the suffix '.flag'. e.g. "wind10.flag" will plot 10m wind as barbs.
With the edit window open, find the line that defines 'plot1':
#Define plot list (min 1- max 4) plot1=["mslp"] # use square brackets when overlaying multiple fields per plot
Change this line to:
plot1=["z500.s","mslp"]
The suffix '.s' means plot the 500hPa geopotential as a shaded plot instead of using contours (this style is not available for all fields)
As above, click the play button and then animate the map that appears.
Compare an animation of the z500 and mslp fields with Figure 1. from Pantillon et al.
When does the cut-off low form (see z500)?
How close do Nadine and the cut-off low get in the analyses?
Task 3: Changing geographical area
Right-click the mouse button on the 'an_1x1.mv' icon and select the 'Edit' menu item.
In the edit window that appears
#Map type: 0=Atl-an, 1: Atl-fc, 2: France mapType=0
With mapType=0, the map covers a large area centred on the Atlantic suitable for plotting the analyses and track of the storm.
With mapType=1, the map also covers the Atlantic but a smaller area than for the analyses. This is because the forecast data in the following exercises does not cover as large a geographical area as the analyses.
With mapType=2, the map covers a much smaller region centred over France.
Change, mapType=0 to mapType=1 then click the play button 
at the top of the window.
Repeat using mapType=2 to see the smaller region over France.
These different regions will be used in the following exercises.
Animate the storm on this smaller geographical map.
Task 4: Wind fields, sea-surface temperature (SST)
The 'an_2x2.mv' icon allows for plotting up to 4 separate figures on a single frame. This task uses this icon to plot multiple fields.
Right-click on the 'an_2x2.mv' icon and select the 'Edit' menu item.
#Define plot list (min 1- max 4) plot1=["mslp"] plot2=["wind10"] plot3=["speed500","z500"] plot4=["sst"]
Click the play button at the top of the window to run this macro with the existing plots as shown above.
Note that each plot can be a single field or overlays of different fields as in the an_1x1.mv macro.
Wind parameters can be shown either as arrows or as wind flags ('barbs') by adding '.flag' to the end of variable name e.g. "wind10.flag".
Animating. If only one field on the 2x2 plot animates, make sure the menu item 'Animation -> Animate all scenes' is selected.
Plotting may be slow depending on the computer used. This reads a lot of data files.
What do you notice about the SST field?
Task 5: Satellite images
Open the folder 'satellite' by doubling clicking (scroll the window if it is not visible).
This folder contains satellite images (water vapour, infra-red, false colour) for 00Z on 20-09-2012 and animations of the infra-red and water vapour images.
Double click the images to display them.
Use the an_1x1.mv and/or the an_2x2.mv macros to compare the ECMWF analyses with the satellite images.
Task 6: Cross-sections
The last task in this exercise is to look at cross-sections through Hurricane Nadine and the cut-off low.
Right click on the icon 'an_xs.mv', select 'Edit' and push the play button.
This generates a plot with a map of MSLP, a red line and underneath a cross-section plot along that red-line.
The default plot shows potential vorticity (PV) and potential temperature roughly through the centre of the Hurricane and the cut-off low.
Changing forecast time
Cross-section data is only available every 24hrs.
This means the 'steps' value in the macros is only valid for the times: [2012-09-20 00:00], [2012-09-21 00:00], [2012-09-22 00:00], [2012-09-23 00:00], [2012-09-24 00:00], [2012-09-25 00:00]
Changing fields
A smaller set of fields is available for cross-sections: temperature (t), potential temperature (pt), relative humidity (r), potential vorticity (pv), vertical velocity (w), wind-speed (speed; sqrt(u*u+v*v))
Changing cross-section location
#Cross section line [ South, West, North, East ] line = [30,-29,45,-15]
The cross-section location (red line) can be changed in this macro by defining the end points of the line as shown above.
Remember that if the forecast time is changed, the storm centres will move and the cross-section line will need to be repositioned to follow specific features. This is not computed automatically, but must be changed by altering the coordinates above.
Look at the PV field, how do the vertical structures of Nadine and the cut-off low differ?
This completes the first exercise.
You have learnt how to use the macros, alter fields for plotting and animate fields. The next exercises use similar macros.
Exercise 2: The operational HRES forecast
Recap
The ECMWF operational deterministic forecast is called HRES. At the time of this case study, the model ran with a spectral resolution of T1279, equivalent to 16km grid spacing.
Only a single forecast is run at this resolution as the computational resources required are demanding. The ensemble forecasts are run at a lower resolution.
Before looking at the ensemble forecasts, first understand the performance of the operational HRES forecast of the time.
Available forecast
Data is provided for a single 5 day forecast starting from 20th Sept 2012, as used in the paper by Pantillon et al.
HRES data is provided at the same resolution as the operational model, in order to give the best representation of the Hurricane and cut-off low interations. This may mean that some plotting will be slow.
Fields available
The fields (parameters) available in the analyses are available in the forecast data.
A new field is total precipitation : tp.
Questions to consider
- How does the HRES forecast compare to analysis and satellite images?
- Was it a good or bad forecast? Why?
Available plot types
For this exercise, you will use the metview icons in the row labelled 'HRES forecast' as shown above.
hres_rmse.mv : this plots the root-mean-square-error growth curves for the operational HRES forecast compared to the ECMWF analyses.
hres_1x1.mv & hres_2x2.mv : these work in a similar way to the same icons used in the previous task where parameters from a single lead time can be plotted either in a single frame or 4 frame per page.
hres_to_an_diff.mv : this plots a single parameter as a difference map between the operational HRES forecast and the ECMWF analysis. Use this to understand the forecast errors.
Parameters & map appearance. These macros have the same choice of parameters to plot and same choice of mapType, either the Atlantic sector or over Europe.
Forecast performance
Task 1: Forecast error
In this task, we'll look at the difference between the forecast and the analysis by using "root-mean-square error" (RMSE) curves as a way of summarising the performance of the forecast. Root-mean square error curves are a standard measure to determine forecast error compared to the analysis and several of the exercises will use them. The RMSE is computed by taking the square-root of the mean of the forecast difference between the HRES and analyses.
Right-click the hres_rmse.mv icon, select 'Edit' and plot the RMSE curve for z500. RMSE of the 500hPa geopotential is a standard measure for assessing forecast model performance at ECMWF (for more information see: http://www.ecmwf.int/en/forecasts/quality-our-forecasts).
Repeat for the mean-sea-level pressure mslp.
Repeat for both geographical regions: mapType=1 (Atlantic) and mapType=2 (France).
1. What do the RMSE curves show?
2. Why are the curves different between the two regions?
Task 2: Compare forecast to analysis
Use the hres_to_an_diff.mv icon and plot the difference map between the HRES forecast and the analysis for z500 and mslp.
1. What differences can be seen?
2. What can said about the quality of the forecast in positioning of the Hurricane and cut-off N.Atlantic low
Look at other fields to study the forecast. For example, jet position, total precipitation (tp).
Other plot types
As for the analyses, the macros hres_1x1.mv, hres_2x2.mv and hres_xs.mv can be used to plot and animate fields or overlays of fields from the HRES forecast.
Group working suggestions
As a group, discuss the plots & parameters to address the questions above given what you see in the error growth curves and maps from task 2.
Look at the difference between forecast and analysis to understand the error in the forecast, particularly the starting formation and final error..
Remember to save plots of interest for later group presentation.
TODO: What more do we need to ask the students to do here?
Exercise 3 : The operational ensemble forecasts
Recap
- ECMWF operational ensemble forecasts treat uncertainty in both the initial data and the model.
- Initial analysis uncertainty: sampled by use of Singular Vectors (SV) and Ensemble Data Assimilation (EDA) methods. Singular Vectors are a way of representing the fastest growing modes in the initial state.
- Model uncertainty: sampled by use of stochastic parametrizations In IFS this means Stochastically Perturbed Physical Tendencies (SPPT) and the spectral backscatter scheme (SKEB)
- Ensemble mean : the average of all the ensemble members. Where the spread is high, small scale features can be smoothed out in the ensemble mean.
- Ensemble spread : the standard deviation of the ensemble members and represents how different the members are from the ensemble mean.
Ensemble exercise tasks
This exercise has more tasks than the previous ones.
Visualising ensemble forecasts can be done in various ways. During this exercise, in order to understand the errors and uncertainties in the forecast, we will use a number of visualisation techniques.
General questions
- How does the ensemble mean 10m wind fields and MSLP compare to the HRES forecast and analysis?
- Examine the initial diversity in the ensemble and how the ensemble spread and error growth develops. What do the extreme forecasts look like?
- Are there any members that consistently provide a better forecast? Can you identify the members close to observations/analysis both from a qualitative and quantitative approach?
Available plot types
For these exercises please use the Metview icons in the row labelled 'ENS'.
ens_rmse.mv : this is similar to the oper_rmse.mv in the previous exercise. It will plot the root-mean-square-error growth for the ensemble forecasts.
ens_to_an.mv : this will plot (a) the mean of the ensemble forecast, (b) the ensemble spread, (c) the HRES deterministic forecast and (d) the analysis for the same date.
ens_to_an_runs_spag.mv : this plots a 'spaghetti map' for a given parameter for the ensemble forecasts compared to the analysis. Another way of visualizing ensemble spread.
stamp.mv : this plots all of the ensemble forecasts for a particular field and lead time. Each forecast is shown in a stamp sized map. Very useful for a quick visual inspection of each ensemble forecast.
stamp_diff.mv : similar to stamp.mv except that for each forecast it plots a difference map from the analysis. Very useful for quick visual inspection of the forecast differences of each ensemble forecast.
Additional plots for further analysis:
pf_to_cf_diff.mv : this useful macro allows two individual ensemble forecasts to be compared to the control forecast. As well as plotting the forecasts from the members, it also shows a difference map for each.
ens_to_an_diff.mv : this will plot the difference between an ensemble forecast member and the analysis for a given parameter.
Getting started
Please refer to the handout showing the storm tracks labelled 'ens_oper' during this exercise. It is provided for reference and may assist interpreting the plots.
Each page shows 4 plots, one for each starting forecast lead time. The position of the symbols represents the centre of the storm valid 28th Oct 2013 12UTC. The colour of the symbols is the central pressure.
The actual track of the storm from the analysis is shown as the red curve with the position at 28th 12Z highlighted as the hour glass symbol. The HRES forecast for the ensemble is shown as the green curve and square symbol. The lines show the 12hr track of the storm; 6hrs either side of the symbol.
Note the propagation speed and direction of the storm tracks. The plot also shows the centres of the barotropic low to the North.
Q. What can be deduced about the forecast from these plots?
The plots in the handout can also be found in the 'pics' folder.
Task 1: RMSE "plumes"
This is similar to task 1 in exercise 2, except now the RMSE curves for all the ensemble members from a particular forecast will be plotted. All 4 forecast dates are shown.
Using the ens_rmse.mv icon, right-click, select 'Edit' and plot the curves for 'mslp'. Note this is only for the European region. The option to plot over the larger geographical region is not available.
Q. What features can be noted from these plumes?
Q. How do these change with different forecast lead times?
Note there appear to be some forecasts that give a lower RMS error than the control forecast. Bear this in mind for the following tasks.
If time
- Explore the plumes from other variables.
- Do you see the same amount of spread in RMSE from other pressure levels in the atmosphere?
Task 2: Ensemble spread
In the previous task, we have seen that introducing uncertainty into the forecast by starting from different initial conditions and enabling the stochastic parameterizations in IFS can result in significant differences in the RMSE (for this particular case and geographical region).
The purpose of this task is to explore the difference in more detail and look in particular at the 'ensemble spread'.
Refer to the storm track plots in the handout in this exercise.
Use the ens_to_an.mv icon and plot the MSLP and wind fields. This will produce plots showing: the mean of all the ensemble forecasts, the spread of the ensemble forecasts, the operational HRES deterministic forecast and the analysis.
Q. How does the mean of the ensemble forecasts compare to the HRES & analysis?
Q. Does the ensemble spread capture the error in the forecast?
Q. What other comments can you make about the ensemble spread?
If time:
- change the 'run=' value to look at the mean and spread for other forecast lead times.
- set the 'members=' option to change the number of members in the spread plots.
e.g. try a "reduced" ensemble by only using the first 5 ensemble members: "members=[1,2,3,4,5]".
Task 3: Spaghetti plots - another way to visualise spread
A "spaghetti" plot is where a single contour of a parameter is plotted for all ensemble members. It is another way of visualizing the differences between the ensemble members and focussing on features.
Use the ens_to_an_runs_spag.mv icon. Plot and animate the MSLP field using the default value for the contour level. This will indicate the low pressure centre. Note that not all members may reach the low pressure set by the contour.
Note that this macro may animate slowly because of the computations required.
Experiment with changing the contour value and (if time) plotting other fields.
Task 4: Visualise ensemble members and difference
So far we have been looking at reducing the information in some way to visualise the ensemble.
To visualise all the ensemble members as normal maps, we can use stamp maps. These are small, stamp sized contour maps plotted for each ensemble member using a small set of contours.
There are two icons to use, stamp.mv and stamp_diff.mv. Plot the MSLP parameter for the ensemble. Repeat for wind field.
Q. Using the stamp and stamp difference maps, study the ensemble. Identify which ensembles produce "better" forecasts.
Q. Can you see any distinctive patterns in the difference maps? Are the differences similar in some way?
If time:
Use the macros to see how the perturbations are evolving; use ens_to_an_diff.mv to compare individual members to the analyses.
Find ensemble members that appear to produce a better forecast and look to see how the initial development in these members differs. Start by using a single lead time and examine the forecast on the 28th.
- Select 'better' forecasts using the stamp plots and use ens_to_an.mv to modify the list of ensembles plots. Can you tell which area is more sensitive in the formation of the storm?
- use the pf_to_cf_diff macro to take the difference between these perturbed ensemble member forecasts from the control to also look at this.
Use 'mapType=1' to see the larger geographical area (please note that due to data volume restrictions, this mapType only works for the MSLP parameter).
Task 5: Cumulative distribution function at different locations
Recap
| The probability distribution function of the normal distribution or Gaussian distribution. The probabilities expressed as a percentage for various widths of standard deviations (σ) represent the area under the curve. |
|---|
Figure from Wikipedia. |
Cumulative distribution function for a normal |
|---|
Figure from Wikipedia. |
Cumulative distribution function (CDF)
The figures above illustrate the relationship between a normal distribution and its associated cumulative distribution function.The CDF is constructed from the area under the probability density function.
The CDF gives the probability that a value on the curve will be found to have a value less than or equal to the corresponding value on the x-axis. For example, in the figure, the probability for values less than or equal to X=0 is 50%.
The shape of the CDF curve is related to the shape of the normal distribution. The width of the CDF curve is directly related to the value of the standard deviation of the probability distribution function. For our ensemble, the width is then related to the 'ensemble spread'.
For a forecast ensemble where all values were the same, the CDF would be a vertical straight line.
Plot the CDF for 3 locations
This exercise uses the cdf.mv icon. Right-click, select 'Edit' and then:
- Plot the CDF of MSLP for the 3 locations listed in the macro.e.g. Reading, Amsterdam, Copenhagen.
- If time, change the forecast run date and compare the CDF for the different forecasts.
Q. What is the difference between the different stations and why? (refer to the ensemble spread maps to answer this)
Q. How does the CDF for Reading change with different forecast lead (run) dates?
Forecasting an event using an ensemble : Work in teams for group discussion
Ensemble forecasts can be used to give probabilities to a forecast issued to the public.
- Plot and animate MSL + 500hPa maps showing track of Nadine
> 1 : Nadine MSLP and T2m (or better SST) tracking 15-20 september
> 2 : Satellite views on the 20th (provided by Etienne, if possible to put on the VM)
> 3 : Studying of the horizontal maps (analysis + forecasts)
> 4 : Studying and building of the vertical x-sections (analysis + forecasts)
Notes from Frederic: email 7/4/16
day 1
day 2
I suggest to focus on the period before the 20-25 September and to study
Nadine and the cutoff, not yet at the impact o the Mediteranean area (we
leave that for later)
Agreed. The first exercise will be to examine the track & changes in the storm using the analyses.
*Here are some inputs concerning Day2 and Nadine's study on day 2:
*1)* T1279 Analysis 0920 + t+96 deterministic forecast 0924 (t+96h) -->Agreed.
focusing on the interaction between Nadine and the cutoff. Maybe an
extra plot of the forecasted rainfall at t+96 over France ?
*2)* Ens T639 forecasts : I saw that T639 is the 2012 operational
ensemble resolution, so we will see the same bifurcation in the
scenarios as explained in Pantillon : the visualization of the spread,
the plumes, the spaghettis, ... will help here. I am sure you have great
ideas on this topic. Maybe we can propose some horizontal maps of each
(or some) members ?
I think the exercises we used last year will fit well here. I will start drafting the exercises on the wiki and ask you to help and comment.
I presume the exercises should be in English? Or should we do a side-by-side English/French version?
*3)* PCA and clustering : if you manage to put it in Metview this will
be great lo look at the 2 distinct patterns. I asked Florian Pantillon
his NCL sources to do the trick. I'll use it to build an extra NCL
exercice with PCA, clustering and compositing, if we have time. The file
format needed will be netcdf.
We think our PCA code can be used to reproduce fig 5. We thought we could also reproduce fig 6 but instead of dots plot the ensemble number. Then the students can build the clusters (Fig. 7) but grouping the ensemble members together?
We (here) need to try this and see how far we can reproduce the rest of the figures in the paper.
*4)* Ensemble runs : initial (EDA+SV) and model (SPPT+SKEB) : same as
last year
SST experiment might be too much, except if we shorten the ensemble study...
See above. My preference after talking with people here is to use the comparison between 2012 operational ensemble and 2016 operational ensemble. The lower res (T319) ensembles; control (EDA+SV), (SPPT_SKEB) ensembles for this case are running now and we can include the data (as long as filesize does not become an issue). But honestly, I do not think there will be time. I will leave it to you to decide!
We counted 7hrs total for the practicals (not including the SCM). Part of that time the students will need to prepare some plots for the discussion on Friday.
I am concerned about the time available. Perhaps my talk on weds 9.30-10 could be shortened to 15mins.
*Day 3
SCM experiments
For the SCM we thought that it might be interesting to use the SCM for a point near Toulouse that experienced very heavy rainfall during HYMEX. Then we get the students to adjust the entrainment rates (similar to the convection exercises here) to see what impact it has on the precipitation?
Exercise 2.
As I am writing I am beginning to wonder if we should not make 2 groups : one for task 4 and one for task 5. Tasks 1-3 would be for all students. This would allow to keep the CDO task. What do you think ?
As I am writing I am beginning to wonder if we should not make 2 groups : one for task 4 and one for task 5. Tasks 1-3 would be for all students. This would allow to keep the CDO task. What do you think ?
On 04/05/16 15:25, FERRY Frédéric wrote:
*T1279 Analysis* : 20121020 0000UTC to 20121025 0000UTC --> Only the 20
september analysis will be looked at but I assume you need to get the
other analysis to compute the RMSE in day 2 ?
*Extended analysis* : 15-20 September just for MSLP and T2m (or better
the SST) --> Nadine tracking before the 20th
15-20th ok. We'll produce all the variables for this date range as the students can animate the fields if they have them. If data size is an issue we'll restrict 15-19th to just T2m & MSLP.
The SST is a good point as this will be interesting to compare between the analysis and the forecasts. I'll add it to the list of fields.
I assumed the students would look at the analyses on all days. The first task would be to look at the track of Nadine & the cut-off low and animate different fields across the date range. This is very easy to do with the metview macros. We can also get them to plot difference maps between the forecasts and the analyses (and yes we need all the days to compute RMSE) if you think that's instructive.
As the data is time-consuming to extract (it will take ~ 7 days), I would rather get more than we think we might use in the exercises to allow the students to explore their own ideas during the workshop.
> *T1279 Deterministic forecast* : 20 000UTC analysis and 20121025 0000
> UTC deterministic forecast (t+120).
Ok. We'll use 6hrly data throughout.
*Extended deterministic forecast* : 20-28 September just for MSLP :
Etienne told me that the ECMWF model of the 20 000UTC proposed a very
extreme situation on the 28th, with a storm over Gibraltar. This would
be a way to illustrate the limits of a deterministic approach.
I'd rather not add another 3 days of data (26-28) to show a single forecast. If this is just a short example to the students during the workshop, can we plot these and include just the figures? (ie. not an interactive exercise)
*Horizontal maps* (analysis + forecast) :
All ok apart from:
3 : Equivalent potential temperature at 850 hPa + winds at 850 hPa +
vertical velocity at 600hPa + MSLP in background --> focussing on the
Can we use 700hPa VV instead of 600, to be consistent with data on other levels? We will need VV on multiple levels in order to plot the x-sections (see below), though these will only be available 00Z on each day. The horiz. maps will have VV available 6hrly but on selected levels only (we're proposing 200, 500, 700, 850).
*Vertical x-sections* in the cutoff and in the low :
**PV + winds (preferably normal winds) + if possible potential
temperature --> to look at the cold core or warm core structure of the
systems on the vertical and the signature in PV and winds.
PV + relative humidity + vertical velocity --> a more classical
x-section that we use to see if a PV anomaly is accompanied with
vertical motion or not.
For these x-sections we can choose 3 or 4 times that appear to be
interesting. Interactivity would be good, to make the students look a
little bit in the code.
We will not have enough space to store all pressure levels for the x-sections 6hrly, so we propose only each day at 00Z. Once we have the data extracted from the archive, we might need to review this.
In Metview it is possible to graphically draw a line on a horizontal map (e.g. MSLP) to identify the x-section location. The end-pts can then be used to plot the x-section itself. The students can experiment finding the centre & plotting x-sections from the different days.
Q: How do you want to plot the ensemble data? Plot individual members, groups of members or x-section of the cluster of members
*Satellite* : we have the satellite images of the situation (IR, WV,
cloud classification, IR-Visible composite). We can send them to you to
put on the VM.
Great!
Proposed tasks for Day 1 :
5 : Beyond D+5 deterministic scenario : MSLP only
See above. I think this will be too much data for us to include beyond day 5.
Concerning the ensemble runs, 6 hourly data is OK. If you have space on
the VM it would be interesting to go up to D+10 (or D+15). This would
allow to try and look at the extreme member over Gibraltar on the 28
September.
Task 1 : forecast error
Task 2 : compare forecast to analysis
we think task 2 fits better on day 1 when the students visualize the analyses & HRES forecast. If time, they look at HRES forecast error though that could be day 2. Day 2 then becomes mostly about the ensemble.
Task 3 : visualize ensembles (plumes, ensemble spread, spaghetti, stamp,
CDO)
--> These 3 tasks from last year are very interesting. To gain time
maybe that we should put a group on each item for task 3 or suppress
task 2 ? The CDO adds a "statistical" taint to the workshop do you think
we can adapt it to our case ?
Maybe we can put the students into the role of the forecaster during HyMEX; given the CDF plots & clusters, what would the students have decided to do and why?
We did this last year asking the question what would the participants do if the Queen had a birthday garden party and the forecast showed a risk of severe weather It was a good discussion.
Perhaps this could be led by Etienne based on Hymex experience?
Task 4 : PCA and clustering
Thanks for this. It will also be interesting to see what the latest operational ensemble does with this case (we do not yet know!). Comparing the two will be interesting.
To sum up : 15-25 for the analysis, 6 hourly. The 15-19 analysis will be for Nadine study mainly. The 20th is the beginning of the forecast. Forecast range is 5 days (20-25). So we need also the 20-25 analyses for the comparison analysis VS forecast. If you can archive some figures of MSLP for the deterministic forecast just to check the storm over Gibraltar. Concerning the VV field 700 is OK to avoid extracting another level. For the x sections 00Z is sufficient, 200 500 700 850 should be enough to see something I hope. I did not know the x-sections was easy to do in Metview, this will save some time ! I’ll ask Etienne to get the satellite images for the 20th. Sorry for the tephigrams ! Day 1 synopsis : I agree that we have to do the comparison deterministic VS analysis on the first day to concentrate on the ensemble on the second day. Task 1 : Animating analysis from 15 to 20, mainly tracking Nadine Task 2 : studying of the deterministic forecasts from 20-25 september, and the interaction between Nadine and cutoff Task 3 : compare the deterministic forecast to the analysis. I like your idea of plotting the difference maps, this could be a better way to localize the area where the model was wrong. We can leave the forecast error and the RMSE if time I am not sure I understand you question about the ensemble data plotting ? I was thinking that on the second day we could visualize the classical ensemble products : plumes, spread, spaghettis, stamps + the figures in Pantillon concerning the cluster composites and the cluster tracks. What do you mean about the x-section of the cluster ? Maybe you have other suggestions.
So day 2 « menu » would be : Looking at the ensemble products and the cluster products and making a decision for Hymex field campaign —> They will have Etienne's forecaster feedback the day after. I’ll ask Etienne his ideas for the workshop tasks on this topic. Looking at the impact of ocean coupling on the ensemble prediction. Tell me if you manage to redo the clustering and the composites in Metview, I hope it will work. If you manage to redo figures 5 6 7 8 and 10 I think I’ll have to tell Jean-Pierre to focus more during his presentation on the vortex-vortex interaction and the CRM sensitivity experiments he made. This will leave the cluster analysis for the students to discover.
Hello Glenn, Here are a few comments concerning your previous emails : 2- Véronique Ducrocq could play the role of an HyMeX operation director being the client of the students' forecast. This forecasting exercise could be done by the 8 students following the forecasting option (with me as their "teacher"), whereas the 18 others (informatic or statistic options) could keep doing more sensitivity tests while manipulating the code of the model (with Frédéric and you). 3- It would be very interesting to briefly tackle with the ECMWF Data Targeting System which was one of the observation strategies used during HyMeX SOP1. I precisely asked Véronique Ducrocq to speak about DTS during her presentation on Day 1. 4- ARPEGE and IFS deterministic charts are available at the French Met School between 18th and 24th sept (except the 20th runs unfortunately !). As far as I was the HyMeX forecaster myself before the 24th sept. event, I would be very interesting in the MSLP fields from the 20th 00UTC run between 25th and 28th sept. , in order to be able to illustrate (in my own Day 3 presentation) the propagation of this impressive "Gibraltar storm" I mentionned into my daily meeting report. A 6h step would be perfect, even if it is only a paper-scanned version...
Appendix
Datasets available
The following datasets are available on the Virtual Machine for this workshop:
Further reading
For more information on the stochastic physics scheme in (Open)IFS, see the article:
Shutts et al, 2011, ECMWF Newsletter 129.
Acknowledgements
We gratefully acknowledge the following for their contributions in preparing these exercises. From ECMWF: Glenn Carver, Sandor Kertesz, Linus Magnusson, Iain Russell, Simon Lang, Filip Vana. From ENM/Meteo-France: Frédéric Ferry, Etienne Chabot, David Pollack and Thierry Barthet for IT support at ENM.