Versions Compared

Old Version 93

changes.mady.by.user Unknown User (nagc)

Saved on

compared with

New Version 94

changes.mady.by.user Unknown User (nagc)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

ens_2016: This dataset is a reforecast of the 2012 event using the ECMWF operational ensemble from March 2016. Two key differences between the 2016 and 2012 operational ensembles are: higher horizontal resolution, and coupling of NEMO ocean model to provide SST from the start of the forecast.

...

Visualising ensemble forecasts can be done in various ways. During this exercise , we will use a number of visualisation techniques in order to understand the errors and uncertainties in the forecast, we will use a number of visualisation techniques.

Key parameters: MSLP and z500.  We suggest concentrating on viewing these fields. If time, visualize other parameters (e.g. PV320K).

...

Panel

For these exercises please use the Metview icons in the row labelled 'ENS'.

ens_rmse.mv : this is similar to the hres_rmse.mv in the previous exercise. It will plot the root-mean-square-error growth for the ensemble forecasts.

ens_to_an.mv : this will plot (a) the mean of the ensemble forecast, (b) the ensemble spread, (c) the HRES deterministic forecast and (d) the analysis for the same date.

ens_to_an_runs_spag.mv : this plots a 'spaghetti map' for a given parameter for the ensemble forecasts compared to the analysis. Another way of visualizing ensemble spread.

stamp.mv : this plots all of the ensemble forecasts for a particular field and lead time. Each forecast is shown in a stamp sized map. Very useful for a quick visual inspection of each ensemble forecast.

stamp_diff.mv : similar to stamp.mv except that for each forecast it plots a difference map from the analysis. Very useful for quick visual inspection of the forecast differences of each ensemble forecast.

 

Additional plots for further analysis:

pf_to_cf_diff.mv : this useful macro allows two individual ensemble forecasts to be compared to the control forecast. As well as plotting the forecasts from the members, it also shows a difference map for each.

ens_to_an_diff.mv : this will plot the difference between the ensemble control, ensemble mean or an individual ensemble forecast member and the analysis for a given parameter.

...

If working in groups, each group could follow the tasks below with a different ensemble forecast. e.g. one group uses the 'ens_oper', another group uses 'ens_2016' and so on.A key question is: which is the better ensemble?

Choose your ensemble dataset by setting the value of 'expId':', either 'ens_oper' or 'ens_2016' for this exercise.

Code Block
languagebash
titleEnsemble forecast datasets available in the macros
#The experiment. Possible values are:
# ens_oper = operational ENS
# ens_2016 = 2016 operational ENS
# ens_both = OpenIFS (EDA+SV+SPPT+SKEB)
# ens_initial = OpenIFS (EDA+SV)
# ens_model = OpenIFS (SPPT+SKEB) 
expId="ens_oper"
expId="ens_oper"

Ensemble forecast Ensemble forecast performance

In these tasks, the performance of the ensemble forecast is studied.

Panel
borderColorred
titleOverall questions to consider
  1. How does the ensemble mean MSLP and Z500 fields compare to the HRES forecast and analysis?
  2. Examine the initial diversity in the ensemble and how the ensemble spread and error growth develops.  What do the extreme forecasts look like?
  3. Are there any members that consistently provide a better forecast?

Comparing the two ensembles, ens_oper and ens_2016, which is the better ensemble for this case study?

Task 1: RMSE "plumes"

This is similar to task 1 in exercise 2, except now the RMSE curves for all the ensemble members from a particular forecast will be plotted.

Using Right-click the ens_rmse.mv icon, right-click, select 'Edit' and plot the curves for 'mslp' and 'z500'.Turn off clustering in the macro

Change 'expID' for your choice of ensemble.

Code Block
languagebash
titleMake sure 'clustering' is off for this task!
useClusters="off"

...

Task 2: Ensemble spread

In the previous task, we have seen that introducing uncertainty into in the forecast by starting from different initial conditions and enabling the stochastic parameterizations in IFS can result in significant differences in the RMSE (for this particular case and geographical region).

...

Use the ens_to_an.mv icon and plot the MSLP and z500. This will produce plots showing: the mean of  all the ensemble forecasts, the spread of the ensemble forecasts, the operational HRES deterministic forecast and the analysis.

Change 'expId' if required.

Animate this plot to see how the spread grows.

...

Panel
borderColorred
titleQuestions
  1. How does the mean of the ensemble forecasts compare to the HRES & analysis?
  2. Does the ensemble spread capture the error in the forecast?
  3. What other comments can you make about the ensemble spread?

If time:

  • try setting the The 'members=' option is used to change the number of members in the spread plots.
      Try creating your own cluster: e.g. try a "reduced" ensemble by only using the first 5 ensemble members: "members=[1,2"members=[1,3,4,5,7,8,9]".

Task 3: Spaghetti plots - another way to visualise spread

A "spaghetti" plot is where a single contour of a parameter is plotted for all ensemble members. It is another way of visualizing the differences between the ensemble members and focussing on features.

Use the ens_to_an_runs_spag.mv icon. Plot and animate the MSLP and z500 fields using your suitable choice for the contour level. Find a value that highlights the low pressure centres. Note that not all members may reach the low pressure set by the contour.

...

There are two icons to use, stamp.mv and stamp_diff.mv.

Plot Use stamp.mv to plot the MSLP and z500 fields in the ensemble and other fields of interest.

The stamp map is slow to plot as it reads a lot of data. Rather than animate each forecast step, a particular date can be set by changing the 'steps' variable.

Code Block
languagebash
titleSet date/time to 24-09-2012 00Z
#Define forecast steps
steps=[2012-09-24 00:00,"to",2012-09-24 00:00,"by",6]

 

Difference stamp maps

Make sure useClusters="off" for this task.
Precipitation over France
Use stamp.mv and plot total precipitation ('tp') over France (mapType=2) for 00Z 24-09-2012 (compare with Figure 2 in Pantillon).
Note, stamp_diff.mv cannot be used for 'tp' as there is no precipitation data in the analyses.
Difference stamp maps
Use the stamp_diff.mv plot to look at the differences between the ensemble members and the Use the stamp_diff.mv plot to look at the differences between the ensemble members and the analysis. It can be easier to understand the difference in the ensembles by using difference stamp maps.
...
 Panel
borderColorred
titleQuestions
  1. Using the stamp and stamp difference maps, study the ensemble. Identify which ensembles produce "better" forecasts.
  2. Can you see any distinctive patterns in the difference maps? Are the differences similar in some way?
If time:
Compare ensemble members
Following analysis of the stamp maps, it can be useful to  Use the macros to see how the perturbations are evolving; use ens_to_an_diff.mv to compare individual members to the analyses.
 Panel
titleUse ens_to_an_diff to compare an ensemble member to the analysis
 Suppose we want to animate the difference of ensemble member 30 to the analysis.
Set:
 Code Block
param="mslp"
ensType="pf30"
 
Find ensemble members that appear to produce a better forecast and look to see how the initial development in these members differsFind ensemble members that appear to produce a better forecast and look to see how the initial development in these members differs. Start by using a single lead time and examine the forecast on the 28th.
  • Select 'better' forecasts using the stamp plots and use ens_to_an.mv to modify the list of ensembles plots. Can you tell which area is more sensitive in the formation of the storm?
  • use the pf_to_cf_diff macro to take the difference between these perturbed ensemble member forecasts from the control to also look at this.
...
  • Plot PV at 320K. What are the differences between the forecast? Upper tropospheric differences played a role in the interaction of Hurricane Nadine and the cut-off low.
 
 
 
...
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Notes from Frederic: email 7/4/16
 
day 2

1) T1279 Analysis 0920 + t+96 deterministic forecast 0924 (t+96h) --> focusing on the interaction between Nadine and the cutoff. Maybe an extra plot of the forecasted rainfall at t+96 over France ?
2) Ens T639 forecasts : I saw that T639 is the 2012 operational ensemble resolution, so we will see the same bifurcation in the scenarios as explained in Pantillon : the visualization of the spread, the plumes, the spaghettis, ... will help here. I am sure you have great ideas on this topic. Maybe we can propose some horizontal maps of each (or some) members ?
 
3) PCA and clustering : if you manage to put it in Metview this will be great lo look at the 2 distinct patterns. I asked Florian Pantillon his NCL sources to do the trick. I'll use it to build an extra NCL exercice with PCA, clustering and compositing, if we have time. The file format needed will be netcdf.
4) Ensemble runs : initial (EDA+SV) and model (SPPT+SKEB) : same as last year
See above. My preference after talking with people here is to use the comparison between 2012 operational ensemble and 2016 operational ensemble. The lower res (T319) ensembles; control (EDA+SV), (SPPT_SKEB) ensembles for this case are running now and we can include the data (as long as filesize does not become an issue). But honestly, I do not think there will be time. I will leave it to you to decide!
 
day 3
Etienne's presentation in the morning
SCM experiments
...
Exercise 2.
Task 2 : compare forecast to analysis
Task 3 : visualize ensembles (plumes, ensemble spread, spaghetti, stamp, CDF)
--> These 3 tasks from last year are very interesting. To gain time maybe that we should put a group on each item for task 3 or suppress task 2 ? The CDF adds a "statistical" taint to the workshop do you think we can adapt it to our case ?
Task 4 : PCA and clustering
Figure 5 shows that EOF1 accounts for 3/4 of the variance. This dipole pattern is typical when tropical interact with mid latitudes. No need to spend a lot of time on this.
Figure 6 is much more interesting. It Allows to see that we can choose 2 clusters containing approximately half of the members. The deterministic forecast is close to the two outliers and the control and the analysis belong to cluster 1.
From Figure 7 we see that cluster 1 corresponds to a cutoff moving eastward over Europe and cluster 2 to a weak ridge over western Europe.
It would be great if we could also do the cluster composite of rainfall from Figure 8 : cluster 1 shows impact on precipitation over The Cévènes whereas cluster 2 shows weak precipitation over the Cévènes.
The plot of the cluster member tracks of Nadine and the cutoff from figure 10 is also very interesting to me, I think we should do it.  We see more clearly that cluster 1 exhibits a weak interaction between cutoff and low and cutoff over Europe. In cluster 2, there is a strong interaction between the cutoff and Nadine and Nadine makes landfall over the Iberian penisula (in model world, is it realistic ?). I don't know if the tracking is easy to do in Metview as it implies to track the cutoff and the low for each member.
Like you said in a previous mail, there is a possibility of interactivity for figures 7 (MSLP and Z500 composites) 8 (wind and RR composite) and 10 (member track). We have to identify by a number the cluster members and if make the students group the members to create the cluster composites. I think it is a good idea.
Task 5 : Sensitivity experiments to the SST coupling
Extended deterministic forecast : 20-28 September just for MSLP : Etienne told me that the ECMWF model of the 20 000UTC proposed a very extreme situation on the 28th, with a storm over Gibraltar. This would be a way to illustrate the limits of a deterministic approach.
Satellite : we have the satellite images of the situation (IR, WV, cloud classification, IR-Visible composite). We can send them to you to put on the VM.
.
 
Before leaving for a long weekend and maybe more, here is some input about the practical session on the 2nd day :
Task 1 : forecast error
Task 2 : compare forecast to analysis
Task 3 : visualize ensembles (plumes, ensemble spread, spaghetti, stamp, CDO)
--> These 3 tasks from last year are very interesting. To gain time maybe that we should put a group on each item for task 3 or suppress task 2 ? The CDO adds a "statistical" taint to the workshop do you think we can adapt it to our case ?
Task 4 : PCA and clustering
If not possible in Metview I can make the students plot with NCL figures 5, 6 and 7 from Pantillon.
Figure 5 shows that EOF1 accounts for 3/4 of the variance. This dipole pattern is typical when tropical interact with mid latitudes. No need to spend a lot of time on this.
Figure 6 is much more interesting. It Allows to see that we can choose 2 clusters containing approximately half of the members. The deterministic forecast is close to the two outliers and the control and the analysis belong to cluster 1.
From Figure 7 we see that cluster 1 corresponds to a cutoff moving eastward over Europe and cluster 2 to a weak ridge over western Europe.
It would be great if we could also do the cluster composite of rainfall from Figure 8 : cluster 1 shows impact on precipitation over The Cévènes whereas cluster 2 shows weak precipitation over the Cévènes.
The plot of the cluster member tracks of Nadine and the cutoff from figure 10 is also very interesting to me, I think we should do it.  We see more clearly that cluster 1 exhibits a weak interaction between cutoff and low and cutoff over Europe. In cluster 2, there is a strong interaction between the cutoff and Nadine and Nadine makes landfall over the Iberian penisula (in model world, is it realistic ?). I don't know if the tracking is easy to do in Metview as it implies to track the cutoff and the low for each member.
Like you said in a previous mail, there is a possibility of interactivity for figures 7 (MSLP and Z500 composites) 8 (wind and RR composite) and 10 (member track). We have to identify by a number the cluster members and if make the students group the members to create the cluster composites. I think it is a good idea.
Task 5 : Sensitivity experiments to the SST coupling
As I am writing I am beginning to wonder if we should not make 2 groups : one for task 4 and one for task 5. Tasks 1-3 would be for all students. This would allow to keep the CDO task. What do you think ?
On 04/05/16 15:25, FERRY Frédéric wrote:
*T1279 Analysis* : 20121020 0000UTC to 20121025 0000UTC --> Only the 20 
september analysis will be looked at but I assume you need to get the 
other analysis to compute the RMSE in day 2 ? 
*Extended analysis* : 15-20 September just for MSLP and T2m (or better 
the SST) --> Nadine tracking before the 20th
...
day 3
Exercise 2.
Task 4 : PCA and clustering
Figure 5 shows that EOF1 accounts for 3/4 of the variance. This dipole pattern is typical when tropical interact with mid latitudes. No need to spend a lot of time on this.
Figure 6 is much more interesting. It Allows to see that we can choose 2 clusters containing approximately half of the members. The deterministic forecast is close to the two outliers and the control and the analysis belong to cluster 1.
From Figure 7 we see that cluster 1 corresponds to a cutoff moving eastward over Europe and cluster 2 to a weak ridge over western Europe.
It would be great if we could also do the cluster composite of rainfall from Figure 8 : cluster 1 shows impact on precipitation over The Cévènes whereas cluster 2 shows weak precipitation over the Cévènes.
The plot of the cluster member tracks of Nadine and the cutoff from figure 10 is also very interesting to me, I think we should do it.  We see more clearly that cluster 1 exhibits a weak interaction between cutoff and low and cutoff over Europe. In cluster 2, there is a strong interaction between the cutoff and Nadine and Nadine makes landfall over the Iberian penisula (in model world, is it realistic ?). I don't know if the tracking is easy to do in Metview as it implies to track the cutoff and the low for each member.
Like you said in a previous mail, there is a possibility of interactivity for figures 7 (MSLP and Z500 composites) 8 (wind and RR composite) and 10 (member track). We have to identify by a number the cluster members and if make the students group the members to create the cluster composites. I think it is a good idea.
Task 5 : Sensitivity experiments to the SST coupling
Extended deterministic forecast : 20-28 September just for MSLP : Etienne told me that the ECMWF model of the 20 000UTC proposed a very extreme situation on the 28th, with a storm over Gibraltar. This would be a way to illustrate the limits of a deterministic approach.
 
*Extended deterministic forecast* : 20-28 September just for MSLP : 
Etienne told me that the ECMWF model of the 20 000UTC proposed a very 
extreme situation on the 28th, with a storm over Gibraltar. This would 
be a way to illustrate the limits of a deterministic approach.
...
All ok apart from:
3 : Equivalent potential temperature at 850 hPa  + winds at 850 hPa + 
vertical velocity at 600hPa + MSLP in background --> focussing on the
Can we use 700hPa VV instead of 600, to be consistent with data on other levels? We will need VV on multiple levels in order to plot the x-sections (see below), though these will only be available 00Z on each day. The horiz. maps will have VV available 6hrly but on selected levels only (we're proposing 200, 500, 700, 850).
Proposed tasks for Day 1 : 
5 : Beyond D+5 deterministic scenario : MSLP only
...
Concerning the ensemble runs, 6 hourly data is OK. If you have space on 
the VM it would be interesting to go up to D+10 (or D+15). This would 
allow to try and look at the extreme member over Gibraltar on the 28 
September.
...
Task 4 : PCA and clustering
Thanks for this. It will also be interesting to see what the latest operational ensemble does with this case (we do not yet know!). Comparing the two will be interesting.
 
So day 2 « menu » would be :
Looking at the ensemble products and the cluster products and making a decision for Hymex field campaign —>  They will have Etienne's forecaster feedback the day after. 
I’ll ask Etienne his ideas for the workshop tasks on this topic.
Looking at the impact of ocean coupling on the ensemble prediction.

Tell me if you manage to redo the clustering and the composites in Metview, I hope it will work.
If you manage to redo figures 5 6 7 8 and 10 I think I’ll have to tell Jean-Pierre to focus more during his presentation on the vortex-vortex interaction and the CRM sensitivity experiments he made. This will leave the cluster analysis for the students to discover.
 Hello Glenn,

Here are a few comments concerning your previous emails :

2- Véronique Ducrocq could play the role of an HyMeX operation director being the client of the students' forecast. This forecasting exercise could be done by the 8 students following the forecasting option (with me as their "teacher"), whereas the 18 others (informatic or statistic options) could keep doing more sensitivity tests while manipulating the code of the model (with Frédéric and you).
3- It would be very interesting to briefly tackle with the ECMWF Data Targeting System which was one of the observation strategies used during HyMeX SOP1. 
I precisely asked Véronique Ducrocq to speak about DTS during her presentation on Day 1.
4- ARPEGE and IFS deterministic charts are available at the French Met School between 18th and 24th sept (except the 20th runs unfortunately !). As far as I was the HyMeX forecaster myself before the 24th sept. event, I would be very interesting in the MSLP fields from the 20th 00UTC run between 25th and 28th sept. , in order to be able to illustrate (in my own Day 3 presentation) the propagation of this impressive "Gibraltar storm" I mentionned into my daily meeting report. A 6h step would be perfect, even if it is only a paper-scanned version...
...