Interpretation of blacklist information.
The IFS monthly blacklist/whitelist information is sent out via e-mail to provide specific details about data usage in the ECMWF data assimilation system.
The IFS data usage decisions are not only determined by the blacklist: There are also other data selection algorithms and a variety of quality control mechanisms in place.
Stations present in the exclusion list (the blacklist) are not necessarily of poor quality. There are many reasons why stations may be excluded from assimilation for a period of time:
- Large differences between observed and model values: This may be due to a problem with either the model or the observations. Observations may be removed while investigations are carried out, or for a period of time until the situation has improved. These decisions are reviewed each month based on the most recent monitoring statistics. The EUMETNET Observations Programme routinely follow up on suspect stations (see below).
- New stations: These will be passively monitored (thus 'blacklisted') for a certain period of time until their quality has been assessed and they can be actively used (assimilated) by the IFS data assimilation.
- Representativity: The model is unable to represent small-scale feature below the scale resolved by the global ECMWF NWP model. The data assimilation system cannot make good use of observations in places where the lack of model resolution creates large errors, for example in complex terrain, on coastlines and steep orography.
- TAC to BUFR migration: The transition from TAC to BUFR is at ECMWF managed through updates to the blacklist. This is clearly indicated in the change document, i.e. denoted by subtype of the observations in the whitelist sections. There are some complications due to this migration (in the blacklist file some entries effectively say "use BUFR from this station"), see the overview at https://confluence.ecmwf.int/display/TCBUF/ .
- Some surface pressure observations are bias corrected at ECMWF. If another NWP centre doesn't bias correct this station the data may be detrimental.
The blacklist information shared in the monthly e-mails is tailored for use with the IFS global model and needs to be used with caution in other NWP systems, especially those with higher spatial resolution, where representativeness is much less of an issue than in the (relatively) lower-resolution global model. It should be noted that parts of the information is encoded in the form of whitelist, that is, listing stations to be used. Please see user User's Guide to Blacklisting page for more information on blacklisting. Table for codetypes and subtypes can be found here.
ECMWF aims to implement IFS blacklist changes in the second week of each month. During the first week, analysts evaluate automated and manual proposals for changes and run model experiments to test the blacklist change. Operational change requests are then submitted in the second week of the month (typically Monday or Tuesday). Occasionally operational IT constraints mean the change is postponed by a few days.
Every three months there is a synchronisation with WMO's OSCAR station database. This generally warrants a short blacklist change proposal of its own.
If you would like to be added to the e-mail distribution of this blacklist information please send an email to ifs_conventional_blacklist-request@lists.ecmwf.int with the word subscribe or unsubscribe as Subject. Please note that this service is available to ECMWF Member and Cooperating States only.
Resolution of station issues
Data quality for in situ observations is routinely monitored by the EUMETNET Observations Programme. Their information should be relied upon for routine monitoring of station quality, identification of issues and issues resolution. Please refer to 'Observations Programme/Observation Monitoring' at the EUMETNET/EUCOS portal: https://portal.eumetnet.eu