TAC To BUFR Migration

This space is made available for a collaborative discussion on the status and challenges of the migration from Traditional Alphanumeric Codes (TAC) to BUFR. The aim is to share the information on the status of the BUFR data as seen by the users and to provide feedback to WMO and the producers to improve the usability of the data.

The latest version of the migration matrix as taken from the WMO page on migration to TDCF shows that the discontinuation of the category 1 data was planned by November 2014. Many of the reports are now available in BUFR and some TAC bulletins have ceased. 

Status on 19 December 2014

The UK, Eire and Netherlands SYNOPs were switched off in early November 2014 as advertised.  ASDE and ASEU TEMP SHIP reports ceased on 21 November (as did research ship DBLK), originally expected 10 November.  The ASAP TEMP reports are available bilaterally for a limited period if you ask EUMETNET.  This week there have been some reductions of SYNOPs from Romania and Estonia.  

On 18 December TEMPs and SYNOPs (and SHIPs?) from New Zealand stopped, there was a METNO about this  but it had been overlooked by ECMWF and others, leaving a gap in our observation coverage.  The timing of the change is unfortunate - we can't now make any changes to observation usage until the new year.  What is also very regrettable is that the New Zealand BUFR radiosonde data is still as separate parts - it is not properly formatted BUFR - see https://software.ecmwf.int/wiki/display/TCBUF/Structure for notes about the problems involved.  (By coincidence I passed an example of errors in the reformatting to New Zealand a week ago.)  I am in favour of moving to BUFR but not at the cost of data coverage or quality.  

The whole situation is rather a mess.  Just over 30% of radiosonde/surface stations aren't producing any BUFR at all (see https://software.ecmwf.int/wiki/display/TCBUF/Data+availability ) and others are switching off TAC observations when the BUFR hasn't been properly validated.  ECMWF started assimilating some subsets of BUFR radiosonde and surface data on 11 November but I am aware of two other NWP centres that can't use BUFR radiosonde data from the GTS yet.  (Most centres don't process BUFR BUOY data yet, the templates have been in a state of flux.)  Some BUFR reports still have rather basic errors, such as wrong positions, and there are plenty of other more subtle errors/features in the data.  ECMWF is taking the pragmatic approach of processing both TAC and BUFR data and gradually switching on more of the BUFR reports after they have been assessed.  Comparing different versions of the same reports is a very good way of finding problems, but it takes time and we can't look at lots of countries all at once.

In general we would like to see an overlap between the TAC and BUFR reports (ideally six months but this may not always be possible) - and it should be the proper version of BUFR (not radiosonde data still in parts; and for SHIPs the new template not the one that is being phased out).  There is a need for more coordination/information of TAC switch off dates via WMO (as described above a blanket switch off in November 2014 was unrealistic), two months warning of TAC cessation from a particular country would allow us to check and activate the BUFR data.  Developing countries in particular may need more time/help to produce acceptable BUFR reports (especially for radiosondes). 

High resolution data

New data have been released and NWP Centres have put a considerable effort in acquiring and processing new BUFR data which are expected to improve the forecast scores. The extra detail in the observations provided by the transition from TAC to BUFR can be visually represented by the temperature profile in the following figure where BUFR data are reported alongside the corresponding TAC values. For more information regarding the use of these data at ECMWF and their quality see ECMWF section of this wiki.