| Table of Contents |
|---|
Introduction
It For the projects like TIGGE, S2S, UERRA or LC_WFV it is important to have good checking tools deployed on the both ends of the processing chain, provider's and ECMWF's ones, to avoid common operational problems related to preparation and dispatching the data for the project.
Find below some examples what we use to do at ECMWF as a part of the operational data processing suites. Please If you are our data provider, please try to install similar checks on your end to minimise our common future support effort. Only by doing that we could handle It would help us handling increasing number of similar projects in a sustainable way and without additional resources.
...
Data encoding check
- the proper data encoding to the required data exchange format (WMO GRIB2) can be checked by the dedicated ECMWF tools
- Checking the file
Comparing data content against reference
- the number of the fields must be as expected
- the actual field list must be the same as expected
- see example in Comparing data content against reference how to create a reference fields list from your input data and compare it against the actual data content
Data quality check
- the best and most complete quality control can be done by a custom made tools only, e.g. during the forecast computation by the model
- the basic quality control can be done using the dedicated ECMWF tools which are simply checking the allowed limits (min/max values in each grid point) of each parameter
Checking data transfer success
If data is pushed to ECMWFour premises, the following basic check should be done after each file transfersteps can ensure smooth dispatching:
Make the transferred file "invisible"
- e.g. by adding the dot before the file name and rename it only after the transfer is finished
- that way the ECMWF's automated acquisition action will not be triggered prematurely
Compare the file sizes before and after the transfer
- the file size must be identical to ensure the file was not corrupted during the transfer do it (should be done still before the file renaming renaming)