A flexible format for exchange of domain-averaged verification scores has been defined and described here. For format of station-based scores please look at the respective documents.

(Rev. 10, 16 July 2012)

Document history:

• Rev 10, 16 July 2012: SEEPS added to the score list;
• Rev 9, 12 January 2012: list of domains extended;
• Rev 8, 5 January 2012: correction of the example;
• Rev 7, 12 December 2011: list of domain labels;
• Rev 6, 7 November 2011: units of msl pressure scores changed to hPa; a precision of the record format;
• Rev 5, 1 November 2011: example made compliant to the format definition;
• Rev 4, 1 September 2011: definition of standard deviation;
• Rev 3, 25 August 2011: introduction of units definition paragraph; formatting.
  
  

Format definition

Every score value is described by the full set of key attributes, like its parameter, domain, date, step etc. The keys describing one score value are organised into a record. Each record corresponds to one score value. A record is a collection of pairs key=value separated by commas. A record spans one line, records are separated by newline characters (CR). The value of a key missing in the current record is inherited from the key of the previous record, except for the value parameter v which has to be present in each record.

The keys are:

• centre : the 4-letter WMO identifier of the originating centre (ammc, cwao, ecmf, edzw, egrr, kwbc, lfpw, rjtd, rksl, rums etc);
• model : free model identifier assigned by the centre (to distinguish between potentially different models provided by the centre);
• par : denotes the parameter and level of the verified field and can have the following forms:
  • for upper-air fields (eg. z500hPa, t850hPa, w250hPa) it has the form <P><L> where
    • <P> is the parameter (z for geopotential height, t for temperature, w for wind, r for relative humidity)
    • <L> is the pressure of the isobaric level in hPa and the string hPa
  • for the mean-sea-level pressure it is mslp;
• sc : is the score abbreviation:
  • rmse for root-mean-square error
  • me for mean error (bias)
  • mae for mean absolute error
  • s1 ("S ONE") for S1 skill score
  • ccaf for forecast anomaly correlation
  • cctf for forecast tendency correlation
  • rmsaf for root-mean-square of forecast anomaly
  • rmsav for root-mean-square of verifying analysis anomaly
  • seeps for SEEPS
  • sd for standard deviation of field
• dom : is the domain name, one of the following: nhem shem tropics eurnafr namer asia austnz npol spol;
• ref : indicated whether the scores have been computed against the analysis (ref=an) or observations (ref=ob);
• refs : reference subtype (reserved, currently not in use);
• d : day of the forecast validity in the form YYYYMMDD or month of the mean in the form YYYYMM;
• t : time of the forecast validity in hours of UTC;
• s : forecast step in hours;
• n : number of observations used to compute the given score (if applicable); for monthly means this should be the average number of observation per observation time;
• v : score value; it can be of any usual numeric format (3, 3., 3.0, 0.3E+1,...) but without any leading zeroes (003); the number of significant digits must be sufficient to ensure description of natural score variability between forecast steps, seasons or different centres; if the value is not available the record either should not be printed at all or the value should be set to NIL (v=nil); every record must contain the key v (value of key v is not inheritable from previous record).

Order of key=value pairs is not significant; new keys can be added in future. Case of characters is insignificant, but it should be kept lower-case. Any text following the hash sign # until the end of the line is ignored.

Units of values

Values should be of units following the definitions of scores in the WMO-CBS document on the updated standard verification system.

Example

# a sample bulletin with daily values, compressed by removing
# redundant (repeated) key=value pairs
centre=ecmf,par=z500hpa,sc=rmse,dom=nhem,ref=an,d=20110101,t=0,s=24,v=9.8
s=48,v=12.0
t=12,s=24,v=9.9
s=48,v=12.3
ref=ob,t=0,s=24,n=204,v=13.8
s=48,v=19.0
t=12,s=24,v=13.6
s=48,v=20.03

As seen in this example the rule “the value of a key missing in the current record is inherited from the previous record” can be applied to significantly reduce the size of data file. It is however up to each data producer whether they will compress the file by removing the repeated key=value pairs.

The data files are exchanged via ftp server of LC-DNV. The rules for posting data files, file name conventions etc. are available here.