Summary -Proposed Tar File Structure


Summary

  • 1 tar file contains 1 variable (for a given start date, for all ensemble members);
  • The number of realizations included are indicated in the tar file name (using n# - n## where # = index of first realization in that tar file  and ## = number of last realization in that tar file);
  • The individual tar file size limit on ECFS is 32 GB. Where this is exceeded, 2 (or more) tar files are used, with the number of ensemble members indicated in the tar file name.
  • The representative figures below are derived from sample data supplied by Meteo France.
  • A checksum file corresponding to each tar file should also be created and supplied with the tar file

For hindcast data

Example below based on sample  Hindcast data from Meteo France (forecast data should be handled in a similar way). They are classified by the data product "Type", "Level" and "Realm"

Note: a total of 25 Ensemble members

ProviderTypeLevelRealmTime ResolutionTar file nameExample variableNumber of ensemble membersEstimated tar file size (GB)Number of variables of this type/level/realm
Meteo France

Hindcast

surface

atmos

6hr

lfpw_System6-v20170501_hindcast_S1993100100_atmos_6hr_surface_tas_n1-n25.tar

tas

25

3.05

6

Meteo France

Hindcast

surface

atmos

day

lfpw_System6-v20170501_hindcast_S1993100100_atmos_day_surface_lwee_n1-n25.tar

lwee

25

1

16

Meteo France

Hindcast

surface

land

day

lfpw_System6-v20170501_hindcast_S1993100100_land_day_surface_mrlsl_n1-n25.tar

mrlsl

25

0.4

6

Meteo France

Hindcast

pressure

atmos

12hr

lfpw_System6-v20170501_hindcast_S1993100100_atmos_12hr_pressure_va_n1-n25.tar

va

25

23.325

5

Meteo France

Hindcast

surface

ocean

6 hr

lfpw_System6-v20170501_hindcast_S1993100100_ocean_6hr_surface_tso_n1-n25.tar

tso

25

2.15

1

Meteo France

Hindcast

surface

seaIce

day

lfpw_System6-v20170501_hindcast_S1993100100_seaIce_day_surface_sic_n1-n25.tar

sic

25

0.2

1

So the first tar file in the table above ( "lfpw_System6-v20170501_hindcast_S1993100100_atmos_6hr_surface_tas_n1-n25.tar" ) would contain all 25 ensemble members for the tas variable for the start date S19931001. Similar tar files would be created for tasmax, tasmin etc.


For forecast data

Note a total of 51 ensemble members; for pressure level variables where the volume of all the members would be greater than 32GB, 2 tar files should be created for each variable (see yellow boxes below)

ProviderTypeLevelRealmTime ResolutionTar file nameExample variableNumber of ensemble membersEstimated tar file size (GB)Number of variables of this type/level/realm
Meteo France

Forecast

surface

atmos

6hr

lfpw_System6-v20170501_forecast_S1993100100_atmos_6hr_surface_tas_n1-n51.tar

tas

51

6.1

6

Meteo France

Forecast

surface

atmos

day

lfpw_System6-v20170501_forecast_S1993100100_atmos_day_surface_lwee_n1-n51.tar

lwee

51

2

16

Meteo France

Forecast

surface

land

day

lfpw_System6-v20170501_forecast_S1993100100_land_day_surface_mrlsl_n1-n51.tar

mrlsl

51

0.8

6

Meteo France

Forecast

pressure

atmos

12hr

lfpw_System6-v20170501_forecast_S1993100100_atmos_12hr_pressure_va_n1-n25.tar

va

25

23.325

5

Meteo France

Forecast

pressure

atmos

12hr

lfpw_System6-v20170501_forecast_S1993100100_atmos_12hr_pressure_va_n26-n51.tar

va

26

23.325

5

Meteo France

Forecast

surface

ocean

6 hr

lfpw_System6-v20170501_forecast_S1993100100_ocean_6hr_surface_tso_n1-n51.tar

tso

51

4.3

1

Meteo France

Forecast

surface

seaIce

day

lfpw_System6-v20170501_forecast_S1993100100_seaIce_day_surface_sic_n1-n51.tar

sic

51

0.4

1


So the forecast pressure level data for va would be sent in 2 tar files, one containing 25 members, and the other containing 26 members (lfpw_System6-v20170501_forecast_S1993100100_atmos_12hr_pressure_va_n1-n25.tar and lfpw_System6-v20170501_forecast_S1993100100_atmos_12hr_pressure_va_n26-n51.tar ).