- Created by Kathy Maskell, last modified by Helene Blanchonnet on May 20, 2022
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
« Previous Version 12 Next »
From January 2022, ECMWF made a wide range of ensemble and high-resolution forecast data openly available
Open data parameters are free of charge when accessed via the public ECMWF website. They can also be delivered via the ECPDS platform to your nominated FTP server, in this case they be subject to service charges.
For additional data, including higher resolutions and more parameters, charges apply and the ECMWF Standard Licence Agreement applies.
Data can be obtained from the ECMWF Product Requirements Catalogue or by contacting a Catalogue Contact Point for more information.
Attribution
The following wording should be attached to the use of this ECMWF dataset:
- Copyright statement: Copyright "© [year] European Centre for Medium-Range Weather Forecasts (ECMWF)".
- Source www.ecmwf.int
- Licence Statement: This data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/
- Disclaimer: ECMWF does not accept any liability whatsoever for any error or omission in the data, their availability, or for any loss or damage arising from their use.
- Where applicable, an indication if the material has been modified and an indication of previous modifications.
The following wording shall be attached to services created with this ECMWF dataset:
- Copyright statement: Copyright "This service is based on data and products of the European Centre for Medium-Range Weather Forecasts (ECMWF)".
- Source www.ecmwf.int
- Licence Statement: This ECMWF data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/
- Disclaimer: ECMWF does not accept any liability whatsoever for any error or omission in the data, their availability, or for any loss or damage arising from their use.
- Where applicable, an indication if the material has been modified and an indication of previous modifications
About this dataset
ECMWF Open Data (real-time) is provided with an open licence (CC-BY-4.0). Data accessed via the public ECMWF web site are free of charge.
This page describes how the data are organised and how they can be downloaded.
A Python package called ecmwf-opendata is available from PyPi that greatly facilitates the access to this dataset. In addition, a series of Jupyter Notebooks have been developed to demonstrate the use of this package.
Availability
The data are released 1 hour after the real-time dissemination schedule.
File format
The files are in GRIB edition 2 format, except for trajectories which are in BUFR edition 4 format. We recommend using ecCodes version 2.24.2 or newer to manipulate the GRIB and BUFR files.
File-naming convention
The files are provided with the following naming convention:
[ROOT]/[yyyymmdd]/[HH]z/[resol]/[stream]/[yyyymmdd][HH]0000-[step][U]-[stream]-[type].[format]
where:
[ROOT]
is the top-level URL of one of the sites hosting the data. See the ECMWF open data: real-time forecasts below for possible values.[yyyymmdd]
is the reference date of the forecasts (base date).[HH]
is the reference time of the forecasts. Values are00
,06
,12
and18
.[resol]
is the horizontal resolution of the data. This is currently set to0p4-beta
and will be changed to0p4
when the beta-testing period is over.[stream]
is the forecasting system that produces the data. Values are:oper
- high-resolution forecast, atmospheric fieldsenfo
- ensemble forecast, atmospheric fields- waef - ensemble forecast, ocean wave fields,
wave
- wave model,scda
- short cut-off high-resolution forecast, atmospheric fields (also known as "high-frequency products")scwv
- short cut-off high-resolution forecast, ocean wave fields (also known as "high-frequency products") andmmsf
- multi-model seasonal forecasts fields from the ECMWF model only.
[step]
is the forecast time step expressed in unitsU
[U]
is the unit used for the time step. Values areh
for hours andm
for month. The latter is only valid for seasonal forecasts (mmsf).[type]
is once offc
(forecast),ef
(ensemble forecast),ep
(ensemble probabilities) ortf
(trajectory forecast for tropical cyclone tracks).[format]
isgrib2
for all fields, andbufr
for the trajectories.
The valid combinations of the above are:
format=bufr, type=tf HH=00/12 stream=enfo/oper, step=240h HH=06/18 stream=enfo, step=144h stream=scda, step=90h format=grib2 HH=00/12 stream=enfo/waef type=ef, step=0h to 144h by 3h, 144h to 360h by 6h type=ep, step=240h/360h stream=oper, wave type=fc, step=0h to 144h by 3h, 144h to 240h by 6h HH=06/18 stream=enfo/waef type=ef, step=0h to 144h by 3h stream= scda /scwv type=fc, step=0h to 90h by 3h HH=00 stream=mmsf, type=fc, u=m, step=1m to 7m
Accessing this dataset
Locations
Currently, these data are available from two different locations:
- ECMWF, with
ROOT
set to https://data.ecmwf.int/forecasts - Microsoft's Azure, with
ROOT
set to "https://ai4edataeuwest.blob.core.windows.net/ecmwf"
You can use the ECMWF link to browse the list of files.
Examples using wget for products based on the Atmospheric Model high-resolution (HRES) forecasts
In the following examples, [ROOT]
should be replaced by one of the locations specified above.
Although all of the examples provided here use wget, curl can similarly be used to download the products.
HRES direct model output
For direct model output from the Atmospheric model high-resolution (HRES) forecast time 00 and 12 UTC, stream=oper and type=fc should be used.
The steps available are 0h to 144h by 3h and 150h to 240h by 6h. The file format is grib2.
The example shows how to download the file containing all of the parameters available at step=24h from the 00UTC HRES forecast run on 25 January 2022:
wget [ROOT]/20220125/00z/0p4-beta/oper/20220125000000-24h-oper-fc.grib2
For direct model output from the Atmospheric model high-resolution (HRES) forecast time 06 and 18 UTC, stream=scda and type=fc should be used.
The steps available are 0h to 90h by 3h only. The file format is grib2.
The example shows how to download the file containing all of the parameters available at step=24h from the 06 UTC HRES forecast run on 25 January 2022:
wget [ROOT]/20220125/06z/0p4-beta/scda/20220125060000-24h-scda-fc.grib2
HRES Tropical Cyclone track products
Tropical cyclone tracks products are only available when there are tropical cyclones observed or forecast.
For the Tropical Cyclone track products from the Atmospheric model high-resolution (HRES) forecast time 00 and 12 UTC, stream=oper and type=tf should be used.
Products are provided only at step=240. The file format is bufr.
The example shows how to download the file containing all of the TC Track products from the 00 UTC HRES forecast run on 25 January 2022:
wget [ROOT]/20220125/00z/0p4-beta/oper/20220125000000-240h-oper-tf.bufr
For the Tropical Cyclone track products from the Atmospheric model high-resolution (HRES) forecast time 06 and 18 UTC, stream=scda and type=tf should be used.
Products are provided only at step=90. The file format is bufr.
The example shows how to download the file containing all of theTC Track products from the 18 UTC HRES forecast run on 25 January 2022:
wget [ROOT]/20220125/18z/0p4-beta/scda/20220125180000-90h-scda-tf.bufr
Examples using wget for products based on the Wave Model high-resolution (HRES-WAM) forecasts
HRES-WAM direct model output
For direct model output from the Wave model high-resolution (HRES-WAM) forecast at time time 00 and 12 UTC, stream=wave and type=fc should be used.
The steps available are 0h to 144h by 3h and 150h to 240h by 6h. The file format is grib2.
The example shows how to download the file containing all of the parameters available at step=06h from the 12 UTC HRES-WAM forecast run on 25 January 2022:
wget [ROOT]/20220125/12z/0p4-beta/wave/20220125120000-12h-wave-fc.grib2
For direct model output from the Wave model high-resolution (HRES-WAM) forecast at time time 06 and 18 UTC, stream=scwv and type=fc should be used.
The steps available are 0h to 90h by 3h only. The file format is grib2.
The example shows how to download the file containing all of the parameters available at step=18h from the 06 UTC HRES-WAM forecast run on 25 January 2022:
wget [ROOT]/20220125/06z/0p4-beta/scwv/20220125120000-18h-scwv-fc.grib2
Examples using wget for products based on the Atmospheric Model ensemble (ENS) forecasts
ENS direct model output products
For direct model output from the Atmospheric model ensemble (ENS) forecast at all times 00, 06, 12 and 18 UTC, stream=enfo and type=ef should be used.
- At time=00 and time=12, the steps available are 0h to 144h by 3h and 150h to 360h by 6h.
- At time=06 and time=18, the steps available are 0h to 144h by 3h only
The file format is grib2.
The example shows how to download the file containing all of the parameters available at step=24h for all 50+1 ensemble members from the 00UTC ENS forecast run on 25 January 2022:
wget [ROOT]/20220125/00z/0p4-beta/enfo/20220125000000-24h-enfo-ef.grib2
ENS Tropical Cyclone track products
Tropical cyclone tracks products are only available when there are tropical cyclones observed or forecast.
For the Tropical Cyclone track products from the Atmospheric model ensemble (ENS) forecast time 00 and 12 UTC, stream=enfo and type=tf should be used.
Products are provided only at step=240. The file format is bufr.
The example shows how to download the file containing all of the TC Track products from the 00 UTC HRES forecast run on 25 January 2022:
wget [ROOT]/20220125/00z/0p4-beta/enfo/20220125000000-240h-enfo-tf.bufr
For the Tropical Cyclone track products from the Atmospheric model ensemble (ENS) forecast time 06 and 18 UTC, stream=enfo and type=tf should be used.
Products are provided only at step=144. The file format is bufr.
The example shows how to download the file containing all of theTC Track products from the 06 UTC ENS forecast run on 25 January 2022:
wget [ROOT]/20220125/06z/0p4-beta/enfo/20220125060000-144h-enfo-tf.bufr
ENS Probability products
For the probability products from the Atmospheric model ensemble (ENS) forecast at all times 00, 06, 12 and 18 UTC, stream=enfo and type=ep should be used.
Products are available at time=00 and time=12 only.
There are no ENS probability products for time=06 or time=18.
The steps available are 0h to 144h by 3h and 150h to 360h by 6h. The file format is grib2.
The name of the file to be downloaded depends on the step. Use:
- step=240h in the file name for products at forecast steps 0h to 144h by 3h and 150h to 240h by 6h
- step=360h in the file name for products at forecast steps 246h to 360h by 6h
For probability products from the time=00 and time=12 ENS forecasts for for steps 0h to 144h by 3h and steps 150h to 240h by 6h, use step=240. The example is for the 00 UTC ENS forecast run on 25 January 2022:
wget [ROOT]/20220125/00z/0p4-beta/enfo/20220125000000-240h-enfo-ep.grib2
For probability products from the time=00 and time=12 ENS forecasts for steps 246h to 360h by 6h, use step=360. The example is for the 12 UTC ENS forecast run on 25 January 2022:
wget [ROOT]/20220125/12z/0p4-beta/enfo/20220125120000-360h-enfo-ep.grib2
Examples using wget for products based on the Wave Model ensemble (ENS-WAM) forecasts
For direct model output from the Wave model ensemble (ENS) forecast at all times 00, 06, 12 and 18 UTC, stream=waef and type=ef should be used.
- At time=00 and time=12, the steps available are 0h to 144h by 3h and 150h to 360h by 6h.
- At time=06 and time=18, the steps available are 0h to 144h by 3h only
The file format is grib2.
The example shows how to download the file containing all of the parameters available at step=24h for all 50+1 ensemble members from the 00UTC ENS-WAM forecast:
wget [ROOT]/20220125/00z/0p4-beta/waef/20220125000000-24h-waef-ef.grib2
ENS-WAM Probability products
For the probability products from the wave model ensemble (ENS) forecast at all times 00, 06, 12 and 18 UTC, stream=waef and type=ep should be used.
Products are available at time=00 and time=12 only.
There are no ENS-WAM probability products for time=06 or time=18.
The steps available are 0h to 144h by 3h and 150h to 360h by 6. The file format is grib2.
The name of the file to be downloaded depends on the step. Use:
- step=240h in the file name for products at forecast steps 0h to 144h by 3h and 150h to 240h by 6h
- step=360h in the file name for products at forecast steps 246h to 360h by 6
For probability products from the time=00 and time=12 ENS-WAM forecasts for for steps 0h to 144h by 3h and steps 150h to 240h by 6h, use step=240. The example is for the 00 UTC ENS-WAM forecast run on 25 January 2022:
wget [ROOT]/20220125/00z/0p4-beta/waef/20220125000000-240h-waef-ep.grib2
For probability products from the time=00 and time=12 ENS forecasts for steps 246h to 360h by 6h, use step=360. The example is for the 12 UTC ENS-WAMforecast run on 25 January 2022:
wget [ROOT]/20220125/12z/0p4-beta/waef/20220125120000-360h-waef-ep.grib2
Download a single field with wget
All of the examples above download a single file containing all of the parameters for that dataset at the specific forecast step or steps.
It is also possible to download of a single field using the HTTP Byte-Range request feature.
To download only the 2m temperature at step=24h from the 00 UTC HRES forecast on 25 January 2022, first download the associated index file by substituting the '.grib2'
extension with '.index'
in the URL:
wget [ROOT]/20220125/00z/0p4-beta/oper/20220125000000-24h-oper-fc.index
Inspect the index file and look for the entry for 2m temperature ('param' : '2t')
... {"domain": "g", "date": "20220125", "time": "0000", "expver": "0001", "class": "od", "type": "fc", "stream": "oper", "step": "24", "levtype": "sfc", "param": "2t", "_offset": 17459800, "_length": 609046} ...
Use the values of _offset
and _length
keys for this record to construct the start_bytes and end_bytes:
start_bytes = _offset = 17459800 end_bytes = _offset + _length - 1 = 17459800 + 609046 - 1 = 18068845
The "_offset" and "_length" values of a specific field will change from forecast run to forecast run. It is necessary to redo this computation for each download.
Use the start_bytes and end_bytes values calculated to pass the range of bytes to be downloaded to wget, this time for the .grib2 file:
wget [ROOT]/20220125/00z/0p4-beta/oper/20220125000000-24h-oper-fc.grib2 --header="Range: bytes=17459800-18068845"
Alternatively, curl can be used:
curl --range 17459800-18068845 [ROOT]/20220125/00z/0p4-beta/oper/20220125000000-24h-oper-fc.grib2 --output 2t.grib2
Multipart byte ranges of the form:
wget ... --header="Range: bytes=17459800-18068845, 18168021-1819654"
are not supported by all servers.
Index files
Each GRIB file is associated with a corresponding index file, accessible by substituting the '.grib2'
extension with '.index'
in the URL. Index files are text files where each line is a JSON record (JSON details here). Each record represents a GRIB field in the corresponding GRIB file, described using the MARS query language, for example:
{"domain": "g", "date": "20220104", "time": "1200", "expver": "0001", "class": "od", "type": "fc", "stream": "oper", "step": "6", "levelist": "1000", "levtype": "pl", "param": "q", "_offset": 3857250, "_length": 609046}
In addition, the keys _offset
and _length
represent the byte offset and length of the corresponding field. This allows the download of a single field using the HTTP Byte-Range request. An example is provided in Download a single field with wget above.
Differences between MARS language and file naming convention
There are some minor differences between the normal MARS request language and the open data file naming.
These are summarised in the table for information.
MARS | File names | ||||
Stream | Type | Step | Stream | Type | Step |
oper/wave | fc | nnn | oper/wave | fc | nnn |
enfo/waef | cf | nnn | enfo/waef | ef | nnn |
enfo/waef | pf | nnn | enfo/waef | ef | nnn |
enfo/waef | em | nnn | enfo/waef | ep | 240 if nnn <= 240 else 360 |
enfo/waef | es | nnn | enfo/waef | ep | 240 if nnn <= 240 else 360 |
enfo/waef | ep | nnn | enfo/waef | ep | 240 if nnn <= 240 else 360 |
msmm | fcmean/em | nnn | mmsf | fc | nnn |
Support level for this dataset
This data is provided as a 'self-serve' - documentation is provided, but no support service.
However, if the data seems broken or corrupt, please report it via our support portal and we will respond during normal office hours.
- No labels