...

Introduction

ERA5 is the fifth generation ECMWF reanalysis for the global climate and weather and currently covers the past 80 years. It covers the period from January 1940 to the present with hourly sampling and continues to be extended forward with 5 days behind real time.

This dataset The ERA5 Analysis Ready Cloud Optimised (ARCO) data on single levels presented here is a subset of some parameters of the full Climate Data Store (CDS) ERA5 single levels hourly dataset on 0.25 degrees resolution and is stored in Analysis Ready Cloud Optimised (ARCO) format, which has been implemented for retrieving long time-series for a single point in a efficient way. It is this in a ARCO format. It allows direct access to a subset of the surface and wave variables (see below) without downloading individual files, enabling efficient and scalable data access and retrieval. The ARCO format has a dual-chunking approach which means that data access can be optimised for retrieving long-time periods for a limited area (geo-chunked) or short-time period for large areas (time-chunked). The ARCO ERA5 geo-chunked is the source of ERA5 data that is used by the ERA-Explorer to ensure response times required for the interactive web-application and by the ERA5 hourly time-series data on single levels from 1940 to present catalogue entry to provide fast access to long time series ERA5 data.

Methodology

...

The process to generate this

...

ERA5 ARCO data involves:

• Fetching ERA5 data

...

• from in GRIB format.
• Apply homogenization conventions, as applicable
  • Rename spatial coordinates to latitude, longitude
  • Ensure latitude, longitude ranges are between [-90, +90], [-180, +180]
  • In the accumulation variables (e.g. Total precipitation), time is expressed in one single dimension called time which in ERA5 netCDFs is usually called valid_time.
  • In the accumulation variables, if the number of hourly samples is incomplete (less than 24) the date is skipped (e.g. 1940-01-01).
  • Drop "number", "step", "surface" and "valid_time".
• Written to an ARCO Zarr archive (which is similar in structure to netCDF)

...

Data organization and access

The ARCO data is stored as Zarr datacubes to provide efficient access to time-chunked and geo-chunked ERA5 single level hourly data:

The variables available in the Single levels atmospheric and Wave Zarr files are listed in Table 1 and Table 2, respectively.

The Zarr datacubes can be accessed using your CDS API key. Below is a simple plug-and-play code snippet and a more detailed Jupyter Notebook example. For more detailed and advanced access examples, please refer to the main Analysis Ready Cloud Optimised (ARCO) Data.

import xarray as xr

# Geo-chunked data for access optimised along the time dimension (e.g. for time-series at a single point)
# Surface
surface_geo_url = "https://arco.datastores.ecmwf.int/cadl-arco-geo-002/arco/reanalysis_era5_single_levels/sfc/geoChunked.zarr"
# Wave
wave_geo_url = "https://arco.datastores.ecmwf.int/cadl-arco-geo-003/arco/reanalysis_era5_single_levels/wav/geoChunked.zarr"

# Time-chunked data for access optimised along the time dimension (e.g. for time-series at a single point)
# Surface
surface_time_url = "https://arco.datastores.ecmwf.int/cadl-arco-time-002/arco/reanalysis_era5_single_levels/sfc/timeChunked.zarr"
# Wave
wave_time_url = "https://arco.datastores.ecmwf.int/cadl-arco-time-003/arco/reanalysis_era5_single_levels/wav/timeChunked.zarr"

# Open one of the Zarr objects with xarray, the default example opens the geo-chunked surface variables
ds = xr.open_zarr(
    surface_geo_url,
    consolidated=True,
    storage_options={
        "headers": {"Authorization": f"Bearer <CDS-API-KEY>"}
    }
)

# Inspect the variables
print(ds)

Time-series access via the CDS

Data organization and access

The ARCO data is stored as Zarr datacubes.

The data is now available from the Climate Data Store (CDS), either interactively through its download web form or programmatically using the CDS API service:

The

ERA5 hourly time-series data on single levels from 1940 to present

Spatial grid

The atmospheric parameters have a grid of 0.25 degrees as the wave parameters have a grid of 0.5 degrees. catalogue entry allows users to download long time series for a single-point or limited area via the CDS download webform or a CDS API request that users can build from the webform. The data download via the time-series entry is as described below with additional aspects:

1. Note
   Please be aware when you select a "Location" on the form, the latitude and longitude values are rounded to the closest neighborhood point of the 0.25 degrees grid

...

Temporal frequency

Temporal frequency of the data is hourly and every day has 24 hourly steps.

Data format

In the CDS, there are the options of retrieving the data in netCDF format or CSV format.

...

1. . Similarly, selecting an “Area” returns all points within the region defined by the nearest neighborhood corner points to the user-specified S, W, N, and E latitude/longitude values (maximum 5° x 5°).

2. The download form allows users to select a choice of netCDF or CSV. The conversion is done directly with xarray with no additional processing from the Zarr description above.
   • Given the computational cost of writing to csv, users are limited to smaller downloads with this option.
3. The surface and single level parameters (Table 1) and the wave parameters (Table 2) are

...

1. in different

...

1. Zarr stores, therefore they will be served in different output files.

Spatial grid

The atmospheric parameters have a grid of 0.25 degrees as the wave parameters have a grid of 0.5 degrees.

Temporal frequency

Temporal frequency of the data is hourly and every day has 24 hourly steps in different output files to avoid any conflict in coordinate definition.

Data update frequency

ERA5 is updated on a daily basis. New daily data in ERA5 becomes available in ERA5 Time Series in a few hours.

Parameters listings 

Table 1 below lists the surface and single level parameters (levtype=sfc) and Table 2 lists the wave parameters.

Anchor
table1 table1

Table 1: surface and single level parameters (instantaneous and accumulations)

Anchor
table2 table2

Table 2: wave parameters: instantaneous

Known issues

Please refer to the ERA5 documentation.

References

Further ERA5 references are available from the ECMWF website.

Related articles