...

Expand
title Click here to expand the list of datasets covered by this document
Deliverable ID Product title Product type (CDR, ICDR) C3S Version Number Public Version Number Delivery date Climate and energy related variables from the Pan-European Climate Database derived from reanalysis and climate projections CDR v1.0 v1.0 22/08/2024

...

Acronyms and abbreviations

Expand
title Click here to expand the list of acronyms and abbreviations
Acronym/abbreviation Definition AM Annual Maxima AOI Angle of Of Incidence API Application Programming Interface AR6 Sixth Assessment Report ASCII American Standard Code for Information Interchange BHI Beam Horizontal Irradiance BIAS Data that have been bias-adjusted C3S Copernicus Climate Change Service CDFt Cumulative Distribution Function transfer CDO Climate Data Operators CDS Climate Data Store CMIP6 Coupled Model Intercomparison Project (sixth phase) CMR5 CMCC-CM2-SR5 CSP Concentrated Solar Power DHI Diffuse Horizontal Irradiance DMP Data Management Plan DNI Direct Normal Irradiance DTU Technical University of Denmark ECE3 EC-Earth3 ENTSO-E European Network of Transmission System Operators for Electricity ERAA European Resource Adequacy Assessment ESFG Earth System Grid Federation ESRI Environmental Systems Research Institute GCM Global Climate Model GHI Surface solar radiation downwards GMC General Climate Model GPU Generation Per Unit GTI Global Tilted Irradiance HOL Hydropower open-loop pumped storage inflow energy HP Hydro Power HPI Hydropower run-of-river with pondage inflow energy HPO Hydropower run-of-river with pondage generation energy HPS Hydro Pumped Storage HRG Hydropower reservoirs generation energy HRI Hydropower reservoirs inflow energy HRO Hydropower run-of-river generation energy HRR Hydropower run-of-river inflow energy HWS High Wind Speed IC Installed Capacity IPCC Intergovernmental Panel on Climate Change LOYO Leave-One-Year-Out MAE Mean Absolute Error MEHR MPI-ESM1-2-HR NDA Non-Disclosure Agreement nMAD normalized Mean Absolute Deviation nMBD normalized Mean Bias Deviation NNSE Normalized Nash-Sutcliffe Efficiency NSE Nash-Sutcliffe Efficiency NUT0 Country level of aggregation NUT2 Sub Country/Provinces level of aggregation ORIG Data that have not been bias-adjusted PECD Pan-European Climate Database PEOF Pan-European Bidding Zones Offshore level of aggregation PEON Pan-European Bidding Zones Onshore level of aggregation POA Plane of Of Array PV Photo Voltaic QGIS Quantum Geographic Information System RF Random Forest SEDAC Socioeconomic Data and Applications Center SFOE Swiss Federal Office of Energy SPV Solar Photovoltaic SSPs Shared Socio-economic Pathways SZA Solar Zenith Angle SZOF Pan-European Zones Offshore level of aggregation SZON Pan-European Zones Onshore level of aggregation TA 2m temperature TAW Population-weighted temperature TOA Top Of the Atmosphere TP Total precipitation TSO Transmission System Operator UTC Coordinated Universal Time VM Virtual Machine WMO World Meteorological Organization WOF Wind power offshore WON Wind power onshore WPP Wind Power Plant WS10 10m wind speed WS100 100m wind speed

Introduction

This document describes the technical methodologies and implementation of the climate and energy indicators underpinning the Pan-European Climate Database (PECD), co-developed within the Copernicus Climate Change Service (C3S) Energy service in close collaboration with the European Network of Transmission System Operators for Electricity (ENTSO-E).

Technical documentation of the workflows and their modules is provided below. The workflows are structured according to the two streams covered by PECD: historical and projections. Each routine is described in terms of its inputs, outputs, and processing steps. There is no operational chain in place for near real-time updates of the datasets. However, regular updates of historical data will be performed annually.A detailed description of the filenames of the provided data is available in the Appendix

Note
In the PECDv4.1, only three Global Climate Models (GCMs) were used. However, it is important to note that using a larger set of models is essential for adequately capturing the uncertainty inherent in climate projections. Future versions, such as PECDv4.2, will incorporate a wider range of models and scenarios to improve the representation of uncertainties.

Files will be provided in two format types: NetCDF and CSV. Please refer to Table 2.2, Table 2.13, Table 3.5 and Table 3.7 for more info on the file format of each variable.

A detailed description of the filenames of the provided data is available in the Appendix in Table 4.1, while Table 4.2 and Table 4.3 contain the description of all the ancillary NetCDF data used for PECDv4.1 and available in the Climate Data Store (CDS) under the widget "Weights and masks".

Note

Please note that PECDv4.1 data will not be extended beyond the year 2021, as these datasets have been frozen prior to the start of the ERAA (European Resource Adequacy Assessment) studies 2023, in agreement with ENTSO-E. By the end of 2024, a new version, PECDv4.2, will be delivered, containing historical data from 1950 to the near present. The historical data for PECDv4.2 will be updated annually. These updates will add new data without modifying the existing datasets, thus maintaining the same version number. The plan agreed with ENTSO-E is to have PECDv4.2 available in 2025. Future versions will include more climate models, emission scenarios, extended time series, and changes in methodologies, aggregation zones, and other aspects according to ENTSO-E requirements.

...

The workflow depicting the historical stream is outlined in Figure 1.1. The retrieval of ERA5 data from the Climate Data Store ( CDS ) to C3S is accomplished using the CDS API (Application Programming Interface), which requires prior installation of Python and the CDS API Python package. Data is retrieved by specifying the required period and variables to be downloaded. Currently, retrievals are performed in monthly chunks. Each variable has been downloaded at a 1-hour resolution for the period 1980 to 2021, within the designated study region known as the PECD domain.

...

 [m] correspond to these two respective heights. The wind shear coefficient is computed at an hourly resolution and stratified by the twelve months of the year (24 hours x 12 months) and it was calculated using Python's Climate Data Operators (CDO) commands.

Alpha characterization

The Alpha coefficients obtained with the procedure described above are then saved into a NetCDF file, which is available for download on the CDS (please refer to Table 4.2 and Table 4.3 for more details).

Alpha characterization

The mean value of the Alpha coefficient calculated over the geographical domain for mean value of the Alpha coefficient calculated over the geographical domain for each hour and month is represented in Figure 2.1. These results are consistent with previous studies, showing higher coefficient values during the cold and stable hours of the night. Conversely, during the day, when the boundary layer is generally well mixed, the Alpha coefficient is lower. For the same reason, the values of the Alpha coefficient are higher in winter compared to summer during the central (and warmer) hours of the day. However, when examining the distribution of the Alpha coefficient across each grid point and month of the year, a more complex picture emerges. Figure 2.2 shows the box plot for each hour over the entire domain, indicating that the interquartile range is broader for the night-time hours compared to the day-time hours. The Alpha coefficient reaches its most negative values, down to -0.4, during the night-time hours.

...

The adopted bias adjustment procedure, applied only to WS10, comprises three steps detailed below (see also Figure 2.9 and 2.10). Once the WS10 is bias-adjusted with this procedure, the WS100 is extrapolated by applying the alpha coefficients (Section 2.2).:

1) Pre-processing: The ORIG 

...

). Raster values represent the number of inhabitants per cell, with sea/ocean pixels assigned to no data values according to the ESRI (Environmental Systems Research Institute) ASCII format. The path to NetCDF file of the population density mask is ‘/data/public/PECD/ANCI/POPM’obtained with the procedure described above is available for download on the CDS (please refer to Table 4.2 and Table 4.3 for more details).

Anchor
Figure2_11 Figure2_11

...

Figure 2.11: Population map from NASA Socioeconomic Data and Applications Center. The map is reported at 0.25° resolution and represents the number of inhabitants within each cell.

Computation of Population-weighted temperature

Anchor
Section2_4_2 Section2_4_2

Population-weighted temperature TAW [°C] is computed by combining the population raster density mask at 0.25° resolution (see Table 4.2 and Table 4.3 for more details), which reports the inhabitants per cell, with the gridded temperature TA at the same resolution and over the same domain. TAW_z of a zone z is calculated according to the equation:

...

Figure 2.12: TA (top) and TAW (bottom) averaged 1980 to 2021 over Bidding Zones bidding zones (SZON).

Spatial aggregation

Anchor
Section2_5 Section2_5

...

Table 2.1: Required spatial aggregation for PECDv4.1.

...

*These shapefiles are not publicly available but the correspondent NetCDF masks are provided in the CDS under the widget "Weights and masks". Please see Table 4.2 and Table 4.3 for more details.

Anchor
Figure2_13 Figure2_13

 

Image AddedImage Added

...

Image RemovedImage Removed

Image ModifiedImage Modified

Figure 2.13: Examples of the original polygons used to derive the float masks.

...

Starting from the shapefiles listed in Table 2.1, floating point NetCDF masks were built to be used for data aggregation purposes, one for each level of aggregation: NUTS 0 regions mask, NUTS 2 regions mask, PEOF regions mask, PEON regions mask, SZOF regions mask, SZON regions mask.

The procedure requires different steps:

...

Figure 2.14 shows an example of a country mask (Italy).

The NetCDF masks for the different levels of aggregation are available for download in the CDS under the widget "Weights and masks". Please refer to Table 4.2 and Table 4.3 for more info on these files, including their filenames' conventions and characteristics.

Anchor
Figure2_14 Figure2_14

...

Figure 2.14: Example of a float mask, for the Italian NUT0 administrative region, showing the fractions of land around the border and coastlines.

The NetCDF mask file will be structured as follows:

coordinates: latitude (PECD domain), longitude (PECD domain), mask (mask code of each polygon).

Spatial aggregation procedure

...

1. Open the NetCDF file containing the data to be aggregated.
2. Open the precalculated mask NetCDF file.
3. Iterate over mask coordinates.
4. For each mask, apply the mask (product of data array) required region mask to the NetCDF to be aggregated , weighted by the cosine of latitude.(one among SZON, SZOF, PEON, PEOF, NUT0, NUT2 regions masks), weighted by the cosine of latitude (the latitude weights mask). Please refer to Table 4.2 and Table 4.3 for more info on these files, including their filenames' conventions and characteristics.
5. Calculate the average over the masked NetCDF.
6. Store the result in a column of a data frame.
7. Store the time axis of the NetCDF file in the same data frame of the aggregated result.
8. Save the dataframe as csv file. 
9. Apply metadata to the CSV file according to the annex. 

...

Combinations of restricted areas were also considered for wind generation and PV modeling. The exclusion area masks/files have been created based on these criteria to accurately represent regions where energy production is not feasible or allowed. For the distance to shore areas, the exclusion layer was generated using the QGIS Buffer tool to create a distance buffer of 100 km in both directions from the continental edge, except for the North Sea, where a buffer of 200 nautical miles has been retained as per ENTSO-E’s specifications. The NetCDF files of the masks that have been used for PECDv4.1 are available for download in the CDS under the widget "Weights and masks" ("Wind power regions mask" and "Solar PV mask"). Please refer to Table 4.2 and Table 4.3 for more info on these files, including their filenames' convention and characteristics.

Table 2.3 provides a detailed description of each exclusion criterion, their sources, and the variables associated with them.

...

For future onshore wind installations, turbines with specific powers ranging from 198 to 335 W/m², as shown in Swisher et al. (2022), are used. For future offshore wind installations, turbines with specific powers of 316 and 370 W/m² are used. The selected specific powers are the same as those used in the PECD 2021 update. An overview of the simulated future wind technologies is given in Table 2.5 and Table 2.6. Wakes are considered for all future technologies (Swisher et al., 2022). The specific power and hub height are the main drivers for variation in the resulting generation time series; the rotor diameter and rated power have limited impact (as a result are given , which also lists the corresponding options found in the widget "Technological specification" in the download form. Each wind technology option is labeled with a number representing a specific combination of hub height (HH) and specific power (SP). For example, "21 (SP316 HH155)" refers to offshore wind power with a specific power of 316 W/m² and a hub height of 155 m. These labels allow users to easily select the desired wind turbine specification from the dataset.

Wakes are considered for all future technologies (Swisher et al., 2022). The specific power and hub height are the main drivers for variation in the resulting generation time series; the rotor diameter and rated power have limited impact (as a result are given in standardized generation, i.e., in values between 0 and 1). Compared to the previous version of the work, the onshore wind turbine-rated power is increased to 5 MW (from 3.6 MW), based on feedback from ENTSO-E.

The more generic model (to run any combination of specific power and hub height), as presented in the previous section is also made available. This enables plant-level power curves to be estimated for any combination of specific power, hub height, and plant size (within the supported range shown in Table 2.4).

...

Table 2.5: Future technology onshore wind turbines.

Anchor
Table2_6 Table2_6

Table 2.6: Future technology offshore wind turbines.

The storm shutdown behavior is modeled as described in Murcia et al. (2021), assuming a direct (non-controlled) shutdown for all existing wind power plants (WPPs), using data from the WindPowerNet WPP installation database for the shutdown wind speeds. For future wind technologies, a 25 m/s cut-off is assumed for onshore wind installations, and the HWS (High Wind Speed) Deep type from Murcia et al. (2021) is used for future offshore wind installations (as in the PECD 2021 update). The shutdown procedure is modeled as a 'hysteresis,' where a restart occurs only after the wind speed has dropped to a sufficiently low value for a restart to take place (see Figure 2.16). The storm shutdown is a dynamic model that captures three aspects:

...

1. all wake modelling considers only intra-farm wakes (wakes between plants are not considered).
2. Literature suggests a range of 5 % to 10 % for the other losses (Mortensen, 2018). The existing installations cover historical installations over tens of years with older technology, whereas the future installations are new installations (no wear-and-tear considered) with modern technology: it was thus considered fair to place them at the opposite sides of the loss range.
3. A suitable mask is used is used to find the potential points for the Future wind technologies runs. This mask ("wind power regions mask") is available for download in the CDS.  Please refer to Table 4.2 and Table 4.3 for more info.
4. Locations of existing wind power plants are not considered in the assessment of the 10-50 % best locations for each region. This is done because the decommissioning of old turbines is expected to free up more space for new installations in the future.
5. The assumed locations of wind power plant installations within a region significantly impact the expected capacity factor on the aggregate level (Swisher et al., 2022). At this point, only one ‘resource grade’ (i.e., the 10-50 % best locations) is simulated; however, simulations covering also the 10 % best locations and the 50 % worst locations (or in principle any other distribution split between 0 and 100 %) could be provided in a later version of the PECD in consultation with ENTSO-E. However, this would multiply the amount of Future wind technology time series.

...

For future wind installations, the starting point is the ERA5 grid points. Masking, based on the Exclusion layers exclusion layers ("wind power regions mask", please refer to Table 4.2 and Table 4.3 for more info) presented in Section 2.9, is then applied to these points to select potential future WPP locations. The potential points are shown in Figure 2.21. After selecting the 10-50% best points (based on 100 m mean wind speeds), the resulting final future installation simulation points can be seen for onshore and offshore wind in Figure 2.22. The selection of 10-50% best points is the average ‘resource grade’ selection following from the work done by Swisher et al. (2022), where also the best 10 % and worst 50 % selection of points were simulated for each region. Similar additional runs can be performed at a later stage in the project in agreement with ENTSO-E, to model the decrease in capacity factor as more and more of the best wind resource locations are used. However, this would multiply the number of wind time series related to future installations.

...

Once the PV capacity factor product is generated for the PECD-constrained ERA5 grid, regional estimates for bidding and study PECD zones are calculated through a spatial average. However, it is important to note that particular (restricted) areas were masked in both the grid-like and regional-based products to produce more accurate results. Specifically, sea and ocean areas (thus, off-shore PV), polar and protected areas, as well as locations with high elevation (above 2000 m a.s.l.) or slope (higher than 10%) were excluded from the computation. While high elevation may be unsuitable as an exclusion criterion at a global scale (notably for Chile), we found that for the PECD area this does not pose issues in terms of final PV estimates. The information to identify such regions was obtained from a range of sources (see Section 2.9). The mask used for PECDv4.1 ("Solar PV mask") is available for download in the CDS under the widget "Weights and masks" (please refer to Table 4.2 and Table 4.3 for more info).

Improvements over the previous methodology

...

1. CSP plants are simulated without energy storage.
   
2. CSP plants with 7h of thermal energy storage.

...

In the widget "Technological specification" in the download form, each CSP option is represented by a number corresponding to whether the plant includes energy storage and whether the energy is considered before or after dispatch. The available options are the following:

• 40 (Pre-dispatch, no storage): indicates potential energy generation before storage, with no storage capacity;
• 41 (Dispatched, no storage): energy actually dispatched from a plant with no storage capacity;
• 42 (Pre-dispatch, 7-hours of storage): potential energy generation from a plant with 7 hours of storage, before dispatch;
• 43 (Dispatched, 7-hours of storage): energy actually dispatched from a plant with 7 hours of storage.

Anchor
Figure2_30 Figure2_30

Image Added

Figure 2.30: Overview of CSP behavior when thermal storage is available. The power from the solar

...

Image Removed

Figure 2.30: Overview of CSP behavior when thermal storage is available. The power from the solar field (dashed green line) is higher than the installed capacity (1.0) and is thus stored. The orange line shows dispatch from the CSP plant.

...

Low scores are mainly due to few years of available data for the training (e.g. 3 years), or to irregularities in the time series of generation which reduce its seasonality. This can be caused by artificial regulations or faulty data records. In some cases, low scores were obtained due to a loss of seasonality in the time series brought by significant changes in the Installed Capacity for that country / bidding zone. The new installations can cause abrupt changes or gradual shifts in the mean observed signal. Since the model is based solely on climate data, it cannot predict this behaviour. A possible solution that’s been attempted is to model directly Capacity Factors (CF), hence normalizing the provided generation data by the annual series of country-aggregated Installed Capacities (IC). This improves the results for some countries, but generally worsens them for countries where the IC doesn’t change significantly with time. This means that generation may not reflect the actual IC at one time. Changes to the IC can occur at the beginning of the reporting year or at any time during the year, therefore likely introducing step changes in the IC. However, a data collection was launched by ENTSO-E to retrieve monthly Installed Capacity time series from the TSOs and some were able to provide them. Therefore, where new installations visibly affected the TSO generation time series, these were normalized with the corresponding monthly IC data provided, the model was trained on the normalized time series, and the output was then multiplied back by the same IC series to re-obtain a generation/inflow time series. This procedure was applied to timeseries of Albania, Switzerland, Hungary, Poland, and Portugal and must be taken into account when comparing projection energy values to historical ones, since in these cases the anomalies are not only due to changes in climate variables, but also to the known changes in IC. It is also important to note that the assumption made for this procedure is that TSO generation and TSO monthly installed capacity series provided for these countries were compatible. Therefore, any inconsistency that may be found between model outputs and expected historical values may come from discrepancies between generation and installed capacity initial input data.

Other time series displayed irregularities arguably attributable to changes in IC but were not provided with monthly IC series. In such cases, the RF model was trained on a recent restricted time window (at least 4 years) of close-to-constant IC. The latter is hence assumed unvaried in time.

Anchor
Figure2_32 Figure2_32

...

Anchor
Table3_6 Table3_6

Table 3.6: Wind run types for projection stream.

Photovoltaic Solar Power conversion model

The climate data used as input are listed in Table 3.4, and the procedure is the same as described in Section 2.9.2. 

...

The energy indicators are the same as described in Section 2.10 for the historical stream computed starting from the climate indicators listed in Table 3.5.

In Table 3.7, the energy variables contained in this database are summarized. Table 3.7 provides detailed information for each variable, including the type, the time period covered, the source of the input data, the domain, the temporal resolution, the spatial aggregation (according to Table 2.1), and, where applicable, the different technologies used to compute the final time series.

Anchor
Table3_7 Table3_7

Table 3.7: Energy indicators provided in the PECDv4.1 for the projection stream. Files provided at ORIG spatial aggregation are gridded (NetCDF format), while all the other levels of aggregation are provided in CSV format.

*Energy data from ENTSO-E Transparency Platform

**Energy data from Transmission System Operators specific for each country

...

Appendix

Anchor
Appendix Appendix

Filenames convention and characteristics

This paragraph aims to explain the filename convention of the PECD datasets. Table 4.1 details the structure and possible fields of the filenames. Specifically, the last column indicates the corresponding section of the CDS catalogue where users can personalize their choice. If "Not applicable" is indicated, it means that the user cannot modify this field, and the data are downloaded with fixed characteristics that are not customizable. Table 4.2 details the structure and filenames of the ancillary NetCDF files that have been used for PECDv4.1 and that are available in the CDS under the widget "Weights and masks".

Anchor
Table4_1 Table4_1

Table 4.1: Filename convention used in the PECDv4.1.

Example of filename: H_ERA5_ECMW_T639_TP-_0000m_Pecd_025d_S198501010000_E198501310000_ACC_MAP_01d_NA-_noc_org_NA_NA---_NA---_NA—_

...

PECD4.1_fv1.nc

This NetCDF file (.nc) contains historical data (H) from ERA5 reanalysis (ERA5 and 7639) originated by ECMWF (ECMW); the variable is total precipitation (TP-) at 0m height (0000m), the coverage is PECD domain (Pecd) with a 0.25° spatial resolution (025d). Data span from 01/01/1985 at 00:00 UTC (S198501010000) to 31/01/1985 at 00:00 UTC (E198501310000). The data are accumulated (ACC), gridded (MAP), with a daily temporal resolution (01d). The lead time is not available (NA-), data are not bias-corrected (noc) and they are original (org), gridded (MAP), with a daily temporal resolution (01d). The lead time is not available (NA-), data are not bias-corrected (noc) and they are original (org). The ensemble number, emission scenario, energy scenario and transfer function are not available (NA_NA---_NA---_NA---). The version number is v4.1.. The ensemble number, emission scenario, energy scenario and transfer function are not available (NA_NA---_NA---_NA---). The PECD version is 4.1 (PECD4.1) while the file version is fv1.

Anchor
Table4_2 Table4_2

Table 4.2: Filename convention for ancillary data used in the PECDv4.1 and that are available in the CDS under the widget "Weights and masks".

Example of filename for the ancillary data: ANCI_LAT-mask_PECD4.1_fv1.nc

This NetCDF file (.nc) contains ancillary data (ANCI) used to adjust the gridded data with the proper latitudinal weights (LAT-mask) during the spatial aggregation procedure. The PECD version is 4.1 (PECD4.1) and the file version is fv1.

Anchor
Table4_3 Table4_3

Table 4.3: Description of the ancillary NetCDF data and their characteristics. These files are available for download in the CDS under the widget "Weights and masks".

Metadata

The header of the time series CSV files will contain the following metadata descriptors. An example of an air temperature variable is presented below, provided as a CSV file with the filename:

...

### The original data sources are ECMWF ERA5 Reanalysis (available at: https://cds.climate.copernicus.eu) # 

How to cite the data*

Info
*If the dataset is to be published in the CDS or ADS, then this may not be needed. Please check with the CUS team.

...