Last modified on Jun 20, 2023 12:55

Contributors: G.E. Thomas (UKRI-STFC RAL Space)

Issued by: STFC RAL Space (UKRI-STFC) / Gareth Thomas

Date: 26/05/2023

Ref: C3S2_D312a_Lot1.2.1.4-v4.0_202305_PUGS_CCICloudProperties_v1.2

Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1

History of modifications

List of datasets covered by this document

Related documents

Acronyms

List of tables

List of figures

General definitions

Table 1: Summary of variables and definitions

Table 2: Definition of processing levels

Table 3: Definition of various technical terms used in the document

Scope of the document

This Product User Guide and Specification (PUGS) document is associated with the Climate Data Store (CDS) catalogue entry: Cloud properties global gridded monthly and daily data from 1982 to present derived from satellite observations. This Climate Data Record (CDR) comprises inputs from two sources: (i) brokered products from the Cloud Climate Change Initiative (ESA’s Cloud_cci), namely those coming from processing of the Advanced Along-Track Scanning Radiometer (A)ATSR) data and (ii) those produced under this contract for the CDS, specifically those coming from processing of the Sea and Land Surface Temperature Radiometers (SLSTR). Both CDRs are produced by Rutherford Appleton Laboratory (RAL) Space using the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm, using the Community Cloud for Climate (CC4CL) processor under the ESA Cloud_cci project. 

This document provides information on how to use these products and it is not part of the official CCI documentation, but produced only in the scope of the brokering to the CDS.

Executive summary

The ESA Climate Change Initiative (CCI) Cloud Properties Climate Data Record (CDR) is a brokered product from the ESA Cloud_cci project, while the extension Interim CDR (ICDR) produced from the Sea and Land Surface Temperature Radiometers (SLSTR) is produced specifically for C3S. The product is generated by RAL Space, using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm.

The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) onboard the ESA second European Research Satellite (ERS-2) and ENVISAT. This CDR is partnered with the ICDR produced from the Sentinel-3 Sea and Land Surface Temperature Radiometers (SLSTR), beginning in 2017 (Sentinel-3A) and 2018 (Sentinel-3B), when data from each platform became available.

The CDR provides level-3 data (monthly means) on a regular global latitude-longitude grid (with a resolution of 0.5°× 0.5°) as well as level-2 data (daily means) on a regular global latitude-longitude grid (with a resolution of 0.1°× 0.1°) and includes these products: cloud fractional cover (CFC), cloud phase (water/ice), cloud optical thickness (COT), cloud particle effective radius (CER), cloud liquid/ice water path (LWP/IWP), and cloud-top pressure (CTP), height (CTH) and temperature (CTT).

These products are brokered to (in the case of (A)ATSR) or produced for the Climate Data Store (in the case of SLSTR) by the Copernicus Climate Change Services (C3S). The respective data products (daily and monthly means) are first described in terms of their input data and a brief overview of the algorithms; their target requirements in the scope of C3S and achieved performances are given; relevant information for usage is provided. The latter comprises geographical grid specifications, the data format, naming conventions, and the acknowledgement policy.

1. Cloud_cci global monthly cloud products

The Cloud_cci global monthly cloud properties product, version 3.0, is brokered to the CDS by the C3S from STFC RAL Space. The SLSTR ICDR, versions 3.1.1 and 4.0, is supplied to the CDS via the same route and uses the same processing software and infrastructure as the CDR. Although the format of the brokered data products differs from the official CCI products, the intellectual property rights remain with the Cloud_cci team. In detail, the official daily and monthly files from Cloud_cci contain more than 180 variables, but the specified C3S format splits them into separate parts per ECV. That way, each file has a reduced data amount as well as an increased usability. The landing page for the official CCI products is https://climate.esa.int/en/projects/cloud/.

The product provides estimates of monthly mean global cloud properties over land and ocean from the Along Track Scanning Radiometer ((A)ATSR) series of satellite sensors, on a 0.5°×0.5° latitude-longitude grid from mid-1995 until early 2012. The SLSTR based ICDR extends the coverage, with a five-year gap, from 2017 onwards and is only available through the CDS. Version 3.x of the ICDR provides data from a single SLSTR (flying on board either Sentinel-3a or -3b), while version 4.0 combines data from both SLSTR instruments into a single combined product (starting with the launch of the Sentinel-3b in late 2018). The processing of version 3.1.1 and 4.0 are identical – the sole difference is whether the two instruments are separately or treated as a single data-stream.

1.1 Product description

The Cloud_cci product brokered to the CDS consists of cloud properties derived from the (A)ATSR instruments; specifically:

• ATSR-2, which flew on the second European Research Satellite (ERS-2) from June 1995 until January 2008. ERS-2 lost global coverage in June 2003, due to failure of the onboard data storage; Cloud_cci data do not extent beyond this date (thus providing approximately one-year of overlap with the succeeding AATSR product).
• Advanced-ATSR (AATSR), which flew on ENVISAT from May 2002 until the satellite’s failure in April 2012.

When referred to in terms of a continuous data record, these two instruments are commonly described as (A)ATSR. The preceding ATSR-1 instrument lacked the visible-NIR channels provided by (A)ATSR and thus isn’t included in the Cloud_cci product.

The Cloud_cci also produced cloud products from the Advanced Very High Resolution Radiometer (AVHRR) series of instruments and the Moderate Resolution Imaging Spectroradiometer (MODIS), using a common retrieval technique, but only the (A)ATSR products are brokered to the CDS.

The analysis algorithm applied in producing the Cloud_cci products is the Optimal Retrieval of Aerosol and Clouds (ORAC), while the overall data processing system used in the project is referred to as Community Cloud for Climate (CC4CL). The algorithm, processing chain and all input and auxiliary data are detailed in the C3S ATBD [D1], Cloud_cci ATBD [D2], Sus et al. (2018) and McGarragh et al. (2018).

The brokered CDR is extended, with a five-year gap, from 2017 with an ICDR derived from the SLSTR instruments, which fly onboard the Copernicus operational Sentinel-3 satellites. From late 2018, a second Sentinel-3 platform is available, which doubles the spatial coverage of SLSTR and allows for nearly complete global coverage twice a day. The SLSTR product is produced using the same processing chain as the (A)ATSR CDR.

1.2 Target requirements

The target requirements for cloud observations are laid-out in the Target Requirements and Gap Analysis Document (TRGAD) [D5], but are summarised here. These requirements originate from the WMO Global Climate Observing System (GCOS) initiative, which defines and lays down targets for the observation of essential climate variables (ECVs). It should be noted that GCOS defines ideal targets for Earth observation systems, which are often not attainable using existing or historical observing systems. Table 1-1 provides an overview of the GCOS requirements for cloud properties.

Table 1-1: GCOS targets for cloud properties ECVs. Note that the Cloud_cci doesn't meet the temporal requirements due to the nature of the satellite observations, but exceeds the spatial resolution and aims to achieve the uncertainty and stability requirements.

1.3 Data usage information

Cloud_cci and SLSTR ICDR products are provided in NetCDF (version 4), which are compliant with the conventions CF 1.8 and the NASA Global Change Master Directory (GCMD) Science Keywords vocabulary. Filenames follow the structure:

C3S-312bL1-L3C-MONTHLY-CLOUD-INSTORACPLATFORM{}YYYYMM{_}_fvVVV.nc,

where INST and PLATFORM refer to the instrument and platform from which data originates (either ATSR2 and ERS2, or AATSR and ENVISAT for the CDR, or SLSTR and Sentinel-3a or -3b for the ICDR), YYYYMM provides the year and month covered by the monthly mean product, and VVV denotes the product version (3.0 for the TCDR, 3.1 for the ICDR).

Data are provided as monthly means of cloud properties, as described in Table 1-2, on a regular latitude-longitude grid, with a spacing of 0.5° in both dimensions (thus grid centres lie at -89.75°, -89.25°, -88.75°, …, 89.75° in latitude and -179.75°, -179.25°, -178.75°, …, 179.75° in longitude).

The list of cloud properties, the variable names used within the NetCDF data files and their units are given in Table 1-2.

Table 1-2: Cloud properties included in Cloud_cci TCDR and SLSTR ICDR monthly products.

For lists and details of known issues and limitations of the data, the user is referred to the referenced ATBDs and papers in Section 1.1, as well as the TRGAD [D5]. In particular, it should be noted that the monthly mean is computed using data from sun-synchronous polar-orbiting satellites, with a relatively narrow swath. Thus, it cannot be considered a representation of the mean properties of clouds across the whole month for two reasons:

• The narrow swath of the (A)ATSR instruments means that it requires approximately three days of measurements to provide full global coverage.
• The sun-synchronous orbit used by ERS-2, ENVISAT and Sentinel-3, means that observations are made near two fixed local solar times¹; 10:00 and 22:00 for the descending and ascending halves of the ENVISAT and Sentinel-3 orbit, and 10:30 and 22:30 for ERS-2. Thus, the diurnal cycle is poorly sampled.

1.3.1 Uncertainties

The ORAC retrieval algorithm is an implementation of Optimal Estimation (Rodgers, 2000) and as such provides uncertainty estimates on all retrieved parameters, propagated from the uncertainty in the input satellite measurements (as well as any forward modelling error included in the uncertainty calculations). The propagation of the L2 uncertainty into L3 products, such as those brokered to the CDS, is not straight forward. Users should refer to the Cloud_cci Comprehensive Error Characterisation Report [D4].

Uncertainty and variability in the monthly mean cloud properties is expressed using three separate values, as described by Bennartz (2017):

1. Standard deviation about the mean. This value is the unweighted standard deviation of the L2 retrieval values about the mean value reported for each L3 pixel.

2. Propagated uncertainty in value. The value represents the L2 uncertainty propagated to the L3 pixel under the assumption that the L2 values used in the L3 average are independent and uncorrelated. It is calculated by summing the L2 uncertainties in quadrature and normalizing by the number of observations:

where σ_i are the uncertainties of the N L2 pixels included in the average, and the <> indicates the mean value.

3. Correlated uncertainty in value. If a L2 pixels are assumed to be correlated with each other, the uncertainty in the resulting average can be calculated using: 

where σ²_std is the unweighted standard deviation about the mean and c is the correlation between the L2 pixels.

2. Cloud_cci global daily cloud products

The Cloud_cci global daily cloud properties product, version 3.0, is brokered to the CDS by the C3S from STFC RAL Space. The SLSTR ICDR, versions 3.1.1 and 4.0 are supplied to the CDS via the same route and uses the same processing software and infrastructure as the CDR. Although the format of the brokered data products differs from the official Cloud_cci products, the intellectual property rights remain with the Cloud_cci team. In detail, the official daily and monthly files from Cloud_cci contain more than 180 variables, but the specified C3S format splits them into separated parts per ECV. That way, each file has a reduced data amount as well as an increased usability. The landing page for the official CCI products is https://climate.esa.int/en/projects/cloud/.

The product provides estimates of daily mean global cloud properties over land and ocean from the Along Track Scanning Radiometer ((A)ATSR) series of satellite sensors, on a 0.1°×0.1° latitude-longitude grid from mid-1995 until early 2012. The SLSTR based ICDR extends the coverage, with a five-year gap, from 2017 onwards and is only available through the CDS. Version 3.x of the ICDR provides data from a single SLSTR (flying on board either Sentinel-3a or -3b), while version 4.0 combines data from both SLSTR instruments into a single combined product (starting with the launch of the Sentinel-3b in late 2018). The processing of version 3.x and 4.0 are identical – the sole difference is whether the two instruments are separately or treated as a single data-stream.

2.1 Product description

The Cloud_cci product brokered to the CDS consists of cloud properties derived from the (A)ATSR instruments; specifically:

• ATSR-2, which flew on the second European Research Satellite (ERS-2) from June 1995 until January 2008. ERS-2 lost global coverage in June 2003, due to failure of the onboard data storage; Cloud_cci data do not extent beyond this date (thus providing approximately one-year of overlap with the succeeding AATSR product.
• Advanced-ATSR (AATSR), which flew on ENVISAT from May 2002 until the satellite’s failure in April 2012.

When referred to in terms of a continuous data record, these two instruments are commonly described as (A)ATSR. The preceding ATSR-1 instrument lacked the visible-NIR channels provided by (A)ATSR and thus isn’t included in the Cloud_cci product.

The Cloud_cci also produced cloud properties from the Advanced Very High Resolution Radiometer (AVHRR) series of instruments and the Moderate Resolution Imaging Spectroradiometer (MODIS), using a common retrieval technique, but only the (A)ATSR products are brokered to the CDS.

The analysis algorithm applied in producing the Cloud_cci products is the Optimal Retrieval of Aerosol and Clouds (ORAC), while the overall data processing system used in the project is referred to as Community Cloud for Climate (CC4CL), and these are detailed in the C3S ATBD [D1], Cloud_cci ATBD [D2], Sus et al. (2018) and McGarragh et al. (2018).

The brokered CDR is extended, with a five-year gap, from 2017 with an ICDR derived from the SLSTR instruments, which fly onboard the Copernicus operational Sentinel-3 satellites. From late 2018, a second Sentinel-3 platform is available, which doubles the spatial coverage of SLSTR and allows for nearly complete global coverage twice a day. The SLSTR product is produced using the same processing chain as the (A)ATSR CDR.

2.2 Target requirements

The target requirements for cloud observations are laid-out in the Target Requirements and Gap Analysis Document (TRGAD) [D5], but are summarised here. These requirements originate from the WMO Global Climate Observing System (GCOS) initiative, which defines and lays down targets for the observation of essential climate variables (ECVs). It should be noted that GCOS requirements are targets and are often not attainable using existing or historical observing systems. Table 2-1 provides an overview of the GCOS requirements for cloud properties.

Table 2-1: GCOS targets for cloud ECVs. Note that the Cloud_cci doesn't meet the temporal requirements due to the nature of the satellite observations, but exceeds the spatial resolution and aims to achieve the uncertainty and stability requirements.

2.3 Data usage information

Cloud_cci products are provided in NetCDF (version 4), which are compliant with the conventions CF 1.8 and the NASA Global Change Master Directory (GCMD) Science Keywords vocabulary. Filenames follow the structure:

C3S-312bL1-L3C-DAILY-CLOUD-INSTORACPLATFORM{}YYYYMMDD{_}fvVVV{_}.nc,

where INST and PLATFORM refer to the instrument and platform from which data originates (either ATSR2 and ERS2, or AATSR and ENVISAT for the CDR, or SLSTR and Sentinel-3a or -3b for the ICDR), YYYYMMDD provides the year, month and day covered by the daily mean product, and VVV denotes the product version (3.0 for the TCDR, 3.1 for the ICDR).

Data are provided as daily means of cloud properties, as described in Table 2-2, on a regular latitude-longitude grid, with a spacing of 0.1° in both dimensions (thus grid centres lie at -89.95°, -89.85°, -89.75°, …, 89.95° in latitude and -179.95°, -179.85°, -179.75°, …, 179.95° in longitude).

Unlike the monthly product, the daily product separates the variables into day and night observations, stored in different variables within the product. In this context "day" measurements are defined as those where the solar zenith angle is low enough that the solar reflectance can be used in the retrieval scheme. In the case of ORAC this means a solar zenith angle of θ0<80°. There are two main reasons for this approach:

1. In a given location and day, observations made by a sun-synchronous satellite during day and night will be separated by approximately 12 hours. By separating the averages into day and night values, we are essentially representing an instantaneous spatial average, rather than combining two instantaneous measurements separated by many hours.
2. Furthermore, if an average of all available data for a given day was taken, some grid-cells would contain data from both day and night observations, some would contain only day and some only night observations. Thus, the resulting product would contain discontinuities which would be solely an artefact of the averaging.

These two factors combine to mean that separate day and night averages are both more geophysically relevant and easier to interpret than a simple average over the whole day.

The list of cloud properties, the variable names used within the NetCDF data files and their units are given in Table 2-2.

Table 2-2: Cloud properties included in Cloud_cci TCDR and SLSTR ICDR daily products.

For lists and details of known issues and limitations of the data, the user is referred to the referenced ATBDs and papers in Section 1.1.

2.3.1 Uncertainties

The ORAC retrieval algorithm is an implementation of Optimal Estimation (Rodgers, 2000) and as such provides uncertainty estimates on all retrieved parameters, propagated from the uncertainty in the input satellite measurements (as well as any forward modelling error included in the uncertainty calculations). The propagation of the L2 uncertainty into L3 products, such as those brokered to the CDS, is not straight forward. Users should refer to the Cloud_cci Comprehensive Error Characterisation Report [D4].

Uncertainty and variability in the monthly mean cloud properties is expressed using three separate values, as described by Bennartz (2017):

1. Standard deviation about the mean. This value is the unweighted standard deviation of the L2 retrieval values about the mean value reported for each L3 pixel.

2. Propagated uncertainty in value. The value represents the L2 uncertainty propagated to the L3 pixel under the assumption that the L2 values used in the L3 average are independent and uncorrelated. It is calculated by summing the L2 uncertainties in quadrature and normalizing by the number of observations: 

where σ_i are the uncertainties of the  L2 pixels included in the average, and the <> indicates the mean value.

3. Correlated uncertainty in value. If a L2 pixels are assumed to be correlated with each other, the uncertainty in the resulting average can be calculated using: 

where σ²_std  is the unweighted standard deviation about the mean and c is the correlation between the L2 pixels.

2.3.2 Gaps and anomalies

A small number of gaps occurs in the daily means of the dataset. These are due to maintenance reasons or outages of instruments. In addition, anomalies appear and lead to a number of days which should be treated with caution. Table 2-3 provides a list of days which are either not available or anomalous.

Table 2-3: Summary of gaps and anomalies for each satellite

3. Data access information

The original repository for the CCI formatted L3 data can be accessed through the Cloud_cci homepage https://climate.esa.int/en/projects/cloud/data/, as can all relevant documentation from the project. It should be noted that Cloud_cci L3 data is not in the same format as data brokered to the CDS, however the same L2 retrieval output was used to produce both products. In detail, the official daily and monthly files from Cloud_cci contain more than 180 variables, but the specified C3S format split them into separate parts per ECV. That way, each file has a reduced data amount as well as an increased usability.

Cloud_cci data should also be available through the ESA_cci data portal http://cci.esa.int/data, which provides online data mining, analysis and visualization tools for CCI data through the CCI Toolbox.

Only the brokered (A)ATSR based CDR is available through CCI, with SLSTR data being supplied only to the CDS.

3.1 Data access through CCI

Within C3S, the distribution will be through the CDS (https://cds.climate.copernicus.eu/) where documentation created for the inclusion of the data in the CDS, such as this PUGS, will be also provided.

3.2 Data access through the CDS

Within C3S, the distribution will be through the CDS (https://cds.climate.copernicus.eu/) where documentation created for the inclusion of the data in the CDS, such as this PUGS, will be also provided.

3.3 Product ordering process

You need to be registered and logged in to order products in the CDS. A login is provided upon registration, all products in the CDS are delivered free of charge.

References

Bennartz, R., 2017: Error propagation in the generation of gridded satellite datasets, DWD Visiting Scientist Report.

McGarragh, G., C.A. Poulsen, G.E. Thomas, A.C. Povey, O. Sus, S. Stapelberg, C. Schlundt, S. Proud, M.W. Christensen, M. Stengel, R. Hollmann, R.G. Grainger, 2018: The Community Cloud retrieval for Climate (CC4Cl) – Part 2: The optimal estimation approach. Atmos. Meas. Tech., 11. 3397-3431, DOI: 10.5194/amt-11-3397-2018.

Rodgers, C.D., 2000: Inverse methods for atmospheric sounding, World Scientific Publishing Pte Ltd, New York, USA.

Sus, O., M. Stengel, S. Stapelberg, G. McGarragh, C.A. Poulsen, A.C. Povey, C. Schlundt, G.E. Thomas, M. Christensen, S. Proud, M. Jerg, R.G. Grainger, R. Hollmann, 2018: The Community Cloud retrieval for Climate (CC4Cl) – Part 1: A framework applied to multiple satellite imaging sensors. Atmos. Meas. Tech., 11, 3373-3396, DOI: 10.5194/amt-11-3373-2018.

Related articles

• Page:
  ERB TSI: Product User Guide and Specification (PUGS)
• Page:
  ERB TSI: Product Quality Assurance Document (PQAD)
• Page:
  ERB TSI: Product Quality Assessment Report (PQAR)
• Page:
  ERB TSI: System Quality Assurance Document (SQAD)
• Page:
  ERB TSI: Algorithm Theoretical Basis Document (ATBD)