Last modified on Oct 09, 2023 16:08

Introduction

The Copernicus Arctic Regional Reanalysis (CARRA) is a high resolution (2.5 km) reanalysis system for two domains of the European part of the Arctic (Figure 1). The CARRA dataset covers the period from September, 1990 and onward (timely updates are planned to be provided until June 2025). CARRA adds value to the ERA5 global reanalysis with the use of more (local) observations, with the focus on the representation of important local processes like cold surfaces and with its higher spatial horizontal resolution which allows for more details. The CARRA system is described in its Full System Documentation, while some key points of the configuration of the system is compared to ERA5 in Table 1.

The main findings of a comprehensive evaluation of the CARRA dataset are presented in this page. However, the CARRA dataset is huge, and a fully exhaustive examination of all parts of the data is simply not possible. The data users are therefore also encouraged to perform their own evaluation, tailored towards their own use of the data and also to seek information in the literature. A selection of studies, which includes evaluation and comparisons of different parts of the CARRA data set can be found here.

Table 1. Some key features of CARRA compared to ERA5.

Figure 1: The CARRA-West and CARRA-East (Copernicus Arctic Regional Reanalysis) domains.

Summary of added value with respect to ERA5

In brief, CARRA provides more spatial and temporal details, and is in general in better agreement with in-situ observations than ERA5. The added value of CARRA varies with parameter, region, season and characteristics of interest (e.g. bias or random errors, extremes  or general statistics, aggregation periods), but is in general higher for near-surface parameters in regions with complex terrain, coastlines and heterogeneous surface properties. However, some clear differences can also be found over sea ice due to different description of sea ice processes. CARRA suffers from covering a shorter time period, regional coverage (European parts of the Arctic) and only 3 hourly analysis output (hourly forecast output is available) compared to ERA5. 

• What are the most pronounced differences between CARRA and ERA5?

CARRA provides more spatial and temporal details for near-surface parameters. In particular in regions with local variations in surface forcing (e.g. topography, complex coastlines and other surface heterogeneity) the differences can be pronounced. Over more homogeneous surfaces (e.g. flat land surface, open ocean) the differences are smaller, but still present. Over sea ice, there also might be large differences between CARRA and ERA5 in periods due to the different representation of sea ice in the reanalysis. CARRA and ERA5 produce very similar large-scale structures, e.g. the location of synoptic scale low- and high- pressure systems. 

• How well is the general agreement between CARRA and observations? 

For most parameters the general agreement between CARRA and point observations is good, with relatively small deviations for most regions. However the agreement varies with parameter, region, season and if the parameter is used in the generation of the reanalysis (assimilated) or not (dependent on the underlying model system). On average, CARRA has a substantially better agreement with observations than ERA5.

• How well is temporal and spatial scales represented in CARRA?

The temporal variability in CARRA has a reasonable agreement with observed variability. However, it varies between regions, and are better for parameters that are used in the generation of the dataset (assimilated), than for parameters, which are not. Furthermore, the agreement with observations is higher for longer (e.g. daily and monthly values) than shorter aggregation periods (e.g. hourly). Also a reasonable representation of the smaller spatial scales are seen, but some of the shortest spatial scales are only partly resolved or missing. However, CARRA resolve both shorter temporal and smaller spatial scales better ERA5.

• How well is weather extremes represented in CARRA?

CARRA provides a more realistic representation of weather extremes than ERA5, both in their climatology and timing. However, also CARRA shows some deficiencies compared to observed extremes, e.g. the details of extremes that involve parameters used in the generation of the dataset are better captured than for the extremes with not assimilated parameters.

• What is the main difference of the accuracy of the analysis versus the forecasts in CARRA?

The CARRA data set include analysis data, which is based on a combination of observations and the underlying model system used, and forecast data which starts at the analysis but is integrated forward in time based on the underlying model system. The analysis shows a better agreement with observations than the forecasts and this is in particular true for parameters directly  used (assimilated) in the generation of the reanalysis. These parameters can also experience a rapid decrease in their accuracy between analysis and forecasts. The differences between analysis and forecast accuracy are smaller for other parameters. The accuracy of the forecasts decrease in general with increasing forecast lead time.  

• What are the key factors influencing the agreement between CARRA and observations?

A full agreement between CARRA and (point) observations can not be expected as 1) they represent different quantities (point versus spatial grid average), 2) not all observations are used in the generation of CARRA, 3) all models have inherent weaknesses, so also the model system which CARRA build on, and 4) observations may also have errors.

• What are the main weaknesses of CARRA compared to ERA5?

CARRA is available from 1991 until present day, only for the European Arctic region, and analysis is only available every 3 hourly (hourly time series can be made by combining analysis and forecasts though this should be done carefully, see some more details below). ERA5 therefore provide a better spatial and temporal coverage than CARRA. Furthermore, the data set is less used than ERA5, and hence also less scrutinised and validated by the scientific community and users in general. Although the added-value of the CARRA data is clear, as for all reanalysis data sets there is expected to be periods, regions or parameters with larger disagreements with observations.

• Where can I  find more detailed information about strength and weaknesses of the CARRA data set?

More details on the summarised information above can be found below. Furthermore, a selection of peer-reviewed work, conference papers and none peer reviewed work, which all include evaluation of parts of the CARRA data or comparison to other data sets can be found here.

More spatial details in high resolution reanalysis

The higher spatial resolution in CARRA, compared to ERA5, provides a more realistic description of surface properties and CARRA therefore includes more spatial details. The 2m air temperature over the southern part of Greenland and Iceland for 15 February 2014 is an example of this (Figure 2). The large scale patterns are very similar, but at Iceland and along the complex coast line of Greenland CARRA provides more details.  Another example is shown in Figure 3, for an extreme precipitation event at Svalbard on 8 November 2016. The daily accumulated precipitation in ERA5 is smoothly distributed in space, with relatively low maximum values (less than 50 mm/day). On the other hand, CARRA shows more spatial details which to a high degree follow the topography of Svalbard with maximum values higher than 150 mm/day. Finally, in Figure 4 the mean absolute difference between CARRA-East and ERA5 for 2m air temperature and 10m wind speed over a winter period is shown. Large differences are seen for both parameters over regions with complex terrain and coastlines, like over Svalbard and the Norwegian coast and mountains. Less differences, but still present, are found over the ocean and more flat terrain as in northern Sweden and Finland. Over the sea ice, a substantial difference in temperature is found due to the different representation of sea ice in the two reanalysis.

Figure 2. 2m air temperature at south Greenland and Iceland 15 February 2014 00 UTC for CARRA (left) and ERA5  (right). Figure adapted from Schyberg et al. (in prep). 

Figure 3. 24 hour extreme precipitation (mm/24h) Svalbard 7-8 November 2016 for CARRA (left) and ERA5 (right). Figure adapted from Schyberg et al. (in prep).

Figure 4. Mean absolute differences between CARRA and ERA5 for 2m air temperature (left) and 10m wind speed (right) during February and March 2018. Figure adapted from Figure 1 in Køltzow et al. (2022).

Errors in near-surface parameters 1998-2018 (to be updated soon with timeseries from 1991 to present)

CARRA is on average, in better agreement with observations than ERA5 as shown in Figure 5 (Note: the figures will be extended to the entire reanalysis period soon). In particular, a pronounced added-value in terms of smaller Root Mean Square Error (RMSE) is seen for the near surface parameters (temperature, specific humidity and wind speed). However, CARRA has also slightly in better agreement with observations of the Mean Sea Level Pressure (MSLP) which represents larger weather scales. Figure 5 also shows a clear annual cycle, with larger errors during winter time for MSLP, temperature and wind speed for both reanalyses. However, the specific humidity, has in absolute terms larger errors in summer than winter (more presence of moisture in warm conditions make it prone to larger errors). Also some inter-annual variability is seen in the RMSE, which reflects that some periods or years are less predictable than others, and that the reanalysis systems better represent some weather conditions than others.

Figure 5. Time series of Root Mean Square Error (RMSE) for the period of 1998-2018 averaged over both CARRA domains for CARRA (red) and ERA5 (blue) for Mean Sea Level Pressure, 2m air temperature, 10m wind speed and 2m  specific humidity. Figure is adapted from Schyberg et al. (in preparation). 

Regional differences 

As shown above, CARRA has a good general agreement with in-situ observations for a selection of near-surface parameters. In this section, it is shown in more detail how the agreement with observations for both CARRA and ERA5 varies for different parameters, regions, between systematic errors (measured by the mean error or bias) and the unsystematic errors (measured by the standard deviation of the error, SDE), and season. Although bias and SDE are presented for regions (with similar weather and climate conditions) it should be noted that the errors also vary between locations within each region.

For Mean Sea Level Pressure (MSLP, Figure 6), the slightly lower RMSE for CARRA compared to ERA5 (seen in Figure 5) is seen for most regions in both bias and SDE. In addition, particular the SDE shows differences between regions (both CARRA and ERA5), e.g. lower errors in coastal areas and larger errors inland. An explanation for this is the uncertainty associated with the reduction of surface pressure to sea level in the latter regions (more topography) for both reanalyses and in the observations. For MSLP the lower errors in RMSE during summer than winter (Figure 5) is mainly due to lower SDE. 

For the 2m air temperature (T2m) errors (Figure 7), the errors of the reanalysis differs from region to region. While CARRA shows relatively small biases for all regions, the SDE is higher over inland Greenland and Scandinavia during winter and at Greenland (coast and inland) in summer. The same feature in SDE is also seen for ERA5. However, the errors for CARRA are less pronounced than for ERA5. ERA5 show larger biases than CARRA for several of the regions.

For the 10m wind speed (Figure 8), there are a relatively large regional variations in the agreement with observations. In winter at Greenland, CARRA has a positive wind bias (stronger wind than observed) and also relatively large SDE. It should be noted that observations from Greenland during winter time may include more observation errors than elsewhere which can play a role in the assessment. Also Iceland and Svalbard show larger SDEs than other regions. The only region with a negative wind bias (underestimation of observed wind speed) in CARRA is the coastal areas of Gulf of Bothnia with an underestimation of ~0.5 m/s. Opposite to this, ERA5 show an underestimation of the wind speed for most regions with the exception of inland Scandinavia and inland Greenland. In summer, both the biases and SDEs are smaller than during winter time, but a general underestimation of the wind speed in ERA5 is still present.

Figure 6: Mean error (bias) plotted against standard deviation of the error for Mean Sea Level Pressure (MSLP) in CARRA (blue) and ERA5 (red) for different regions: Svalbard (marked S), Norwegian Coast and fjords (NC), Scandinavian inland (SL), Coast of Gulf of Bothnia (BC), Iceland coast and fjords (IC), Iceland inland (IL), Greenland coast (GC) and Greenland inland (GL). The scores are calculated against synop observations over the period 2010-2019 for December, January and February (left) and June, July and August (right). Note that the axes are different for the two periods. 

Figure 7: Mean error (bias) plotted against standard deviation of the error for 2m air temperature in CARRA (blue) and ERA5 (red) for different regions: Svalbard (marked S), Norwegian Coast and fjords (NC), Scandinavian inland (SL), Coast of Gulf of Bothnia (BC), Iceland coast and fjords (IC), Iceland inland (IL), Greenland coast (GC) and Greenland inland (GL). The scores are calculated against synop observations over the period 2010-2019 for December, January and February (left) and June, July and August (right). Note that the axes are different for the two periods.

Figure 8: Mean error (bias) plotted against standard deviation of the error for 10m wind speed in CARRA (blue) and ERA5 (red) for different regions: Svalbard (marked S), Norwegian Coast and fjords (NC), Scandinavian inland (SL), Coast of Gulf of Bothnia (BC), Iceland coast and fjords (IC), Iceland inland (IL), Greenland coast (GC) and Greenland inland (GL). The scores are calculated against synop observations over the period 2010-2019 for December, January and February (left) and June, July and August (right). Note that the axes are different for the two periods.

Hourly, daily and monthly values

So far, the comparison with observations has been done with hourly observations for all 3-hourly analyses from CARRA. However, many users of reanalysis either use daily, monthly or even yearly values. While biases remain the same for different time aggregations, the non-systematic errors may change when looking at longer time aggregation periods. In Figure 9, the SDE for hourly, daily and monthly values for 2m air temperature, Mean Sea Level Pressure, 10m wind speed and 2m relative humidity are shown for CARRA and ERA5. A clear tendency towards lower SDE for longer aggregation periods are found, i.e. the monthly and daily mean values are in better agreement with observations than hourly values. This is reasonable as it is easier to represent monthly means compared to daily and even hourly variations within the given month. A tendency for the added value in terms of difference in SDE decrease is also seen which also is reasonable since ERA5 are relatively better at describing larger temporal and spatial scales than the smaller scales.

Figure 9. Standard deviation of the error in CARRA and ERA5 as a function of aggregation periods (hourly, daily and monthly) for 2m air temperature, MSLP, 10m wind speed and 2m relative humidity. Scores are averaged over all available observation sites from 2010 - 2019.

Analysis versus Forecasts?

CARRA provides analysis only 3-hourly (00, 03, 06, 09, 12, 15, 18 and 21 UTC), while ERA5 provides hourly analysis. If hourly values are required from CARRA it is therefore necessary to combine analysis with short forecasts. However, the characteristics of the analysis and the forecasts and how they agree with observations may differ. In Figure 10, the RMSEs for 2m air temperature, mean sea level pressure, 10m wind speed and the 2m dew point temperature in CARRA from forecasts (divided by the RMSE in the analysis) are shown as a function of forecast length. For example a value of 1.5 implies that the RMSE is 1.5 times larger in the forecast than in the analysis. Note that the RMSEs are calculated for the same time for both forecasts and analysis, i.e. that e.g. the RMSE of 3h forecasts initiated at 21 UTC and valid for 00 UTC are compared with the RMSE from the analysis at 00 UTC. 

The error growth for 2m temperature and dew point temperature is rapid during the first hours and then slows down. MSLP grows slower, but continues to grow at a larger rate for longer forecast lengths. Opposite to this is the 10m wind speed, for which the RMSE grows very slowly with forecast length. An interpretation of these results is that the three first parameters are included in the assimilation process of the reanalysis and thereby more drawn towards the true state of the observed climate system, and then they tend to drift back to the climate of the underlying model system with increasing forecast lengths, while 10m wind speed is not assimilated and hence is already at the analysis time more similar to the characteristics of the underlying model system. This underlines that the construction of time series combining forecasts and analysis from CARRA should only be done with care.

It should  be noted that in this discussion errors (as represented by RMSE) are defined in terms of deviations from observations. But observations are point measurements which have their own errors, including representation errors relative to the mean value over a model grid box area which the reanalysis tries to represent. Taking observation errors (with possibly temporal error correlations) into account, the relative difference in the real errors between forecast and analysis may be smaller than indicated here. (See also discussion below on "factors influencing the agreement between CARRA and observations.)

Figure 10. RMSE of forecasts and analyses divided by the RMSE of the CARRA analysis as function of forecast length. Parameters shown are 2m air temperature (black), MSLP (red), 10m wind speed (blue) and 2m dew point temperature (cyan) for CARRA-East and CARRA-West domain. At +0 hour the RMSE of ERA5 divided by the RMSE of CARRA (circles) is also included.

Precipitation

Daily precipitation from CARRA and ERA5 over northern Norway, Sweden and Finland are compared with in-situ observations in Figure 11 (Note: the figures will be extended to the entire reanalysis period soon). All precipitation observations are associated with uncertainties and errors, and one particular challenge is the under-catchment of (in particular) solid precipitation in windy conditions. The evaluation is therefore split into liquid and solid precipitation, where the results from the latter are more uncertain and the following discussion focus on the results from the liquid precipitation evaluation. The total precipitation amounts in both reanalyses are larger than in the observations, but CARRA has a smaller overestimation than ERA5 (Figure 11a). The correlation for daily precipitation is higher (better) in ERA5 than in CARRA (Figure 11b), while CARRA scores better in identifying days with precipitation (higher ETS in Figure 11c) and has slightly higher 99 percentiles values than ERA5, and are thereby in better agreement with the observations of high-precipitation events (Figure 11d).  

Comparing the precipitation from two forecast systems with widely different resolution is challenging. For example, the high-resolution CARRA will be harder punished by small displacement errors of small precipitation systems (double-penalty error and likely worse correlation score), while the coarser resolution ERA5 will not be able to predict the highest precipitation amounts, but instead provide a too smooth precipitation field (and likely worse score for high precipitation events and in deciding precipitation days). These aspects can be illustrated by the extreme precipitation event at Svalbard shown in Figure 3. The maximum observed precipitation during the episode was 86.8 mm/day at Ny-Ålesund which is almost the double of the maximum value in ERA5. Compared to this, CARRA may produce such high values, but locate the maxima slightly wrong, but still contain useful information on the general precipitation patterns. Furthermore, there are many attributes of precipitation estimates that are important depending on the use (e.g. total precipitation amount, frequency of precipitation, high-precipitation events, time aggregation etc). In addition, since a major part of the precipitation will be in solid form, the observation uncertainty further complicates the evaluation in the Arctic region. The answer on the goodness of the precipitation estimates from  CARRA and ERA5, is therefore not straightforward. However, for several precipitation forecast attributes it seems like CARRA can add value to the ERA5 precipitation, but not necessarily in all respects.

Precipitation is taken from forecasts from the underlying model system and is not assimilated. Hence, a question of what forecast lead times are appropriate to use is relevant. The accuracy of the forecasts are reduced for forecasts with increasing lead times, while the shortest lead times may experience a spin-up effect before the model system finds its balance. In CARRA, the spin up is modest and relatively short. The difference in precipitation amount  is less than 5% between the shortest and longer lead times and negligible after 6h. Hence, a way to minimise the impact of the spin up is to combine 12 h accumulated precipitation by the difference of precipitation at lead time 18 and 6h from forecasts initiated at 00 UTC and 12 UTC. However, if accuracy in timing of precipitation events is of high importance, or hourly resolution is required, an option could be to combine hourly forecasts from each of the 8 analysis times (00, 03, 06, 09, 12, 15, 18, 21 UTC) for lead times 1,2 and 3, since the spin up for these lead times is relatively small. Also some care should be made if precipitation is used at locations close (~100-200 km) to the lateral boundaries of the model domain as a lateral spin up, varying along the boundaries, is present.

Figure 11. Verification of daily precipitation (mm/day) for CARRA (red) and ERA5 (blue) for liquid (left) and solid (right) precipitation. Each box plot summarises all observation sites in Norway, Sweden and Finland available for a given year. Comparisons shown: (a) reanalysis / observed precipitation, b) correlation, c) Equitable Threat Score for precipitation/no precipitation and d) reanalysis / observed 99 percentile of yearly precipitation. Figures are from Schyberg et al. (in preparation).

Factors influencing the agreement between CARRA and observations

There are number of reasons why reanalysis may differ from point observations, and some of these are briefly described below as they can help  to understand the strengths and weaknesses of the CARRA dataset.

• Reanalysis and (point) observations represent different quantities

While reanalysis represents a grid box average (2,5 x 2,5 km in CARRA, 31 x 31 km in ERA5) observations represent a point and can usually vary within the grid boxes of the reanalysis. This sub-grid variability can not be resolved or represented by the reanalysis. This constitutes a substantial part of the difference between reanalysis and observations and is discussed in some detail in Køltzow et al. (2022). Furthermore, the sub-grid variability varies with parameter and are larger in absolute terms for ERA5 than CARRA.

• Observation applied in the assimilation process

While observations of some parameters are used in the generation of reanalysis, others are not used. In the comparison with the final reanalysis product it is therefore expected that the agreement with observations are better for the assimilated parameters (not independent observations), than for parameters not assimilated (independent observations). In the assimilation process of CARRA, near-surface observations of surface pressure, 2m air temperature and humidity and snow depth is used (assimilated), while e.g observations of 10m wind speed and precipitation are not assimilated. The parameters not used directly experience therefore a weaker adjustment towards the observed conditions than the more directly used parameters. It should be noted that the majority of the assimilated observations uptake in the reanalysis are not near-surface observations, but taken from other measurements, e.g. by satellites, aircrafts and radiosondes.

• Model errors

A reanalysis describes the climate system and it is a combination of observations and an underlying model system. No model systems are perfect and both systematic and more non-systematic errors exist and will contribute to deviations from observations. These errors vary with parameters, weather conditions, regions and forecast lengths (to mention a few). For example regions with a high-density surface observation network depend more on the assimilated observations, than similar regions with less dense network where the output depend more on the underlying model system. The available near-surface observation sites for CARRA is shown in Figure 12. The corrections in the surface analysis (analysis increments) for temperature are reduced to 2/3 of its value 70 km away from the observation site. A rough estimate shows that less than 20% of Greenland, 50% of Svalbard and 65-70% of Iceland and Scandinavia are closer than 70 km to an observation site. Similarly, the analysis increment is reduced to 1/3, 120 km away from the observations. This means that the influence from surface observations is small at about 65% of Greenland and 20% of Svalbard, Iceland and Scandinavia. Furthermore, model errors depend on the type of weather, e.g. a well known problem in many model systems is the capability to represent the stable boundary layer properly. 

• Observation errors

Although observations used in both assimilation and verification of the CARRA reanalysis has undergone quality control, there still might be remaining observational errors or uncertainty. In particular the harsh climate and remote location of Arctic observations make the Arctic a region with relatively high observational errors. This yields for all parameters, but maybe in particular for solid precipitation as discussed briefly above. 

Figure 12: The two CARRA domains (black frames) with observation sites (red dots) available for the entire or parts of the CARRA period. The observation sites may observe different sets of parameters and not necessarily during the entire CARRA period.

Relevant CARRA documents

Nielsen, K. P. et al.: Copernicus Arctic Regional Reanalysis (CARRA): Data User Guide. Available at Copernicus Arctic Regional Reanalysis (CARRA): Data User Guide

Yang, X., et al., 2020: C3S Arctic regional reanalysis - Full System documentation. Available at https://datastore.copernicus-climate.eu/documents/reanalysis-carra/CARRAFullSystemDocumentationFinal.pdf

Yang, X., et al., 2020: Complete test and verification report on fully configured reanalysis and monitoring system. Available at https://datastore.copernicus-climate.eu/documents/reanalysis-carra/CARRATestVerificationFinal.pdf

Bojarova J., 2020: Uncertainty estimation method. Available at https://datastore.copernicus-climate.eu/documents/reanalysis-carra/CARRAUncertainty%20estimationFinal.pdf

Copernicus Arctic Regional Reanalysis: List of references. Copernicus Knowledge Base (CKB) article. Copernicus Arctic Regional Reanalysis (CARRA): list of references

Uncertainty information for the Copernicus Arctic Regional reanalysis. Copernicus Knowledge Base (CKB) article. Copernicus Arctic Regional Reanalysis (CARRA): known issues and uncertainty information#Uncertaintyinformation

Known issues for the Copernicus Arctic Regional reanalysis. Copernicus Knowledge Base (CKB) article. Copernicus Arctic Regional Reanalysis (CARRA): known issues and uncertainty information#Knownissues

Related articles

• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): known issues and uncertainty information
• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): Data User Guide
• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): Added value to the ERA5 global reanalysis
• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): list of references
• Page:
  Copernicus Arctic Regional Reanalysis (CARRA)