Point Rainfall (from "ecPoint")

In April 2019 ECMWF introduced a new type of experimental product - "Point Rainfall" - into ecCharts, following several years of development work. The issue it aims to address is illustrated on Fig8.1.7-1.


Fig8.1.7-1: Radar-derived rainfall totals over part of Northern Ireland for the 12h period ending 00UTC 29 July 2018. Scale is in mm. Flash floods occurred in some locations.  Figure based on data from netweather.tv (external website).

Let us assume here that the radar-derived totals shown are accurate, and also indicate what would have been measured locally by raingauges. Then we  consider the ENS gridbox highlighted. Within this box, whilst the gridbox average rainfall total is about 17mm, the minimum and maximum rainfall amounts are about 2mm and 60mm respectively. This implies a lot of sub-grid variability. A completely accurate ENS member forecast would predict 17mm. But clearly this of itself would give the user no idea that locally there was much more (and indeed much less) than this amount. And to cause flash floods, as were observed, probably a 17mm total, locally, would not have been sufficient. The point rainfall aims to estimate the range of totals likely within the gridbox, and indeed deliver probabilities for different point values within that gridbox (albeit without saying where the largest and smallest amounts are likely to be). In other scenarios (e.g. frontal) rainfall totals will be much more uniform across grid boxes, but there are also recorded instances of even larger sub-grid variability. An unusual event in southern Spain in 2018 lead to a range of 12h rainfall in one ENS gridbox from 0 to ~350mm.

So three key aspects to appreciate from the outset are:

  • The new product is a post-processed product, based on calibrating ENS fields, including the Control run
  • It delivers probabilistic forecasts of rainfall totals for points - i.e. that would be measured by a raingauge randomly located within a model gridbox (hence the word 'point')
  • By way of comparison the raw ENS delivers probabilities for gridbox-average rainfall (in the current ENS that means ~18x18km boxes, up to day 15)
  • The term rainfall here means all precipitation types: rain, rain plus snow, snow etc., always converted into mm of rain equivalent.

Terminology:

  • "ecPoint" is the post-processing philosophy and software package. It can in principal be applied to any model variable.
  • "Point Rainfall" is the new product itself, specifically the output of the ecPoint package applied to rainfall totals.
  • "Gridbox-weather-type" is a classifier for the weather type forecast for a given gridbox during the period of interest (e.g. such a type could be ">75% of the rainfall total forecast is classified as convective, the maximum CAPE forecast is >500J/kg, ...")

There are two reasons why point rainfall and ENS rainfall distributions commonly differ:

  • Sub-grid variability, which itself varies according to the "gridbox-weather-type"
  • Model biases, on the grid-scale, also intrinsic to the "gridbox-weather-type"

The sub-grid variability issue was illustrated on Fig8.1.7-1.  Model bias in ENS rainfall forecasts, on the grid-scale, can also be discussed by referencing this figure. Let's say that, when a particular gridbox-weather-type is forecast in an ENS member, which was what was occurring in the highlighted gridbox on this day, that ENS member will tend, on average, to over-predict the gridbox rainfall by 15%, then one would expect the ENS to deliver a forecast not of 17mm, but of ~20mm. During the post-processing ecPoint should, as well as accounting for sub-grid variability for that ENS member, also reduce the gridbox average from 20mm to 17mm.

In practice, ecPoint post-processing creates, on the basis of the calibration results (see below), an "ensemble of ensembles", that is 100 new point rainfall realisations for each ENS member, and so comprises, at one intermediate stage in the computations, 5100 equally probable values of point rainfall totals within each ENS gridbox. However, prior to saving, the values for each gridbox are sorted, and distilled down into 99 percentile fields (1,2,..99). As a set these percentile fields constitute the Point Rainfall product, that can be displayed in different ways. Put together they also make a point rainfall distribution, which one could compare directly with an equivalent distribution formed by putting together the 51 members of the raw ENS (compare red and blue curves on Fig8.1.7-2).

Fig8.1.7-2 shows CDF examples for different sites on two different occasions for the full ensemble, illustrating both bias correction and the addition of sub-grid variability within ecPoint (notably for the first plot), but showing also that the point rainfall and raw model grid-scale distributions may sometimes be quite similar (second plot).


Fig8.1.7-2: Two CDF examples comparing raw grid-scale (red), post-processed bias-corrected grid-scale (green), and post-processed point rainfall (blue). The first plot is for a site in Spain in summer at day 2. The second plot is for a southern England site in autumn, at day 4. Note that for each case the areas to the left of the green and blue curves, that represent the mean gridbox rainfall over the ENS, should be the same.

Output formats

Output from ecPoint can in principle be for pre-defined periods of time, or instances in time. For point rainfall we currently make available overlapping 12h periods up to day 10, namely T+0-T+12, T+6-T+18,... T+234-T+246. In future we plan also to make available equivalent products for 6 hour and 24 hour intervals.

Calibration

As with any post-processing system ecPoint has to be calibrated. For this it uses short-range Control run forecasts of 12h rainfall covering one year (the "training period"), which are individually compared with rainfall observations, for the same times, within the respective grid boxes. The full procedure is not described here, but involves segregation according to gridbox-weather-types, which each have different sub-grid variability structures and/or different bias corrections associated. The 12h point rainfall system introduced into operations in April 2019 incorporated 214 such types. The type definitions are currently based on the following parameters: convective rainfall fraction, total 12h precipitation forecast, 700hPa wind speed, CAPE, 24h clear-sky solar radiation.

Verification

1 year of global verification of 12h point rainfall products indicates that when compared to point observations, and relative to raw ENS forecasts, the point rainfall forecasts are much more reliable, and have a much better discrimination ability. The net frequency of point observations of no (measurable) rain is much higher in reality than it is in raw ENS gridbox forecasts, whereas the net frequency in point rainfall forecasts is almost perfect, as shown by the verification reliability metric. Meanwhile large totals, such as 50mm/12h, are much better delineated by the point rainfall; using the ROC area metric day 10 forecasts in the point rainfall are as good as day 1-2 forecasts from the raw ENS for this threshold.

Some Uses of Point Rainfall

By giving, for example, non-zero probabilities for very large totals that had a zero probability in the raw ENS output, the Point Rainfall can provide a useful new pointer to when flash floods are possible locally. Likewise if one wants a better idea of how likely it is that a given period remains dry, the point rainfall can usually provide better guidance; indeed in convective situations the point rainfall probabilities for dry should be much better. And where a certain criteria has to be met as the basis for a warning or alert, which might not always be a large total, again the point rainfall should overall provide customers with better guidance than the raw ENS.

The ecCharts Products

Users can type "point" into the "Add Layers" filter box to find point rainfall layers. The first release of point rainfall products into ecCharts in April 2019 came with two new layer options for map display: 

  • The user can select a particular percentile (1,2,..99) and display the amount corresponding to that percentile
  • The user can select a threshold of interest and display the probability of exceeding that threshold (or alternatively of being less than the threshold)

In each case the output forecast values should be thought of as corresponding to any point, within the requisite gridbox, that the forecaster may require a forecast for. The lead times available are overlapping 12h periods up to T+246h, as stated above.

For percentiles 4 colour-fill styles and one contouring style are available for the point rainfall. For probabilities there are 2 colour-fill styles and one contouring style. All 6 colour-fill styles are illustrated in legend format on Fig8.1.7-3 These styles have also been added as options for the raw ENS ("total precipitation percentile" and "total precipitation probability") layers to facilitate direct comparison with the point rainfall; within ecCharts users need to scroll down through the "style" options to find them.


Fig8.1.7-3: Colour fill options for a user-defined percentile (first plot, amounts in mm), and a user defined threshold (second plot, probability in %)


Fig 8.1.7-4 and Fig8.1.7-5 below show some ecCharts examples of new products, comparing with raw ENS, for lead time T+78-T+90. Because of the sophisticated post-processing undertaken, the relationship between point rainfall and raw ENS products can vary a lot, according to the amount and range of forecast rainfall totals, and the range of forecast gridbox-weather-types (which all depend also on lead time). The user-selected threshold is also of fundamental importance. However the following generalisations usually hold for map-based products, and may help forecasters to interpret the output (examples of some of these can be found on the plots below):

  • Point rainfall charts are usually smoother than raw ENS charts
  • Point rainfall output usually broadens the raw ENS distribution
  • Values for high percentiles (e.g. 95+) are often much greater in the point rainfall
  • The 50th percentile (median) is usually less in the point rainfall
  • The percentile where point rainfall and raw ENS are similar tends to be around 85%
  • Probabilities of dry (or more strictly "no measurable rain") are usually greater in the point rainfall
  • Outlier extreme rainfall values in say 1 or 2 raw ENS members will usually be reduced down in the point rainfall
  • Bias correction effects can very occasionally be very large - e.g. doubling or halving the rainfall
  • It is more common for bias correction to reduce amounts than to increase amounts, but usually such changes are small in magnitude (e.g. up to ~20%)
  • Point rainfall charts from successive forecasts, for a given valid time, tend to be less jumpy than equivalent raw ENS charts


Fig8.1.7-4: 98th percentile of 12h rainfall from the raw ENS (first plot) and point rainfall (second plot)


Fig8.1.7-5: Probabilities of >10mm from the raw ENS (first plot) and point rainfall (second plot)

Limitations

Users should be aware of some known limitations of point rainfall output, which are listed below:

  1. Rainfall events that are themselves extreme for the world as a whole, such as those related to tropical cyclones, are unlikely to have been adequately represented in the calibration dataset, so point rainfall output can incorporate some misleading aspects - e.g. a finite probability of zero rain close to the TC track. We would recommend using the raw ENS rainfall near to TC tracks.
  2. Similar to (1) above, if the rainfall characteristics of a given site are known to depend very strongly on nearby topographic/coastal features then it may be that these are not adequately captured in the calibration process. In which case local knowledge or local MOS (model output statistics) may outperform the point rainfall.
  3. In a situation of large-scale (i.e. non-convective) rainfall over unresolved (sub-grid) orography, when cloud level winds are not light, users can reasonably expect the higher values in the point rainfall distribution to be on the upwind side, and the smaller values on the downwind side of any topographic barrier. However the magnitude of orographic enhancement, and potentially also the magnitude of the rain shadow effect, may be substantially under-estimated. "Spill over" of rainfall onto the lee side may also complicate the picture. Note also that the point rainfall probabilities ordinarily denote what is a spatially random draw from a gridbox; here we are advising that user experience/knowledge may be able to preferentially locate the smaller and larger values. This is something which would not be appropriate or possible in a situation of e.g. convection over an inland plain. Whilst biases in large scale rainfall over mountains may not be well handled, there is evidence that large biases in convective rainfall over mountains can sometimes be helpfully corrected for in the point rainfall.
  4. Diurnal cycle errors in convection are not currently catered for in the point rainfall.
  5. Calibration for very small totals is dependant on measurable rain ("trace" in observation records is counted as zero during the calibration). So if one wanted to count small (unmeasurable) amounts of rain as not dry, then using ecCharts to display "probability = 0mm" for the point rainfall would mostly give the user an overestimate of the probability of dry weather.
  6. Whilst ecPoint can increase or reduce, using a multiplying factor, the net rainfall in a given forecast from one ENS member (i.e. bias correct), it will never convert a forecast of zero rainfall to anything other than zero. This means that if all ENS members have zero rain in a given 12h period, the point rainfall will also show a 100% chance of zero rain in that period.
  7. Calibration data for ecPoint comes from land sites only, so strictly speaking forecasts might not be as valid for sea areas (particularly for surface-based convection). Nonetheless experience suggests that we do not generate unrealistic-looking output over sea areas.

Future products may address items 3 and 4, and include a cdf comparison option.

Upgrades to ecPoint

From the 12UTC ensemble runs on Monday 23rd May 2022 onwards the post-processing algorithm differs in the following three ways:

  1. The mathematical representation of the mapping functions, that convert ensemble member gridbox rainfall into point rainfall, essentially changes from a discretised to a continuous format, whilst remaining non-parametric. The distribution tails, which are all-important for capturing localised extremes, improve as a result. Map plots attain a smoother appearance, particularly at short leads.
  2. The mapping functions and decision tree structure have been completely re-built for model cycle 47r3, partly to account for the fact that 47r3 included substantial changes to model physics.
  3. Whilst the five governing variables in the decision tree are basically the same, two are now more representative of reality:
    1. The "maximum CAPE" variable now uses the maximum of 12 CAPE values during the valid period whilst previously it used just 3.
    2. Our use of the "clear sky solar radiation" variable now takes better account of aerosol.

Together these changes should result in post-processed forecasts that are more reliable and more skillful.

See https://confluence.ecmwf.int/display/FCST/ecPoint+output+improved for further information.

Additional Sources of Information

(Note: Some aspect of older material may now be out of date)

Read a paper preprint on ecPoint
View a recording of an ecPoint webinar
Read more on New point-rainfall forecasts for flash flood prediction.
Read more on Forecasting Convective Events in late May.