The Extreme Forecast Index (EFI) is computed from the difference between Cumulative Distribution Function (CDF) curves of the M-climate and the forecast distribution of the current ensemble (ENS).  The calculations are made so that more weight is given to differences in the tails of the (climatological) distribution.

The Extreme Forecast Index is calculated according to the formula:

where Qf(Q) denotes the proportion of ENS members lying below the Q quantile of the M-climate record.  This is shown diagramatically in Fig8.1.9.2-1 where Q-Qf(Q) is represented by the blue line and the green shaded area. The term Q(1-Q) gives more weight towards the extremes of M-climate.

One can visually estimate the EFI by assessing the area between the M-climate (black) and ENS forecast (red) curves, and dividing this by what the area would be if all the ENS members predicted the M-climate extreme (i.e. a vertical line that meets the black curve at y=100%).  Whilst the answer derived by this method is only approximate it can nonetheless be a very useful aid to understanding.


  

Fig8.1.9.2-1: Schematic CDF diagram showing positive EFI as the area between the M-climate curve and the ENS curve.  The area is positive where the ENS curve (red line) is to the right of (i.e. values greater than) M-climate (black line).  Note forecast values beyond the limits of the M-climate (green dashed line) are not used in evaluating EFI and so how extreme these actually are is not accounted for. The Shift of Tails (SOT) concept was developed in part to address this disadvantage.



Fig8.1.9.2-2: Schematic CDF diagram showing negative EFI as the area between the M-climate curve and the ENS curve.  The area is negative where the ENS curve (red line) is to the left (i.e. values less than) M-climate (black line).  Note forecast values beyond the limits of the M-climate (green dashed line) are not used in evaluating EFI and so how extreme these actually are is not accounted for. The Shift of Tails (SOT) concept was developed in part to address this disadvantage.


Fig8.1.9.2-3: Schematic CDF diagram illustrating the impact on the EFI when there are both positive and negative contributions to the integral.  This arises whenever the ENS (red) and M-climate (black) curves cross.


Fig8.1.9.2-4: Schematic CDF diagram for rainfall showing positive EFI as the area between the M-climate curve and the ENS curve.  The area is positive where the ENS curve (red line) is to the right (i.e. values greater than) M-climate (black line).  Note forecast values beyond the limits of the M-climate (green dashed line) are not used in evaluating EFI and so how extreme these actually are is not accounted for. The Shift of Tails (SOT) concept was developed in part to address this disadvantage.



Fig8.1.9.2-5: Schematic CDF diagram for rainfall showing negative EFI as the area between the M-climate curve and the ENS curve.  The area is negative where the ENS curve (red line) is to the left (i.e. values less than) M-climate (black line).  The lower bound of the ENS forecast precipitation can be no lower than 0mm.  The lower bound of the M-climate for the vast majority of locations is also 0mm.  Thus negative values of EFI for 24hr total precipitation do not provide sensible information and should not be used.  In most places a dry day is not considered extreme or severe anyway. 

Since generally there can be no ENS members below the lower limit of the M-climate the Shift of Tails (SOT) concept generally does not have a meaning for the dry ENS extremities of the CDF.  Theoretically at some very rare locations where there has never been any dry days, it is possible for the minimum of M-climate to be above 0mm.   In these cases the Shift of Tails (SOT) concept can apply when no rain is forecast but it is not very informative.


The forecast Cumulative Distribution Function (CDF) or the associated Probability Density Function (PDF) curve (see Fig8.1.9.2-6 for some comparisons) normally does not agree precisely with the M-climate CDF or PDF.  Therefore the EFI normally takes non-zero values.

  • EFI = 0 (or 0%) where the ENS forecast probability distribution agrees precisely with the M-climate distribution, or when the overall total of positive and negative area contributions is zero.
  • EFI = +1 (or 100%) where all ENS members forecast values to be above the absolute maximum of the M-climate.
  • EFI = –1 (or –100%) where all ENS members forecast values to be below the absolute minimum of the M-climate. 

"Significant values" of EFI may be considered to be:

  • EFI of 0.5 to 0.8 (50% to 80%)  or –0.5 to –0.8 (–50% to –80%) generally signify an unusual” event. 
  • EFI values above 0.8 (80%) or below –0.8 (–80%) usually signify a very unusual or extreme event.

Negative EFI for precipitation for 24-hour accumulations does not make sense because the model climate (M-climate) precipitation curve is bounded by 0.  This is because completely dry days occur in almost all places and they will be incorporated when creating the model climate (M-climate).  However, negative EFI does make sense for accumulations over longer periods as few places consistently experience completely dry conditions during these longer intervals, and that should be reflected in a long-period model climate.  Negative precipitation EFI in this case does show the risk of anomalously dry weather.

Similarly, a negative wave height EFI would indicate relatively calm seas.


Time sequence of cumulative density functions and probability density functions

A convenient and powerful way to show the temporal evolution of successive ENS forecasts for a given day is to overlay the CDFs corresponding to each of those runs.  


 Fig8.1.9.2-6: A schematic illustration of the CDF (left) and PDF (right) for forecasts of 12hr accumulated precipitation showing the ENS T+48hr forecast (light green),  ENS T+96hr forecast (dark green) and ENS T+144hr (blue), together with the M-climate (black) verifying at the same time in the future.  The CDFs and PDFs both give, in different ways, a visual indication of mean, spread and asymmetry.

In Fig8.1.9.2-6 the area between the CDF lines and M-climate, and hence the EFI, is becoming greater as the verifying time approaches.  This suggests increasing probability of an unusual rainfall event.  EFI approaches +1 on the T+48 forecast suggesting very unusual rainfall compared to climatology.  The steepness of the CDF and hence the peaked shape of the PDF charts at T+48 indicate that many of the ENS members are showing similar results and thus an extreme event of the magnitude indicated (on the x-axis) can be considered quite likely (assuming of course that the forecasts are not systematically biased).



Fig8.1.9.2-7: Schematic set of idealised CDFs from a series of ENS runs (cyan earliest, red latest), for a variable for which the climatological distribution is approximately Gaussian (e.g. 2m temperature).  If the M-Climate (or forecast) CDF curve resembles a "skewed S-shape" then that distribution is approximately Gaussian. 

Fig8.1.9.2-7 shows how one may expect successive forecasts to behave as the valid date approaches:

  • At long lead-times, the CDFs may be fairly similar to the M-climate because of a broad variation within the ENS forecast results.
  • As forecast lead-time shortens, the ENS forecast results should become increasingly similar, with successive CDFs showing smaller and smaller variations relative to one another.
  • As forecast lead-time shortens, the CDFs should become steeper as the members within each successive ENS forecast become increasingly similar, implying higher confidence.

In practice there is often some non-uniformity of progress through this idealised sequence of events, particularly for maximum wind gust and rainfall parameters.  The user needs to identify patterns in the series of CDFs as forecast lead-time shortens and investigate any departures from the expected changes.

If the median of a new forecast lies outside the range of the previous forecast then a substantial "ensemble forecast jump" (for a given parameter) has occurred.  This would usually also be shown as a big change in EFI.  Such jumps are very uncommon, although probably they occur slightly more often than they should on purely probabilistic grounds.  They can be very disconcerting for forecasters.  Examples of meteorological scenarios when such behaviour is more likely are organised convective rainfall and extreme cyclonic windstorms.