Comparison of deterministic forecasts generated by computer models and manual methods

Forecasts from NWP models and human forecasters cannot really be compared, because they have different aims:

  • NWP modellers try to provide forecast systems with optimum accuracy.  Their aim is to model all scales of atmospheric motions, irrespective of whether they are predictable or not. (see Fig6.3-1).
  • Weather forecasters do the opposite. They disregard and dampen down unpredictable features in order to improve accuracy and reduce jumpiness in their categorical forecasts (see Fig6.3-2).


  

 Fig6.3-1: Typical root mean square error and variability of good (blue) and poor (red) high-resolution NWP models.

A good NWP model represents the whole spectrum of resolvable atmospheric scales throughout the forecast.  Thus errors trend towards a higher level but variability remains fairly constant.
A poor NWP model suffers from a gradual reduction of atmospheric scales through the forecast (due to excessive diffusion or coarse numerical resolution).  Thus errors trend towards those of a forecast using climate alone and the variability decreases.
 

  

 Fig6.3-2: Typical root mean square error and variability of experienced (blue) and naïve (red) forecast practices:

An experienced forecaster or process (blue) disregards or damps less likely synoptic features.  Thus errors tend towards those of a forecast using climate alone and variability reduces.  This is because the less predictable scales are gradually removed.
A naïve forecaster or process (red) just reads off raw output from a good NWP model.  Thus errors tend towards a higher level while variability remains fairly constant.  This is because the forecasts maintain the whole spectrum of resolvable atmospheric scales, whether predictable or not.


In summary, a "Good” deterministic forecast performance cannot be judged with the same yardstick used by NWP modellers, forecasters or end-users.  What looks bad might be good, what looks good might be bad.
If the reduction in categorical forecast errors occurs:
  • during the forecast integration, due to deficiencies in the NWP model, then this is "Bad".  This is  because the whole spectrum of resolvable atmospheric scales has not been captured.
  • after the forecast integration, by some subjective or objective post-processing of the NWP output, then it is “Good”.  This is because experience allows less likely features to be removed.

Any “competition” between NWP modellers and forecasters has no relevance outside the meteorological community.  The usefulness of a forecast depends upon both the NWP forecast and human interpretation.