Linear error in probability space (LEPS)

LEPS measures the error in probability space as opposed to measurement space, where CDFo( ) is the cumulative probability distribution of the observations, determined from an appropriate climatology (Ward and Folland, 1991).

The rationale behind transferring to probability space can be seen with the following example. A forecast is made of a temperature 7oC above average, and the verifying observation is 5oC above average. On another occasion the forecast is 1oC above average and the verifying observation is 1oC below average. In measurement space both forecasts are equally good, but it can be argued that the first forecast, which successfully forecasts an extreme value, is the better of the two. Transforming to probability space reflects this argument, as can be seen in the figure below.

For notational convenience, let Pf = CDFo(Fi ), Pv = CDFo(Oi ). Then LEPS is simply |Pf-Pv|.
 
 

The basic form of LEPS has some disadvantages. Potts et al. (1996) adapted the score to overcome these. The revised LEPS score is 3(1-|Pf-Pv|+Pf2-Pf+Pv2-Pv)-1.

This score is doubly equitable, it has simple values for unskilful (0) and for perfect forecasts (1), and there is no 'bending back' for categorical data. This last property refers to the fact that in the simpler form of LEPS it is possible, for ordered categorical data, that forecasts which are maximally incorrect are penalised less than those which are slightly less erroneous.

LEPS scores can be used on both continuous and categorical data, and a skill score, taking values between –100 and 100 (or –1 and 1) for a set of forecasts can be constructed based on the LEPS score, though it is not doubly equitable.
 

References
Potts, J.M., C.K. Folland, I.T. Jolliffe, and D. Sexton, 1996: Revised "LEPS" scores for assessing climate model simulations and long-range forecasts. J. Climate, 9, 34-53.
Ward, M.N. and C.K. Folland, 1991: Prediction of seasonal rainfall in the Nordeste of Brazil using eigenvectors of sea-surface temperature. Int. J. Climatol., 11, 711-743.
 

Ian Jolliffe, February 2003