Center during the winters (DJF) of 1981/82 to

and during the winters 1980/81 to 1984/85. Winter

1989/90. Operational weather forecasters usually

is de¬ned as the November to March cold season.

consider 60% as a threshold for useful forecasts.

From Branstator [61].

From Kalnay et al. [209].

and P = P ’ C to represent the corresponding

Brier skill score [285]. This decomposition pro-

vides useful insights into the interpretation of the anomalies.

Brier skill score and both the time and anomaly If forecasts are veri¬ed over time at a ¬xed

location (i.e., t = i) we ¬nd that the mean squared

correlation skill scores. The Murphy“Epstein de-

composition will not be used subsequently in this error and correlation skill scores (18.3, 18.4) are

book so readers may feel free to skip this section. given by

Suppose that a set of n forecasts Fi and n

S F P = E (F ’ P)2 = E (F ’ P )2

2

predictands Pi are available for the veri¬cation of

E FP

forecasts. Usually the index i refers either to time

ρF P =

or to space. In the former case the index i refers

Var P Var F

to different times, so that Pi is a predictand such

as temperature observed at a ¬xed location at time If forecasts are veri¬ed across space at a ¬xed time

t = i. In the latter case the index i refers to a (i.e., x = i) then the mean squared error and the

location x, so that Pi represents the predictand at anomaly correlation coef¬cient (18.17) (the time t

a ¬xed time at location x = i. We can use the is omitted) are

Murphy“Epstein decomposition of the Brier skill

(S A )2 P = (Fx ’Px )2

score in both cases. F

(Fx ’ F )(Px ’ P )

ρF P =

A

(Px ’ P )2 (Fx ’ F )2

18.3.2 The Correlation Decomposition. Let C

be any reference climatology. If the forecasts are where F and P represent the spatial means.

indexed by the time (i.e., t = i), this reference C The derivation of the Murphy“Epstein decom-

is the (constant) long-term mean so that C = E(P). position is formally carried out for the t = i

If the index refers to space (i.e., x = i), then C is case, but it can be done in the same way for the

x = i case. To accomplish this the correlation

the long-term mean ¬eld at the location x, that is,

C x = E(Px ). In either case, we use F = F ’ C skill score ρ F P must be replaced by the anomaly

18.3: The Murphy“Epstein Decomposition 401

correlation coef¬cient ρ F P and the E(·)-operator b)E F . In the special case of a = 0 and E F =

A

E P = 0 we ¬nd b = 1 and therefore that the

has to be replaced by the spatial averaging operator

· . Some of the terms in the decomposition vanish forecast is not only unconditionally unbiased but

in the t = i case but have been retained because also conditionally unbiased.

they are needed for the x = i case.

By simultaneously adding and subtracting

18.3.3 Forecasts of the Same Predictands at

E P and E F to the mean square error S F P we2

Different Times. In the t = i case, when Ft

¬nd, after some algebraic manipulation, that

is a series of forecasts of the same predictand Pt

(18.19) at various times t, C is taken as the climatology

= Var P + Var F

S2 F

P

of that predictand and thus C = E(P). Therefore,

2

’ 2Cov P , F + E P ’ E F . because E P = 0, we have

If we replace F with C in this formula we ¬nd B FC P = (18.23)

2

2

EF

S 2 = Var P + E P 2

σF

(18.20)

PC

ρ2 ’ ρP ’ ’

F

σP σP

P F

simply because C = 0. Finally, after some further

manipulation, we see that the Brier skill score may

and

be expressed as

S 2 = Var(P). (18.24)

A2 ’ B 2 ’ D 2 + E 2 PC

=

B FC P (18.21)

1 + E2 If F is unconditionally unbiased then E F is

zero and we ¬nd the Brier skill score is identical

where

to the proportion of explained variance R 2 P

σF F

A = ρP F , B = ρP F ’ (18.6). Even when this happens we still have (see

σP

equation(18.16))

E P ’E F EP

D= , and E = .

B FC P < ρ 2 .

σP σP (18.25)

P F

Decomposition (18.21) is the Murphy“Epstein Thus we see that, as a general rule, the correlation

decomposition of the Brier skill score. skill score overestimates the ˜true™ forecast skill.

The ¬rst term A2 in (18.21) is the correlation This is why the correlation skill should generally

skill score squared. be regarded as a measure of potential skill; it

To understand the second term B 2 we will only represents the actual skill if the forecast is

assume that P and F are jointly normal so that unbiased. In this case the Brier skill score, the

we can use (18.13) and write the expected value of squared correlation skill and the proportion of

P conditional on F = f as explained variance are equivalent.

E P |F = f = a + bf . (18.22)

18.3.4 Forecasts of Different Predictands at the

We see from (18.13) that ρ P F = (σ F /σ P )b so Same Time. In the x = i case the reference

that forecast is climatology C so that the unconditional

bias represents the error in predicting the spatial

2

B 2 = (b ’ 1)(σ F /σ P ) .

mean. This bias might be large if the forecast

This term vanishes only if b = 1, that is, region is small. Murphy and Epstein [285]

if the forecasts are not systematically biased.8 computed the relative contributions of the terms

Murphy and Epstein [285] call this term the in the Murphy“Epstein decomposition to the Brier

conditional bias because it re¬‚ects the extent to skill score for a series of medium range forecasts

which the mean observation E P (conditional prepared by the US National Meteorological

upon a forecast f ) re¬‚ects that forecast. Center (NMC) in the mid 1980s.

2 vanishes only if

The third term in (18.21) D

E F = E P . This term therefore represents 18.3.5 The Correlation Skill Score and

the unconditional bias of the forecast. If (18.22) the Mean Squared Error. Equation (18.19)

holds then E P = a + bE F , so that for provides a decomposition of the mean square

an unconditionally unbiased forecast a = (1 ’ error [286]. We will assume that the forecast F

is unconditionally unbiased (i.e., E F = E P )

8 Note that b = 1 does not imply σ

F = σ P but rather that

σ F < σ P if ρ P F < 1. so that the last term in (18.19) vanishes, and now

18: Forecast Quality Evaluation

402

18.3.6 Correlation Skill Score Thresholds at

which the Brier Skill Score Becomes Positive.

If we accept the notion that the Brier skill score

as the best indicator of the presence or absence

of skill relative to a reference forecast, we can

derive a threshold for the correlation skill score (in

the t = i case) and for the anomaly correlation

coef¬cient (in the x = i case) at which the

Brier score becomes positive [285]. To derive the

threshold we assume that F is an unconditionally

unbiased forecast of P, that E P = 0, and that