• Any m-dimensional vector y can be projected

transformations. This becomes important when

onto an EOF e of X by computing the

different variables such as precipitation and

inner product y, e . Vector y can then by

temperature are combined in a data vector. The k

approximated by y ≈ i=1 y, e i e i .

EOFs of such a random vector depend on the units

in which the variables are expressed.

• Where are the units? When we expand the

A special case of transformation (13.15) occurs

random vector X into EOFs

when X has already been transformed into EOF

coordinates using (13.10), k

X≈ ±i e i (13.19)

Z = ± = P † X. i=1

That is, L = P † . Using transformation (13.18), we

with ±i = X, e i , where do we place the

see that the ±-EOFs are

units of X on the right side of approximation

P ± = LP = P † P = I. (13.19)? Formally the answer is that the

coef¬cients carry the units while the patterns

Thus, in the new coordinates the EOFs are unit are dimensionless. However, in practice

vectors. This fact may be used to test EOF approximation (13.19) is often replaced by

programs.

k

+

±i+ e i

X≈ (13.20)

13.1.11 Further Aspects. Some other aspects of

i=1

EOFs are worth mentioning.

with re-normalized coef¬cients

• Empirical Orthogonal Functions may be

generalized to continuous functions, in which

1

±i+ = √ ±i

case they are known as Karhunen-Lo` ve e (13.21)

»i

functions. The standard inner product ·, · is

replaced by an integral, and the eigenvalue

and patterns

problem is no longer a matrix problem but

an operator problem. (See, e.g., North et al.

+

= »i e i

ei

[296].) (13.22)

13.2: Estimation of Empirical Orthogonal Functions 299

so that Var ±i+ = 1. The re-normalized • After [13.1.3] the EOFs form an orthonormal

set of vectors that is most ef¬cient in

pattern then carries the units of X, and

representing the variance of X (13.3). Thus

represents a ˜typical™ anomaly pattern if we

regard ±i+ = ±1 as a ˜typical event™. another reasonable approach is to use a set

of orthonormal vectors that represent as much

The decomposition of the local variance, as

of the sample variance of the ¬nite sample as

given by equation (13.7), takes a particularly

possible.

simple form with this normalization, namely

The two approaches are equivalent and lead to the

m

following.

Var(Xk ) = |ek |2 .

i+

(13.23)

i=1

13.2.3 Theorem. Let Σ = n n (x j ’ 1

j=1

+

Note that the coef¬cient ±i can be expressed µ)(x j ’ µ)

† , where † indicates the conjugate

transpose and µ = n n x j , derived from a

1

as j=1

sample {x1 , . . . , xn } be the estimated covariance

1 +

(13.24) matrix of n realizations of X. Let »1 ≥ »2 ≥ · · · ≥

±i+ = X, e i .

»i »m be the eigenvalues of Σ and let e 1 , . . . , e m be

corresponding eigenvectors of unit length. Since Σ

13.2 Estimation of Empirical is Hermitian, the eigenvalues are non-negative and

the eigenvectors are orthogonal.

Orthogonal Functions

(i) The k eigenvectors that correspond to

Outline. After having de¬ned the »1 , . . . , »k minimize

13.2.1

eigenvalues and EOFs of random vector X as n k 2

parameters that characterize its covariance matrix, = xj ’ xj, e i e i . (13.25)

k

the question naturally arises as to how to estimate j=1 i=1

these parameters from sample {x1 , . . . , xn } of

k

realizations of X. It turns out that useful estimators

(ii) k = Var (X) ’ »j. (13.26)

may be de¬ned by replacing the covariance matrix

j=1

Σ with the sample covariance matrix Σ and

by replacing the expectation operator E(·) with m

averaging over the sample. An important little (iii) Var (X) = »j, (13.27)

j=1

trick for reducing the amount of calculation

when the sample size n is less than the where Var (X) = tr(Σ).

dimension of X (as is often true) is presented in

[13.2.5]. A computational alternative to solving 13.2.4 The Estimated Covariance Matrix Σ.

the eigenproblem is to perform a singular value The covariance between the jth and kth elements

decomposition [13.2.8]. of X is estimated by

n

1

σ jk = (x ji ’ x j )(xki ’ xk ),

13.2.2 Strategies for Estimating EOFs. The

n

eigenvalues and EOFs are parameters that charac- i=1

terize the covariance matrix of a random vector where x ji and xki are the jth and kth elements

X. In practice, the distribution of X, and thus of xi . This sum of products can be expressed as

the covariance matrix Σ and its eigenvalues and a quadratic form:15

eigenvectors, is unknown. They must therefore be

1 1 1

estimated from a ¬nite sample {x1 , . . . , xn }. Σ = X (I ’ J )(I ’ J )X † (13.28)

n n n

There are two reasonable approaches for

where X is the data matrix16

estimation.

«

x11 x12 . . . x1n

• Since the eigenvalues and EOFs characterize ¬ x21 x22 . . . x2n ·

¬ ·

the covariance matrix Σ of X, one reasonable X = ¬ . . ·,

. .. (13.29)

. . .

. .

. .

approach is to estimate the covariance matrix

xm1 xm2 . . . xmn

and then estimate the eigenvalues »i and

eigenvectors e i with the eigenvalues » j 15 A quadratic form is a matrix product of the form AA† or

j of the estimated

and the eigenvectors e A† A.

16 Sometimes also called the design matrix.

covariance matrix Σ.

13: Empirical Orthogonal Functions

300

• When X is multivariate normal, the distri-