a basic problem is to test H0 : µcontr ol =

µex periment , that is, the null hypothesis that

6.5 Multivariate Problems the changes do not affect the mean state of

the simulated climate. Examples are given in

6.5.0 Overview. The spatial covariance charac- Section 7.2, where we compare two simulated

teristics of the climate system have a profound ef- climates, and Section 7.1, where a simulation is

fect on the analysis of just about any climate quan- compared with the observed climate.

tity that is distributed in space. Subsection 6.5.1

describes a prototypical problem in which we

6.5.2 The Effect of Spatial Correlation

might want to use a multivariate test or multiple

on Multiple Univariate Tests. The simplest

univariate tests. In both cases it is necessary to be

approach to comparing the mean states of the

aware of the relevant spatial covariance structure to

climates simulated in a pair of GCM experiments

interpret the results correctly. In subsection 6.5.2

is to conduct a univariate difference of means test

we discuss the interpretation of multiple univariate

[6.6.1] at every grid point. This is called the local

tests, conducted, for example, at each grid point

test approach because a local null hypothesis is

of a GCM. Another approach is to conduct a

tested at each grid point.

multivariate test on the entire ¬eld [6.5.3].

There can, however, be dif¬culty with the global

However, we often have information that can

interpretation of the results of a collection of local

be used to sharpen the alternative hypothesis

tests.

and therefore improve the ef¬ciency of the

Assume, for the moment, that the treatment

multivariate test. The impact of ignoring this

applied to the experimental simulation has no

information is discussed in subsection 6.5.4. The

effect on the simulated mean state. Then the local

prior information is expressed as a set of ˜guess

equality of means hypothesis is true everywhere.

patterns™ [6.5.6] and it is used by projecting the

The global null hypothesis that corresponds to the

observed ¬elds onto the space spanned by the

collection of local hypotheses is that the mean

guess patterns, therefore reducing the dimension

¬elds are equal. Now suppose that the local null

of the multivariate testing problem. There are

hypothesis is tested at each of m grid points at

also practical considerations that motivate the

the 5% signi¬cance level. Under the global null

dimension reduction [6.5.5]. Even after dimension

hypothesis we expect that roughly 5% of the local

reduction, it may be possible to further increase the

test decisions will be reject decisions. Each test is

sensitivity of the test by searching for a pattern

analogous to the toss of a fair 20-sided die that has

in the space spanned by the guess patterns that

19 black faces and 1 white face. The white face

optimizes the signal-to-noise ratio [6.5.7]. Finally,

will come up 5% of the time on average, but the

it is sometimes possible to develop a hierarchy of

proportion of white faces observed varies between

nested sets of guess patterns, and this inevitably

replications of an m-roll die-rolling experiment. In

leads to a step-wise testing procedure [6.5.8].

the same way there is variability in the number

of reject decisions that will be made in any one

6.5.1 GCM Experiments. Analyses of GCM replication of the climate simulation experiment.

experiments are usually multivariate in nature If decisions made at adjacent grid points are

simply because such models produce ¬elds, such independent of each other, then the 20-sided die

as monthly mean 500 hPa height ¬elds, as model can be used to predict the probability

output. GCM experiments are either sensitivity distribution of the number of reject decisions under

experiments or simulations of the present or a past the global null hypothesis. In fact, the probability

climate of Earth or another planet. of making reject decisions at k or more grid

A typical sensitivity study will consist of two points is given by the binomial distribution that

climate simulations. One run, labelled the control has cumulative distribution function Fm (k) =

m

i=k B(m, 5%)(i). For example, if the local test is

run is conducted under ˜normal™ conditions, and

the other, the experimental run, is conducted with, conducted at m = 768 grid points, the probability

6.5: Multivariate Problems 109

of obtaining more than 48 local rejections under

the global null hypothesis is 5%. Thus, in

this example with independent grid points, a

reasonable global test is to reject the global null

hypothesis if the local reject decision is made at

the 5%-signi¬cance level at 49 or more grid points.

In the real world, decisions made at adjacent

grid points are not independent because meteo-

rological ¬elds are spatially correlated. Thus the

binomial distribution does not provide the appro-

priate null distribution for the number of local

reject decisions.

This was demonstrated in an experiment

in which seven independent integrations were

conducted with a simpli¬ed GCM [384]. Each

integration produced one monthly mean ¬eld. The

runs were identical except for small variations in

their initial conditions. Because small-scale errors

Figure 6.10: The spatial distribution of false

quickly cascade to all resolved spatial scales in

rejections of local null hypotheses in a Monte

AGCMs, this produced a set of seven independent

Carlo experiment [384].

realizations of the same geophysical process.

The set of K = 7 simulated monthly mean ¬elds

was arbitrarily split up into two sets, the ¬rst i and

the last K ’i. The ¬rst set was used to estimate the to ¬nd the correct distribution for the number of

statistical parameters of the simulated geophysical false local rejections under the null hypothesis.

process. Each realization in the second set of ¬elds Livezey and Chen [257] have suggested methods

was tested at each grid point to see if it belonged that are widely used [6.8.1“3]. Another is to use

to the population represented by the ¬rst set. The multivariate techniques such as the Hotelling test

local rejection rate was subsequently calculated. or a permutation test [6.6.4“7].

On average, the reject decision was made 5.2% of The multivariate method induces strategic and

the time, nearly the nominal 5% rate speci¬ed by technical problems related to the dimension of the

the null hypothesis. However, there are instances observed climate ¬elds. We discuss these in the

in which the rate of incorrect decision was as next two subsections.

high as 10%. We would expect reject rates to vary

between 3.4% and 6.6% in the absence of spatial

correlation. Thus it appears that spatial correlation

6.5.4 Strategic Problems. The strategic prob-

affects the variability of the proportion of reject

lem arises because the signal induced by the

decisions.

experimental ˜treatment™ may not be present in

The effect of spatial correlation is illustrated in

all components of the observed ¬eld. Often it

Figure 6.10 where we see one ¬eld of erroneous

resides in a low-dimensional subspace spanned

rejections. Note that erroneous rejections do

by only a few vectors. The total m-dimensional

not occur at isolated points. Rather, the spatial

space that contains the climate realizations may be

correlation structure results in pools of reject

represented as a sum of two spaces S and N

decisions. On average these pools will occupy

with dimensions m S and m N respectively, where

5% of the map. Map to map variation in the

m S + m N = m. The signal is con¬ned to S .

area covered by the pools depends on the average

Both S and N contain variations due to random

size of the pools, which in turn is determined by

¬‚uctuations. A multivariate test of the equality of

the spatial correlation structure of the ¬eld. The

means hypothesis (i.e., the signal is absent) will be

map to map variation is smallest when the ˜pools™

more powerful if is restricted to S because the

degenerate to isolated points that are not spatially

signal-to-noise ratio in S is greater than it is in

correlated.