of treatment combinations is applied to the lend themselves well to analysis using methods

simulations conducted on each computer. The appropriate for designed experiments, and partly

effects of some treatment combinations will be because the cost of properly designed climate

confounded with the block effect in a fractional model experiments was prohibitively high in the

design. The art of designing a fractional factorial past, although this situation is now changing.

experiment depends primarily on making informed We will describe applications of ANOVA

choices about the effects that are likely to be small to the analysis of interannual variability in

enough to be safely confounded with the block an experiment consisting of multiple AMIP1

effect.

1 The AMIP (Atmospheric Model Intercomparison Project)

encompasses most of the world™s climate modelling groups

9.1.5 What is ANOVA and How is it (see Gates [137] for a description of the project and its

Different from Regression Analysis? There is goals). All participants ran a standard 10-year atmospheric

simulation imposing observed 1979“88 monthly mean sea-

a very strong connection between the experimental

surface temperatures and sea-ice extents at the lower boundary.

design and the subsequent analysis of variance Several groups, such as the Canadian Centre for Climate

used to analyse the data generated by the Modelling and Analysis, ran multiple AMIP simulations from

experiment. Formally, the models ¬tted using randomly selected initial conditions.

9.2: One Way Analysis of Variance 173

simulations conducted with the CCC GCMII We complete this section by brie¬‚y introducing

(see McFarlane et al. [270] for a description the CCCma multiple AMIP simulations.

of CCC GCMII; and see Zwiers [444, 449]

and Wang and Zwiers [414] for analysis of the

9.1.8 Example: Multiple AMIP Simulations.

AMIP experiments).2 We will also describe an

AMIP is a Level 2 model intercomparison as

application of so-called space ¬lling experimental

de¬ned by the WGNE (Working Group on Numer-

designs to the problem of parameter speci¬cation

ical Experimentation). The more primitive Level

in a basin scale ocean model (Gough and Welch

1 intercomparisons apply common diagnostics to

[145]).

climate simulations as available. At Level 2, sim-

ulations are conducted under standard conditions,

common diagnostics are computed, and validation

9.1.7 Outline. The models and methods used

is made against a common data set. Level 3 encom-

in one way analysis of variance are described in

passes Level 2 and also requires that models use a

Section 9.2. These are methods suitable for use

common resolution and common subroutines.

in simple experiments that intercompare the mean

An AMIP simulation (see Gates [137]) is a

responses to a number of different treatments,

10-year simulation conducted with an atmospheric

or levels of one treatment. One way ANOVA

climate model in which the monthly mean sea-

methods are also appropriate when it is necessary

surface temperatures and sea-ice boundaries are

to intercompare the means of two or more samples.

prescribed to follow the January 1979 to December

Both ¬xed and random effects models are

1988 observations.

discussed in Section 9.2. A ¬xed effects model

The CCCma AMIP simulations were conducted

describes the effect of a treatment as a change

with a spectral model ([270] and [52]) that operates

in the mean of the response variable. This is a

at ˜T32™ horizontal resolution (approximately

deterministic response to a treatment that can be

3.75—¦ — 3.75—¦ ), has 10 layers in the vertical,

replicated from one realization of the experiment

and a 20-minute time step. The ¬rst simulation,

to the next. A random effects model describes the

conducted on a Cray XMP, was initiated from

effect of the treatment with a random variable, a

1 January 1979 FGGE (First GARP Global

form of response that can not be replicated from

Experiment [44]) conditions. Five additional

one experiment to the next. Methods of inference

AMIP simulations, performed on a NEC SX/3,

are discussed for both types of one way model.

were started from previously simulated 1 January

The relationship between ANOVA and regression

model states. These initial states were selected

is described at the end of Section 9.2.

from the control run at two-year intervals. Analysis

The models and methods used in two way of the AMIP simulations begins in June of the ¬rst

analysis of variance are described in Section 9.3. simulated year. That is, the ¬rst ¬ve months of

These models are used to analyse experiments con- each simulation is regarded as a ˜spin-up™ period

ducted with randomized complete block designs during which the model forgets about its initial

or completely randomized designs in which two conditions, and slow (primarily land surface)

different kinds of treatment have been applied. The processes equilibrate with the imposed lower

discussion in this section is limited to ¬xed effects boundary conditions. Because the atmosphere

models. forgets its initial state very quickly, the effect of

The Canadian Centre for Climate Modelling and selecting different initial conditions is basically to

Analysis (CCCma) AMIP experiment is used as select independent realizations of the simulated

a working example throughout Sections 9.2 and climate™s path through its phase space. For all

9.3. This experiment is analysed in more detail in intents and purposes, these six simulations can be

Section 9.4 with a two way model containing a regarded as having been initiated from randomly

mixture of ¬xed and random effects. An additional selected initial states.

example is discussed in Section 9.5, where we

describe Gough and Welch™s [145] use of space

¬lling designs to study the sensitivity of a basin 9.2 One Way Analysis of Variance

scale ocean GCM to its parameter settings.

9.2.1 The One Way ANOVA Model. Suppose

2 Several other analyses of ensembles of climate variability

that an experiment has been conducted that results

have recently appeared in the climate literature, including

in J samples of size n represented by random

Rowell [336], Rowell and Zwiers [337], Kumar et al. [232],

Folland and Rowell [123], Stern and Miyakoda [358], and variables Yi j , for i = 1, . . . , n and j = 1, . . . , J .

The subscript j identi¬es the sample, and the

Anderson and Stern [11].

9: Analysis of Variance

174

result of a different treatment (the speci¬ed sea-

subscript i identi¬es the element of the sample.

surface temperature and sea-ice regime) applied

Assume that the sampling is done in such a way

to a different experimental unit (a year in a

that all random variables are independent, normal,

simulation). Because the AMIP simulations are

and have the same variance. Also assume that the

conducted with an atmospheric model, it seems

means are constant within samples. That is, in

reasonable to assume that consecutive mean DJF

sample j

states simulated by the model are approximately

E Yi j = µ j independent of each other. Thus a simulation can

be thought of as the outcome of a completely

for all i = 1, . . . , n, or equivalently that

randomized experiment in which each of the J =

E Yi j = µ + a j 9 treatments is applied once. Each simulation in an

ensemble of AMIP simulations can be considered

for all i = 1, . . . , n, where µ is the overall mean a replication of the nine treatment experiment.

given by Because the AMIP simulations in the six member

CCCma ensemble were started from randomly

J

1 selected initial conditions, the replications can also

µ= µj,

J j=1 be assumed to be independent of one another.

Thus it appears that seasonal mean data from the

and a j is the difference CCCma AMIP experiment can be analysed using

a one way ANOVA appropriate for data obtained

aj = µj ’ µ from a replicated completely randomized design

between the expectation of Yi j and the overall with J = 9 treatments and n = 6 replicates.

mean. The coef¬cients a j are often called

treatment effects. 9.2.3 Partitioning Variance into Treatment

An appropriate statistical model for this type of and Error Components. In regression analysis

data is (see Chapter 8) we started with a model such

as (9.1), developed parameter estimators, and

Yi j = µ + a j + Ei j , (9.1) slowly proceeded towards an analysis of variance

where the errors Ei j are iid zero mean normal that partitioned the total sum of squares into