SSB/(J ’ 1)

F= block effect. The result is that the test for the

SSE/((I ’ 1)(J ’ 1))

sea-surface temperature effect on 850 hPa DJF

with F(J ’ 1, (I ’ 1)(J ’ 1)) critical values (see temperature rejects the null hypothesis at the 10%

Appendix G). signi¬cance level over a slightly larger area (66.1%

One possible reason for testing for a block effect of the globe).

is to determine whether or not the block sum of The F-ratios for the block effect on 850 hPa DJF

squares can be pooled with the sum of squared temperature are shown in Figure 9.4. The F-ratio

errors. If this can be done, that is, if (9.17) is not exceeds the 10% critical value for F(5, 40) over

rejected, then the between blocks variation can be about 13.1% of the globe. Previous experience

used to improve the estimate of error variance and with ¬eld signi¬cance tests (see Section 6.8)

hence increase the power of the test for treatment conducted with ¬elds with comparable spatial

effects. In this case we compute covariance structure suggests that this rate is not

signi¬cantly greater than 10%. However, the same

SSA/(I ’ 1)

F= test conducted with 500 hPa DJF geopotential

(SSB + SSE)/(I (J ’ 1))

(not shown) resulted in a rejection rate of 23%,

and compare with critical values from which is likely ¬eld signi¬cant. Therefore, while

F(I ’ 1, I (J ’ 1)). It is easily shown that a block, or run, effect is dif¬cult to detect in

this test is equivalent to the test for treatment lower tropospheric temperature, it appears to be

effects in the one way model with ¬xed effects detectable in the integrated temperature of the

[9.2.4]. lower half of the atmosphere.

The interaction terms in (9.15) are confounded The CCCma experiments were actually con-

with error when the experiment is not replicated. ducted on two computers. One 10-year simula-

Then the mean sum of squared errors, SSE/((I ’ tion was conducted on a Cray-XMP while the

1)(J ’ 1)), is in¬‚ated by the interaction terms; it remaining ¬ve were conducted on a NEC SX/3.

9.3: Two Way Analysis of Variance 183

and comparing F with critical values from

F(J ’ 2, (I ’ 1)(J ’ 1)). The null hypothesis

that there is additional inter-run variation not

explained by the computer change is rejected at the

10% signi¬cance level over 9.4% of the globe.

The run effect is observed much more strongly

in June, July, August (JJA) 500 hPa geopotential

for which (9.18) is rejected over 52% of the globe

(primarily in the tropics).

The differences between the Cray and NEC

simulations were not primarily due to the

differences between machines (see Zwiers [449]).5

Figure 9.4: The natural log of the F-ratios for

the block or run effect in the CCCma AMIP It turns out, however, that the change in machine

experiment. Each contour indicates a doubling of type coincided with a change in the source

the F-ratio. The shading indicates ratios that are of initialization data. CCCma™s initialization

signi¬cantly greater than 1 at the 10% signi¬cance procedure diagnoses the atmospheric mass from

level. the initialization data. The model subsequently

conserves that mass for the duration of the

simulation. The resulting atmospheric mass for the

We label the Cray ˜block™ as block number 1. The

Cray simulation is equivalent to a global mean

hypothesis that the block effect for the Cray was

surface pressure of 985.01 hPa. In contrast, the

equal to that for the NEC was tested with the

masses diagnosed from the initial conditions used

contrast c = (1, ’0.2, ’0.2, ’0.2, ’0.2, ’0.2).

for the NEC simulations varied between 984.55

Speci¬cally, the null hypothesis

and 984.58 hPa. This difference between the Cray

and NEC simulations, approximately 0.44 hPa,

J

cjbj = 0

H0 : (9.18) corresponds to a change in 500 hPa geopotential

height in the tropics of about 3.5 m. The large,

j=1

and unexpected, block effect described above is

was tested against the alternative that the contrast

primarily the result of the change in the source

is nonzero. The contrast was estimated by

of initialization data. This example illustrates that

computing

it is dif¬cult to design an experiment so that

it excludes unwanted external variability, since

J

wc = c j Y—¦ j . such variability often arrives from unanticipated

sources.

j=1

Under (9.18), the squared contrast has expectation

9.3.4 Two Way Model with Interaction. We

2 now brie¬‚y consider the two way ¬xed effects

σE

J J

2

E wc = + c2 .

2 model with interaction given by (9.15) in the

cjbj j

I case in which each treatment or treatment/block

j=1 j=1

combination is replicated n times. The calculation

Therefore (9.18) can be tested by comparing

of the variance components is easily extended

to the case in which each combination is

wc

2

F= not replicated equally. However, the tests for

SSE J 2

j=1 c j

I (I ’1)(J ’1) treatment, block, and interaction are then only

approximate (see [9.2.10]) if the corresponding

with critical values from F(1, (I ’ 1)(J ’ 1)).

sum of squares has more than one df.

We obtained a rejection rate of 18.1% when (9.18)

In the setup of (9.15) the total sum of squares is

was tested in DJF 850 hPa temperature at the 10%

partitioned into four independent components for

signi¬cance level (not shown).

treatment, block, interaction, and error, as follows:

We can test whether there is signi¬cant inter-run

SST = SSA + SSB + SSI + SSE,

variation that is orthogonal to contrast (9.18) by (9.19)

computing 5 Differences in the way in which the two machines

represented ¬‚oating point numbers did lead to surface elevation

SSB’I w c / J c2

2

j=1 j changes at three locations on the latitude row just north of the

J ’2

F= equator, but these were not judged to be the cause of large-scale

SSE effects in the tropical climate.

(I ’1)(J ’1)

9: Analysis of Variance

184

of σE that has n IJ ’ (I + J ’ 1) df instead of

2

where

(n ’ 1)IJ df is given by

I J n

SST = (Yi jl ’ Y—¦—¦—¦ )2 , (9.20) SSI + SSE

ˆ2

σE =

ˆ .

i=1 j=1 l=1

n IJ ’ (I + J ’ 1)

I The effect of pooling interaction and sum of

SSA = n J (Yi—¦—¦ ’ Y—¦—¦—¦ )2 , (9.21) squared errors (when it can be done) is particularly

i=1 dramatic if the number of replicates is small.

J

SSB = n I (Y—¦ j—¦ ’ Y—¦—¦—¦ )2 , (9.22) 9.4 Two Way ANOVA with Mixed

j=1

Effects of the CCCma AMIP

Experiment

I J

SSI = n (Yi j—¦ ’ Yi—¦—¦ ’ Y—¦ j—¦ + Y—¦—¦—¦ )2 , We continue the analysis of [9.3.3] by introducing

i=1 j=1

a two way model with interaction terms and a

(9.23) mixture of ¬xed and random effects.

The data we use are monthly means of 850 hPa