approaches a normal distribution as the sample

µ0 , in dimensionless units, is

size increases.

¯

√ X ’ µ0

The effect of increasing sample size on the

Z= n .

distribution of the sample mean is illustrated in σ

Figure 4.1. The distribution becomes increasingly

compact as the sample size increases. Conse- Random variable Z has mean zero and unit

quently the true population mean, µ, becomes variance regardless of the scale on which the

observations are made. It is normally distributed

better known as sample size increases.

when random variables Xi are normal. When this

is not true, the Central Limit Theorem [2.7.5]

states that the distribution of Z will approach the

4.3.2 The Sample Variance. Another example standard normal distribution as the sample size

of a relatively simple statistic is the sample grows large.

variance, which is given by When that variance is not known, we can

estimate it with S 2 and compute the t statistic or,

n as it is also often called, Student™s t statistic

1 ¯

S= (Xi ’ X)2

2

(4.5)

n ’ 1 i=1 ¯

√ X ’ µ0

T= n .

n

S

1 ¯

= Xi ’ n X2 .

2

n ’ 1 i=1 Again, we have a measure that is independent of

the scale of measurement and it can be shown that

the asymptotic distribution is normal with unit var-

By using (2.16) and (2.17) it can be shown that

iance. When samples are ¬nite and consist of in-

dependent, identically distributed normal random

ES 2 = σ 2. (4.6) variables with mean µ0 , T has the t distribution

12 The χ 2 (k) distribution is discussed in [2.7.8]. Figure 2.5

11 Independence of the X s is not a necessary condition

i

shows the χ 2 (k) distribution for four different degrees of

for obtaining convergence results such as the Central Limit

freedom k.

Theorem. Similar results can often be obtained when the

13 This is the principle of invariance. Statistical methods that

Xi s are dependent on one another, although in this case the

asymptotic variance of X will only be proportional rather than are not invariant under transformations of scale should not be

equal to σ 2 /n. The constant of proportionality depends upon trusted because users can manipulate the inferences made with

the nature of the dependence. such methods by using a suitable transform.

4: Concepts in Statistical Inference

78

with n ’ 1 degrees of freedom (see [2.7.9]). One 4.3.4 The F-ratio. Suppose now that we have

two collections of independent and identically

way to show this is to factor T as

distributed random variables X1 , . . . , Xn X and

¯ ’ µ0

√X Y1 , . . . , Yn Y representing two random samples of

T= n

size n X and n Y respectively. A natural way to

S

√

¯ compare the dispersion of the two samples is to

n(X ’ µ0 )/σ

= compute

1/2

(n ’ 1)S 2 /σ 2 /(n ’ 1)

2

SX

Z

F= 2,

=√ . (4.8)

Y/(n ’ 1) SY

2

where S X is the sample variance of the X sample

Here

2

and SY is the sample variance of the Y sample.

¯

√ X ’ µ0 When both samples consist of independent and

Z= ∼ N (0, 1)

n

σ identically distributed normal random variables,

(n ’ 1)S 2 and the random variables in one sample are

Y= ∼ χ 2 (n ’ 1), independent of those in the other, the random

σ 2

variable (σY /σ X )2 F is independent of both scales

of observation and F ∼ F(n X ’ 1, n Y ’ 1) (see

and Y is independent of Z. This exactly character-

izes a t distributed random variable (see [2.7.9]). T

[2.7.10]). This is shown by factoring F so that

has zero mean and variance n’1 . it can be expressed as a ratio of independent χ 2

n’3

random variables, each divided by its own degrees

Figure 2.6 shows that the t distribution is

of freedom. In fact, by (2.29), we have

slightly wider than the standard normal distribu-

tion and that it tends towards the normal dis-

tribution as sample size increases. Indeed, the F = (n X ’ 1)S X /σ X /(n X ’ 1)

2 2

(n Y ’ 1)SY /σY /(n Y ’ 1)

2 2

two are essentially the same for samples of size

n > 30. The extra width in the small sample χ X /(n X ’ 1)

= ,

¯

case comes about because the distance between X χY /(n Y ’ 1)

and µ0 is measured in units of estimated rather

than known standard deviations. The additional with χ X ∼ χ 2 (n X ’ 1) and χY ∼ χ 2 (n Y ’ 1).

variability induced by this estimate is re¬‚ected in Several examples of the F distribution are dis-

the slightly wider distribution. played in Figure 2.7.

5 Estimation

5.1 General of the parameters they are estimating, their scatter

decreases with increasing sample size, and their

In Chapter 4 we describe some of the general scatter is related to the scatter within the sample.

concepts of statistical inference, including the For the moment, estimators are mere functions

basic ideas underlying estimation and hypothesis of the sample without any qualitative properties.

testing. Our purpose here is to discuss estimation The art is to ¬nd good estimators that yield

in more detail, while hypothesis testing is estimates in a speci¬ed neighbourhood of the true

addressed further in Chapter 6. value with some known likelihood. The objective

of estimation theory is to offer concepts and

measures useful for evaluating the performance of

5.1.1 The Art of Estimation. We stated

estimators.

in Chapter 4 that statistical inference is the

Because estimators are random variables, they

process of extracting information from data about

are subject to sampling variability. An estimator

the processes underlying the observations. For

can not be right or wrong, but some estimators

example, suppose we have n realizations xi

are better than others. Examples of admittedly silly

of a random variable X. How can we use

estimators of the mean µ and the variance σ 2 are

these realizations to make inferences about the

distribution of X? µ s = X1

The ¬rst step is to adopt some kind of statistical

σ s2 = (X1 ’ X2 )2 /2.

model that describes how the sample {x1 , . . . , xn }

was obtained. It is often possible to use the Note that µs has n times the variance of estimator

˜standard normal setup™ introduced in Section 4.3. (4.2), and σ s2 has n ’ 1 times the variance of

It represents the sample as a collection of n estimator (4.5).

independent and identically distributed normal