g(X) = (X(1|1) , . . . , X(n|n) )T .

has a probability density function that is constant

Note that the elements of g(X) are no longer inde- inside the interval and zero outside. Such a density

pendent or identically distributed; their marginal function is given by

distributions (see [2.8.3]) are complicated func-

1/(b ’ a) for all x ∈ (a, b)

f X (x) =

tions of the distribution of X. The random variables

0 elsewhere,

X( j|n) for j = 1, . . . , n are called order statistics.

L-moments are de¬ned as the expectations of and the cumulative distribution function is given

linear combinations of these order statistics. by

The ¬rst three L-moments are de¬ned as ±

for x ¤ a

0

»(1) = E X(1|1) (x ’ a)/(b ’ a) for x ∈ (a, b)

FX (x) =

for x ≥ b.

1 1

»(2) = E X(2|2) ’ X(1|2)

2

We use the shorthand X ∼ U(a, b) to indicate that

1

»(3) = E X(3|3) ’ 2X(2|3) + X(1|3) . (2.20) X has a uniform distribution.

3

It is readily shown that the mean, variance,

The general kth L-moment is given by skewness, and kurtosis of a U(a, b) random

variable are given by

1 k’1 j k’1

(k)

» = (’1) E X(k’ j|k) .

j 1

k j=0

E U(a, b) = (a + b)

2

(2.21)

1

Var U(a, b) = (b ’ a)2

12

Thus, the ¬rst L-moment is the expected

γ1 U(a, b) = 0

smallest value in a sample of one. Since there

is only one value in such a sample, the ¬rst L- γ2 U(a, b) = ’1.2 .

moment is equal to the conventional ¬rst moment.

The second L-moment is the expected absolute Thus, the uniform distribution is symmetric

difference between any two realizations (note that (skewness = 0) and less peaked than a normal

2: Probability Theory

34

distribution (kurtosis < 0). The L-moments are

[183]:

1

»(1) = (a + b)

2

1

»(2) = (b ’ a)

6

γ1L =0

γ2L = 0.

2.7.2 Probability and Likelihood. The uni-

form distribution illustrates very clearly that a

probability density is not a probability. When the Figure 2.2: Probability density functions for

distribution is de¬ned on an interval of length less normal random variables with mean 0 and

variances 1 and 9 (standard deviations σ = 1 and

than 1, the density is uniformly greater than 1

throughout the interval, even though probabilities 3 respectively).

are never greater than 1. Only integrated density

functions provide probabilities.

Nevertheless, the density function describes the about the mean, values near the mean are more

relative chances of observing speci¬c events. In likely than values elsewhere, and the spread

particular, when f X (x1 ) > f X (x2 ) it is more likely

of the distribution depends upon the variance.

that we will observe values of X near x1 than near Larger variance is associated with greater spread.

x2 . Therefore we call the values of the density Changes in the mean shift the density to the left or

function likelihoods. For the uniform distribution, right on the real line.

all values of X in the range (a, b), including the Also, note that the likelihood of obtaining a

mean, are equally likely. This is not true in the large realization of a normal random variable

other distributions of continuous random variables. falls off quickly as the distance from the mean

increases. Observations more than 1.96σ from the

2.7.3 The Normal Distribution. The distribu- mean occur only 5% of the time, and observations

tion most frequently encountered in meteorology more than 2.33σ from the mean occur only 1% of

and climatology is the normal distribution. Many the time.

variables studied in climatology are averages or The mean, variance, skewness, and kurtosis of a

integrated quantities of some type. The law of normal random variable X are:

large numbers, or Central Limit Theorem [2.7.5],

states (under fairly broad regularity conditions) E(X) = µ

that random variables of this type are nearly

normally distributed regardless of the distribution Var(X) = σ

2

γ1 = 0

of the variables that are averaged or integrated.

γ2 = 0,

The form of the normal distribution is entirely

determined by the mean and the variance. Thus,

we write X ∼ N (µ, σ 2 ) to indicate that X has a and the L-moments are:

normal distribution with parameters µ and σ 2 .

In the climatological literature, the normal »(1) = µ

distribution is also often referred to as the (2)

Gaussian distribution, after C.F. Gauss who » = σ/π

γ1L = 0

introduced the distribution some 200 years ago.

The normal density function is given by γ L = 0.1226 .

2

(x’µ)2

1 ’

f N (x) = √ for all x ∈ R.

2σ 2

e The cumulative distribution function cannot be

2π σ

given explicitly because the analytical form of

(2.25) ’t 2 /2 dt does not exist. But the cumulative

x

’∞ e

The density functions of normal random distribution function is related in a simple manner

variables with different variances are illustrated in to the error function, erf, which is available from

Figure 2.2. Note that the distribution is symmetric subroutine libraries (for example in the Numerical

2.7: Example of Continuous Random Variables 35

in this book. This function, which is tabulated in

Appendix D, can also be evaluated by numerical

integration or by using simple approximations. For

most purposes, the approximation

1 + sgn(x) 1 ’ e’2x

2 /π

FN (x) ≈ /2

(2.27)

(where sgn(x) = 1 if x > 0 and sgn(x) = ’1 if

x < 0) is adequate and eliminates the use of tables.

2.7.5 The Central Limit Theorem. The

Figure 2.3: Cumulative distribution functions of Central Limit Theorem is of fundamental

normal random variables for µ = 0 and σ = 1 importance for statistics because it establishes the

dominant role of the normal distribution.

and 3.

If Xk , k = 1, 2, . . ., is an in¬nite series of

independent and identically distributed random

Recipes [322]). Speci¬cally,

variables with E(Xk ) = µ and Var(Xk ) = σ 2 ,

then the average n n Xi is asymptotically

1

(t’µ)2

x

1 ’ k=1

FN (x) = √ e 2σ 2 dt normally distributed. That is,

2π σ ’∞