tracers.

Xt , the resulting process Yt = Xt + cos(ωt)

The movement of a particle, perhaps emitted

is cyclo-stationary and has distribution function

from a smoke stack, is determined by the deter-

FYt (y) = FX (y ’ cos(ωt)).

ministic ¬‚ow U and many small unpredictable

A time series that exhibits both a trend and displacements. If the particle is located at R(t) at

a cyclo-stationary component is the famous CO2 time t, then its location at time t + 1 is given by

concentration curve measured at Mauna Loa in R(t + 1) = R(t) + U + Zt , where Zt represents

Hawaii. A segment of this series is shown in white noise, and its location at time t + l is given

Figure 10.4. Note that the trend is not strictly by R(t + l) = R(t) + lU + t+l’1 Zs . If many

s=t

linear; both the rate of change and the amplitude of particles are ˜emitted™ and transported this way,

the annual cycle increase with time. The maximum the time evolution of the concentration may be

CO2 concentrations occur during northern winter, modelled in three dimensions.

and the minima during northern summer. The result of such a simulation is shown in

The time series displayed in Figure 10.3 are Figure 10.6. The left hand panel displays a 24-hour

approximately stationary with means near zero forecast of the 1000 hPa height ¬eld over Western

and nearly equal variances. A scatter diagram Europe. Note the cyclonic around the low over the

illustrating the joint variation of the two time coast of Norway. A pollutant, SO2 , was injected

series in units of standard deviations is plotted into the simulated atmosphere at a constant rate

in Figure 10.5. The time series appear to be from a point source in east England. The right hand

jointly normal. In particular, note that the points panel displays the simulated SO2 distribution at

are scattered symmetrically about the origin with the end of the 24-hour period. Evidence of both

maximum density near the origin. deterministic advection processes and random

diffusive processes can be seen.

10.2.8 Example: A Random Walk and the

Long-range Transport of Pollutants. If Zt is 10.2.9 Ergodicity. Unfortunately, stationarity,

or weak stationarity, alone is not enough to

white noise, then Xt , given by

ensure that the moments of a process can be

estimated from a single time series. Koopmans

t

(10.4) [229] elegantly illustrates this with the following

Xt = Zj,

example.

j=1

10.3: Auto-regressive Processes 203

1000 hPa height 24 hours after initialization Concentration after 24 hours of emissions

Figure 10.6: Example of a simulation of long-range transport of air pollutants.

Left: Simulated 1000 hPa height ¬eld 24 hours after model initialization.

Right: Distribution of pollutant continuously emitted in east England after 24 hours.

From Lehmhaus et al. [250].

obtained by extending the time series.5 Clearly

Consider a stochastic process Xt such that each

realization is constant in time. That is, suppose this does not happen in Koopmans™s example.

xt = a, where a is a realization of an ordinary However, ergodicity is not generally a problem in

random variable A. Every realization of Xt is thus climate research.

a line parallel to the time-axis. It is easily shown

that the process Xt is weakly stationary; the mean

10.3 Auto-regressive Processes

and variance of the process, which are equal to

E(A) and Var(A), respectively, are independent

10.3.0 General. We will explore the properties

of time and all covariances Cov(Xt , Xs ) are also

of auto-regressive processes in some detail in this

equal to Var(A) and hence independent of time.

section. The collection of all weakly stationary

However, the usual estimator of the process mean,

auto-regressive models forms a general purpose

n n

t=1 Xt = n t=1 A = A, does not converge

1 1

n

class of parametric stochastic process models.

to the process mean, E(A), as the length of the

This class is not complete but, given any weakly

averaging interval increases. Since the individual

stationary ergodic process {Xt }, it is possible

realizations of the process do not contain any

to ¬nd an auto-regressive process {Yt } that

variability, a single realization of this process

approximates {Xt } arbitrarily closely.

does not provide suf¬cient information about

Auto-regressive processes are popular in cli-

the process to construct consistent estimators of

mate research, mainly because they represent dis-

process parameters.

cretized versions of ordinary differential equa-

Stochastic processes must be ergodic as well tions [10.3.1]. Conventional auto-regressive pro-

as stationary in order to ensure that individual cesses operate with constant coef¬cients and gen-

realizations of the process contain suf¬cient erate weakly stationary time series. By allowing

information to produce consistent parameter the coef¬cients to vary periodically, the result-

estimates. A technical description of ergodicity ing time series become weakly cyclo-stationary.

is beyond the scope of this book (see, e.g.,

5 Another way of describing an ergodic process is to say that

Brockwell and Davis [68], Koopmans [229]

or Hannan [157]). However, in loose terms, it does not have excessively long memory. Thus the ergodic

ergodicity ensures that the time series varies property is often rate of decay of the auto-covariance function

expressed in terms of a ˜mixing condition™

that involves the

quickly enough in time that increasing amounts with increasing lag. A typical mixing condition speci¬es that

of information about process parameters can be the auto-covariance function should be absolutely summable.

10: Time Series and Stochastic Processes

204

An auto-regressive process of order p, or an

Such processes are called seasonal auto-regressive

AR( p) process, is generally de¬ned as follows:

processes [10.3.8]. The name ˜auto-regressive™

indicates that the process evolves by regressing {Xt : t ∈ Z} is an auto-regressive process of order

past values towards the mean and then adding p if there exist real constants ±k , k = 0, . . . , p,

noise. with ± p = 0 and a white noise process {Zt : t ∈ Z}

The plan for the remainder of the section is as such that

follows. An ordinary auto-regressive (AR) process

p

is de¬ned in [10.3.1] and its mean and variance are

Xt = ±0 + ±k Xt’k + Zt . (10.6)

derived in [10.3.2]. Some speci¬c AR processes

k=1

are examined in [10.3.3,4], and the conditions

under which an AR process is stationary are The most frequently encountered AR processes

discussed in [10.3.5]. As noted above, AR are of ¬rst or second order; an AR(0) process is

processes can be thought of as discretized white noise. Note that Xt is independent of the part

differential equations. We show, in [10.3.6], the of {Zt } that is in the future, but that it is dependent

effect that the ˜dynamics™ of these processes upon the parts of the noise process that are in the

have on their time evolution. Next, we introduce present and the past.

the notion in [10.3.7] that these processes have

a ˜memory™ that can be described in general

10.3.2 Mean and Variance of an AR( p) Process.

terms by a characteristic time. We generalize the

Taking expectations on both sides of (10.6) we see

AR processes so that seasonal behaviour is also

that

accounted for in [10.3.8,9], and the concept is

±0

extended to multivariate processes in [10.3.10].

E(Xt ) = . (10.7)

Looking ahead, we will take a short excursion p

1’ ±k

k=1

into stochastic climate modelling in Section 10.4,

If we set µ = E(Xt ), then (10.6) may be rewritten

but will then return to the subject of parametric

stochastic models in Section 10.5 where we will as

see that the class of AR models is one of three

p

more or less equivalent classes of models.

Xt ’ µ = ±k (Xt’k ’ µ) + Zt . (10.8)