Top: Evolution of the dynamical component Xt = a representation of a stochastic process as the

Dt from an arbitrary initial value when noise is inverse Fourier transform of a random complex

absent. valued function (or measure) that is de¬ned in

Middle: A ˜white noise™ component Nt . the frequency domain rather than the time domain

Bottom: Evolution from an arbitrary initial value (i.e., the so-called Wiener spectral representation

when noise is present. The noise is the same as that of a stochastic process [229, 422]). The spectrum

used in Figure 10.1. is then de¬ned as the expectation of the squared

modulus of the random spectral measure and,

¬nally, the auto-covariance function is shown to be

of Xt+„ . In that sense, the case in which the the inverse Fourier transform of the spectrum.

dynamical component Dt evolves independently

A dif¬culty with the conventional approach,

of the stochastic component exhibits unlimited

however, is that the dynamical aspects of the

predictability. For example, the mean temperature

studied process are obscured. Hence, here we use

in Hamburg in the winter of 3130 will be lower

a non-conventional time domain characterization

than the mean temperature in summer of that year.

of stochastic processes. We return to more

However, the system is inherently unpredictable

conventional approaches in Chapters 11 and 12.

beyond a certain time lag when the evolution of

Another dif¬culty with the conventional ap-

the dynamical part depends on the noise.2

proach concerns the way in which the spectrum

is estimated from a time series. Suppose that

10.1.2 The Probabilistic Structure of Time the stochastic process is observed at times t =

Series. We consider, for the moment, processes 0, 1, . . . , T and, for convenience, that T is even.

in which the dynamical state is determined by Most spectral estimators use the Fourier expansion

the history of the noise. To fully describe the

T /2

stochastic, or probabilistic, structure of such a

ak e’i2π kt

process it is necessary to specify joint density xt =

k=’T /2

functions f (Xt1 , Xt2 , . . . , Xt N ) for an arbitrary

number N of observing times and arbitrary to represent the observed time series. When this

2 Note that this statement is not related to ideas concerning approximation is inverted, a line spectrum |ak |2 ,

for k = 0, ±1, . . . , T /2 is obtained that can be

chaos or nonlinear dynamics in general.

10.2: Basic De¬nitions and Examples 199

interpreted as a raw estimator of the spectrum.

This raw estimator is not generally very useful,

as is easily demonstrated by calculating it for

a white noise time series. The true spectrum

is ¬‚at (˜white™) but the raw estimate exhibits

many large peaks, which are not manifestations

of the ˜dynamics™ of the white noise process. In

fact, when the calculation is repeated for another

realization of the white noise process, peaks

appear at entirely different frequencies.3

Figure 10.3: A two-dimensional representation of

The mathematical inconsistency is that the

the MJO for 1986 [388].

trigonometric expansion is de¬ned only for ¬nite

time series and periodic in¬nite time series, but

stochastic processes are neither ¬nite nor periodic.

In Section 10.5 we deal with two concepts of lesser

Thus, the expansion does not converge as the

importance in climate research, namely the large

length of the time series increases. Note also that

class of linear processes called auto-regressive

a line spectrum is a discrete object, de¬ned for

moving average processes and a special class

frequencies 0, 1/T, 2/T, . . . , 1/2. The spectrum

of nonlinear processes called regime-dependent

of the sampled stochastic process, on the other

auto-regressive processes.

hand, is continuous on the interval [0, 1/2].

However, this approach can still be used to

10.2 Basic De¬nitions and Examples

construct consistent estimates of the spectrum,

provided it is done carefully. These are powerful

10.2.1 Introduction: Characteristic Times. A

methods when properly applied, but misleading

time series is a ¬nite sequence of real or complex

conclusions about the spectrum are frequently

numbers or vectors that are ordered by an index

obtained when they are used naively.

t and understood to be a realization of part of a

stochastic process. The index usually represents

10.1.3 Overview. In this chapter we ¬rst time but could also represent some other non-

introduce the concepts of characteristic times stochastic variable that imposes order on the

and stochastic processes (Section 10.2). Auto- process, such as distance along a transect or depth

regressive processes are the most widely used type in an ice core. Figure 10.3 shows a pair of real

of stochastic process in climate research, since time series that jointly form a (bivariate) index

they may be seen as approximations of ordinary of the so-called Madden-and-Julian Oscillation

linear differential equations subject to stochastic (MJO; [388], see [1.2.3], [15.2.4]). Both time

forcing (Section 10.3). As such they represent series exhibit the typical features of a process in

an important special case of Hasselmann™s which the dynamical component is affected by

˜Stochastic Climate Models™ (Section 10.4; [165]). noise. In particular, the time series lack any strict

regularity; unlike time series of, for example, tidal

3 This observation, and the realization that the spectral

sea level, prediction at long lead times appears to

analysis of a stochastic time series can not be done by simply

be impossible.

extending the time series periodically, are relatively recent

developments. Indeed, at the turn of the twentieth century Despite the absence of strict periodicities, the

there was a frenzy of efforts to detect periodicities in all kinds

two time series do exhibit some regularities.

of data, particular weather-related data, at almost all possible

For example, the series exhibit ˜memory™ in

frequencies. Various climate forecast schemes were built on

the sense that, if a series is positive, it will

this futile approach, some of which can still be found in the

literature. tend to stay positive for some time. That is,

The search for regular weather cycles resulted in a 1936

P (Xt+„ > 0|Xt > 0) > 0.5 for small values of

monograph that contained a four and half page list, entitled

„ . However, for suf¬ciently large ˜lags™ „ , we ¬nd

˜Empirical periods derived from the examination of long series

that knowledge of the sign of Xt does not inform

of observations by arithmetic manipulation or by inspection,™

describing supposed periodicities varying from 1 to 260 years us about the sign of Xt+„ . Thus,

in length (Shaw [347], pp. 320“325). In the light of our present

P Xt+„ > 0|Xt > 0 = 0.5,

understanding of the climate system, this search seems rather (10.1)

absurd, but modesty is advised. Modern workers also often

use allegedly ˜powerful,™ poorly understood techniques in order

for all „ greater than some limit „ . The smallest

to obtain ˜interesting™ results. Future climate researchers will

„ satisfying (10.1), labelled „ M , is a characteristic

probably ¬nd some of our present activities just as absurd and

time that represents the time after which there is no

amusing as the search for periodicities.

10: Time Series and Stochastic Processes

200

forecast skill;4 „ M is a measure of the ˜memory™ of Random variables Xt and Xs are usually depen-

the stochastic process. Inspection of Figure 10.3 dent. This does not prevent the estimation of

indicates that „ M is at least 10“20 days for both process parameters, but it does compromise the

time series. various interval estimation approaches discussed

in Section 5.4 because the dependence violates the

There are various other ways to de¬ne

characteristic times, and [10.3.7] shows that „ M fundamental ˜iid™ assumption. Similarly, most hy-

pothesis testing procedures described in Chapter 6

is not particularly useful in many applications.

no longer operate as speci¬ed when the data are

Another time scale is the average waiting time

serially correlated or otherwise dependent.

between successive local minima or maxima. By

this measure, it would appear that both time

series in Figure 10.3 exhibit quasi-periodicity of 10.2.3 Example: White Noise. White noise, an

about 40 days. Note that, even though the quasi- in¬nite sequence of zero mean iid normal random

periodicities occur on a similar time scale, they variables, is the simplest example of a stochastic

are shifted relative to each other. In the words process. Such processes contain no memory by

of spectral analysis, the two time series vary construction, that is, for every t, element Xt is