The Spectral Representation of Stationary Time Series

Stationary time series satisfy the properties:

1. Constant mean (E(xt) = )

2. Constant variance (Var(xt) = 2)

3. Correlation between two observations (xt, xt + h) dependent only on the distance h.

These properties ensure the periodic nature of a stationary time series

-15

-10

-5

0

5

10

15

0 20 40 60 80 100

and X1, X1, … , Xk and Y1, Y2, … , Yk are independent independent random variables with

sincos1

k

iiiiit tYtXx

222 and iii YEXE 0 ii YEXE

where 1, 2, … k are k values in (0,)

Recall

is a stationary Time series

With this time series

2

1

2. cosk

i ii

h h

and

1

3. cos0

k

i ii

hh w h

k

jj

iiw

1

2

2

where

1. 0tE x We can give it a non-zero mean, , by adding to the equation

We now try to extend this example to a wider class of time series which turns out to be the complete set of weakly stationary time series. In this case the collection of frequencies may even vary over a continuous range of frequencies [0,].

The Riemann integral

10

0

limb n

i i ix

ia

g x dx g c x x

The Riemann-Stiltjes integral

10

0

limb n

i i ix

ia

g x dF x g c F x F x

If F is continuous with derivative f then:

b b

a a

g x dF x g x f x dx If F is is a step function with jumps pi at xi then:

b

i iia

g x dF x p g x

First, we are going to develop the concept of integration with respect to a stochastic process.

Let {U(): [0,]} denote a stochastic process with mean 0 and independent increments; that is

E{[U(2) - U(1)][U(4) - U(3)]} = 0 for

0 ≤ 1 < 2 ≤ 3 < 4 ≤ .and

E[U() ] = 0 for 0 ≤ ≤ .

In addition let

G() =E[U2() ] for 0 ≤ ≤ and assume G(0) = 0.

It is easy to show that G() is monotonically non decreasing. i.e. G(1) ≤ G(2) for 1 < 2 .

Now let us define:

analogous to the Riemann-Stieltjes integral

0

).()( dUg

0

( ) ( ).g dF

Let 0 = 0 < 1 < 2 < ... < n = be any partition of the interval. Let .

Let idenote any value in the interval [i-1,i]Consider:

Suppose that and

there exists a random variable V such that

* 1max iin

n

iiiin UUgV

11

* )]()()[(

0lim n

n

0lim 2

VVE nn

Then V is denoted by:

0

).()( dUg

Properties:

0).()(0

dUgE

2

2

0 0

( ) ( ). ( ) ( ).E g dU g dG

0

21 ).()()( dGgg

0

2

0

1 ).()().()( dUgdUgE

1.

2.

3.

The Spectral Representation of Stationary Time Series

Let {X(): [0,]} and {Y(): l [0,]} denote a uncorrelated stochastic process with mean 0 and independent increments. Also let

F() =E[X2() ] =E[Y2() ] for 0 ≤ ≤ and F(0) = 0.

Now define the time series {xt : t T}as follows:

00

)()sin()()cos( dYtdXtxt

Then

00

)()sin()()cos( dYtdXtExE t

00

)()sin()()cos( dYtEdXtE

0

Also

00

)()sin()()cos( dYhtdXhtE

tht xxEh

00

)()sin()()cos( dYtdXt

0

00

)()cos()()cos( dXtdXhtE

00

)()sin()()sin( dYtdYhtE

00

)()sin()()cos( dYtdXhtE

00

)()cos()()sin( dXtdYhtE

0

)()cos()cos( dFtht

00)()sin()sin(0

dFtht

0

)cos()cos( tht

)()sin()sin( dFtht

0

)()cos( dFh

Thus the time series {xt : t T} defined as follows:

00

)()sin()()cos( dYtdXtxt

is a stationary time series with:

0 txE

0

)()cos( and dFhh

F() is called the spectral distribution function:

If f() = Fˊ() is called then is called the spectral density function:

00

)cos()()cos( dfhdFhh

Note

00

)(0 dfdFxVar t

The spectral distribution function, F(), and spectral density function, f() describe how the variance of xt is distributed over the frequencies in the interval [0,]

00

)cos( dfedfhh hi

The autocovariance function, (h), can be computed from the spectral density function, f(), as follows:

)sin()cos( iei

Also the spectral density function, f(), can be computed from the autocovariance function, (h), as follows:

1

1 10 cos( )

2 h

f h h

00

02

h

hh

Example: Let {ut : t T} be identically distributed and uncorrelated with mean zero (a white noise series). Thus

and

1

1 10 cos( )

2 h

f h h

if 2

2

Graph:

frequency

f( )

Example:

Suppose X1, X1, … , Xk and Y1, Y2, … , Yk are independent independent random variables with

sincos1

k

iiiiit tYtXx

222 and iii YEXE 0 ii YEXE

Let 1, 2, … k denote k values in (0,)

Then

k

iii hh

1

2 cos

If we define {X(): [0,]} and{Y(): [0,]}

ii i

ii

i YYXX::

)( and )(by

)( )()( 2

:

2

:

2

YEXVarXEFii i

ii

i

Note: X() and Y() are “random” step functions and F() is a step function.

k

iii h

1

2 cos

0

)()cos( dFhh

00

)()sin()()cos(then dYtdXtxt

sincos1

k

iiiii tYtX

2

1

Note: 0k

ii

0

)()cos( dFhh

0

)()cos( dfh

)()( and fF

Another important comment

In the case when F() is continuous

then

dfhdfhh s)cos()cos(0

in this case

0

)cos(1

02

1

hs hhf

Sometimes the spectral density function, f(), is extended to the interval [-,] and is assumed symmetric about 0 (i.e. fs() = fs (-) = f ()/2 )

It can be shown that

0

)cos(2

01

andh

hhf

Hence

From now on we will use the symmetric spectral density function and let it be denoted by, f().

dfhh )cos(

0

)cos(1

02

1 and

h

hhf

Linear Filters

sstst xay

Let {xt : t T} be any time series and suppose that the time series {yt : t T} is constructed as follows: :

The time series {yt : t T} is said to be constructed from {xt : t T} by means of a Linear Filter.

input xtoutput yt

Linear Filter

as

thty yyEh

Let x(h) denote the autocovariance function of {xt : t T} and y(h) the autocovariance function of {yt : t T}. Assume also that E[xt] = E[yt] = 0.Then: :

kktk

sshts xaxaE

s kktshtks xxaaE

s kktshtks xxEaa

s k

xks kshaa

s k

xks dfkshaa

cos

s k

xkshi

ks dfeaa

Re

dfeaa x

s k

kshiksRe

dfeaae x

s k

ksiks

hiRe

dfeaeae

k

kik

s

sis

hiRe

dfeae x

s

sis

hi

2

Re

dfeae xs

sis

hi

2

Re

dfeah xs

sis

2

cos

Hence

dfAhh xy

2cos

functionTransfereaAs

sis the

where

hy

of the linear filter

dfeah xs

sis

2

cos

Note:

dfAh x

2cos

x

s

sisxy feafAf

22

hence

hy

dfh ycos

Spectral density functionMoving Average Time series of order q, MA(q)

qtqtttt uuuux 2211

Let 0 =1, 1, 2, … q denote q + 1 numbers.

Let {ut|t T} denote a white noise time series with variance 2.

Let {xt|t T} denote a MA(q) time series with = 0.

Note: {xt|t T} is obtained from {ut|t T} by a linear filter.

Now 2

2

uf

u

q

s

sisux fefAf

2

0

2

Hence

2

0

2

2

q

s

sise

Example: q = 1

21

0

2

2

s

sisx ef

2

1

2

12

ie

ii ee 11

2

112

ii ee

121

2

12

cos212 1

21

2

Example: q = 2

22

0

2

2

s

sisx ef

2221

2

12

ii ee

2

212

21

2

112

iiii eeee

ii ee

21122

21

2

12 22

2ii ee

2cos2cos212 2211

22

21

2

3.02.01.00.00.0

0.1

0.2

0.3

0.4

0.5

0.6

3.02.01.00.00.0

0.1

0.2

0.3

0.4

0.5

0.6

3.02.01.00.00.0

0.1

0.2

0.3

0.4

Spectral Density fuction of an MA(2) Series

1 = 0.70

2 = -0.20

Spectral density function for MA(1) Series

Spectral density functionAutoregressive Time series of order p, AR(p)

2211 tptpttt uxxxx

Let 1, 2, … p denote p + 1 numbers.

Let {ut|t T} denote a white noise time series with variance 2.

Let {xt|t T} denote a AR(p) time series with = 0.

Note: {ut|t T} is obtained from {xt|t T} by a linear filter.

or 2211 tptpttt uxxxx

Now 2

2

uf

x

p

s

sisxu fefAf

2

1

21

Hence

x

p

s

sis fe

2

1

2

12

or

2

1

2

12

or

p

s

sis

x

e

f

Example: p = 1

21

1

2

12

s

sis

x

e

f

2

1

2

12

ie

ii ee 11

2

112

ii ee

12

1

2

12

cos212 12

1

2

Example: p = 2

xf

2cos2cos1212 22122

21

2

Example : Sunspot Numbers (1770-1869) 1770 101 1795 21 1820 16 1845 40 1771 82 1796 16 1821 7 1846 64 1772 66 1797 6 1822 4 1847 98 1773 35 1798 4 1823 2 1848 124 1774 31 1799 7 1824 8 1849 96 1775 7 1800 14 1825 17 1850 66 1776 20 1801 34 1826 36 1851 64 1777 92 1802 45 1827 50 1852 54 1778 154 1803 43 1828 62 1853 39 1779 125 1804 48 1829 67 1854 21 1780 85 1805 42 1830 71 1855 7 1781 68 1806 28 1831 48 1856 4 1782 38 1807 10 1832 28 1857 23 1783 23 1808 8 1833 8 1858 55 1784 10 1809 2 1834 13 1859 94 1785 24 1810 0 1835 57 1860 96 1786 83 1811 1 1836 122 1861 77 1787 132 1812 5 1837 138 1862 59 1788 131 1813 12 1838 103 1863 44 1789 118 1814 14 1839 86 1864 47 1790 90 1815 35 1840 63 1865 30 1791 67 1816 46 1841 37 1866 16 1792 60 1817 41 1842 24 1867 7 1793 47 1818 30 1843 11 1868 37 1794 41 1819 24 1844 15 1869 74

189018601830180017700

100

200

Example B: Annual Sunspot Numbers (1790-1869)

Autocorrelation function and partial autocorrelation function

-1.0

0.0

1.0

rh

h

10 20 30 40

x t

-1.0

0.0

1.0

x t

kk

10 20 30 40k

Spectral density Estimate

0.50.40.30.20.10.00

2000

4000

6000

8000

Smoothed Spectral Estimator (Bandwidth = 0.11)

frequency

Period = 10 years

Assuming an AR(2) model

3.02.01.00.00

300

600

Spectral density of Sunspot data

f( )

Period = 8.733 years

A linear discrete time series

Moving Average time series of infinite order

332211 ttttt uuuux

Let 0 =1, 1, 2, … denote an infinite sequence of numbers.

Let {ut|t T} denote a white noise time series with variance 2.

– independent– mean 0, variance 2.

Let {xt|t T} be defined by the equation.

Then {xt|t T} is called a Linear discrete time series.Comment: A linear discrete time series is a Moving

average time series of infinite order

The AR(1) Time series

ttt uxx 11

Let {xt|t T} be defined by the equation.

2211 ttt uuu

Then tttt uuxx 1211

11122

1 1 ttt uux 111231

21 1 tttt uuux

22

1112

1133

1 1 tttt xuux

where

1

321 1

111

and

ii 1

An alternative approach using the back shift operator, B.

ttt uxx 11

tt uxBI 1

The equation:

can be written

Now 33221

11 11

BBBIBI

since

tt uxBI 1

The equation:

has the equivalent form:

IBBBIBI 332211 11

tt uBIBIx 11

11

3322

1 11 BBBI

tuBBBI 33221 11

32

1 11 I

33

22

11 11 tttt uuuu

The time series {xt |t T} can be written as a linear discrete time series

pp BBBIB 2

21 where

tt uxB

and

For the general AR(p) time series:

33

221

1 BBBIB

tt uBBx 11

[(B)]-1can be found by carrying out the multiplication

IBBBIB 33

221

can be written:

tt uxB

where

Thus the AR(p) time series:

221

1 BBIBB

tt uBBx

tuB

1

Hence tt uBx 1

2211 ttt uuu

This called the Random Shock form of the series

can be written:

tt uxB

where

Thus the AR(p) time series:

221

1 BBIBB

tt uBBx

tuB

1

Hence tt uBx 1

2211 ttt uuu

This called the Random Shock form of the series

An ARMA(p,q) time series {xt |t T} satisfies the equation:

pp BBBIB 2

21 where

tt uBxB

and

The Random Shock form of an ARMA(p,q) time series:

qq BBBIB 2

21

Again the time series {xt |t T} can be written as a linear discrete time series

namely

where

33

221

1 BBBIBBB

(B) =[(B)]-1[(B)] can be found by carrying out the multiplication

BBBBIB 33

221

tt uBBBx 11

tuB 11

Thus an ARMA(p,q) time series can be written:

2211 tttt uuux

where

33

221

1 BBBIBBB

p

211

1

and

The inverted form of a stationary time series

Autoregressive time series of infinite order

An ARMA(p,q) time series {xt |t T} satisfies the equation:

pp BBBIB 2

21 where

tt uBxB

and qq BBBIB 2

21

Suppose that

qq xxxx 2

21 1

exists. 1B

This will be true if the roots of the polynomial

all exceed 1 in absolute value.

The time series {xt |t T} in this case is called invertible.

Then

where

33

221

1 BBBIBBB

tt uBxBB 11

or tt uxB *

11 and 11*

B

Thus an ARMA(p,q) time series can be written:

tttt uxxx *

2211 where

33

221

1 BBBIBBB

q

21

*

11 and

This is called the inverted form of the time series.

This expresses the time series an autoregressive time series of infinite order.