Upload
doandat
View
259
Download
1
Embed Size (px)
Citation preview
Introduction to Stochastic Processes
Eduardo RossiUniversity of Pavia
October 2013
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 1 / 28
Stochastic Process
Stochastic ProcessA stochastic process is an ordered sequence of random variables defined on aprobability space (Ω,F ,P).
Yt(ω), ω ∈ Ω, t ∈ T , such that for each t ∈ T , Yt(ω) is a random variable onthe sample space Ω, and for each ω ∈ Ω, Yt(ω) is a realization of the stochasticprocess on the index set T (that is an ordered set of values, each corresponds to avalue of the index set).
Time Series
A time series is a set of observations Yt , t ∈ T0, each one recorded at aspecified time t. The time series is a part of a realization of a stochastic process,Yt , t ∈ T where T ⊇ T0. An infinite series Yt∞t=−∞.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 2 / 28
Stochastic Process
A statistical time series is a mathematical sequence, not a series.
Yt is assumed to be real valued, but we shall also consider sequences ofrandom vectors Yt, or variables with values in the complex numbers.
We are interested in the joint distribution of the variables Yt. The easiestway to ensure that these variables possess joint laws, is to assume that theyare defined as measurable maps on a single underlying probability space.
One of the purposes of time series analysis is to study and characterizeprobability distributions of sets of variables Yt that will typically bedependent.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 3 / 28
Stochastic Process
The statistical problem is to determine the probability distribution of the timeseries given observations Y1,Y2, . . . ,YT .
The statistical model can be used for1 understanding the stochastic system;2 predicting the ’future’, i.e. YT+1,YT+2, . . .
In order to have any chance of success at these tasks it is necessary toassume some apriori structure of the time series.
If the Yt could be completely arbitrary random variables, thenY1, . . . ,YT would constitute a single observation from a completelyunknown distribution on RT .
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 4 / 28
Mean, Variance and Autocovariance
The unconditional mean
µt = E [Yt ] =
∫Yt f (Yt)dYt
Autocovariance function: The joint distribution of
(Yt ,Yt−1, . . . ,Yt−h)
is usually characterized by the autocovariance function:
γt(h) = Cov(Yt ,Yt−h)
= E [(Yt − µt)(Yt−h − µt−h)]
=
∫. . .
∫(Yt − µt)(Yt−h − µt−h)f (Yt , . . . ,Yt−h)dYt . . . dYt−h
The autocorrelation function
ρt(h) =γt(h)√
γt(0)γt−h(0)
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 5 / 28
Mean, Variance and Autocovariance
Both functions are symmetric about 0.
The autocovariance and autocorrelations functions are measures of (linear)dependence between the variables at different time instants.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 6 / 28
Stationarity
A basic type of structure is stationarity. This comes in two forms.
Strict Stationarity
The process is said to be strictly stationary if for any values of h1, h2, . . . , hn thejoint distribution of (Yt ,Yt+h1 , . . . ,Yt+hn) depends only on the intervalsh1, h2, . . . , hn but not on the date t itself:
f (Yt ,Yt+h1 , . . . ,Yt+hn) = f (Yτ ,Yτ+h1 , . . . ,Yτ+hn) ∀t, h
Strict stationarity implies that all existing moments are time invariant.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 7 / 28
Stationarity
Weak (Covariance) Stationarity
The process Yt is said to be weakly stationary or covariance stationary if thesecond moments of the process are time invariant:
E [Yt ] = µ <∞ ∀t
E [(Yt − µ)(Yt−h − µ)] = γ(h) <∞ ∀t, h
Stationarity implies γt(h) = γt(−h) = γ(h).
A strictly stationary time series with finite second moments is also stationary.
Gaussian Process
The process Yt is said to be Gaussian if the joint density of (Yt ,Yt+h1 , . . . ,Yt+hn),f (Yt ,Yt+h1 , . . . ,Yt+hn), is Gaussian for any h1, h2, . . . , hn.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 8 / 28
Ergodicity
The statistical ergodicity theorem concerns what information can be derived froman average over time about the common average at each point of time.Note that the WLLN does not apply as the observed time series represents justone realization of the stochastic process.
Ergodic for the mean
Let Yt(ω), ω ∈ Ω, t ∈ T be a weakly stationary process, such that
E [Yt(ω)] = µ <∞ and E [(Yt(ω)− µ)2] = σ2 <∞ ∀t. Let yT = T−1∑T
t=1 Yt
be the time average. If yT converges in probability to µ as T →∞, Yt is said tobe ergodic for the mean.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 9 / 28
Ergodicity
To be ergodic the memory of a stochastic process should fade in the sensethat the covariance between increasingly distant observations converges tozero sufficiently rapidly.
For stationary process it can be shown that absolutely summableautocovariances, i.e.
∑∞h=0 |γ(h)| <∞, are sufficient to ensure ergodicity.
Ergodic for the second moments
γ(h) = (T − h)−1T∑
t=h+1
(Yt − µ)(Yt−h − µ)P→ γ(h)
Ergodicity focus on asymptotic independence, while stationarity on thetime-invariance of the process.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 10 / 28
Example
Deterministic trigonometric series:
Yt = A cos (tλ) + B sin (tλ)
E [A] = E [B] = 0, E [A2] = E [B2] = σ2 defines a stationary time series.
E [Yt ] = E [A cos (tλ) + B sin (tλ)] = 0
The Autocovariance function
γ(h) = E [YtYt+h]
= cos (t + h)λ cos (tλ) Var[A] + sin (t + h)λ sin (tλ) Var[B]
= σ2 cos (hλ)
Once A and B have been determined the process behaves as a deterministictrigonometric function. This type of time series is an important building block tomodel cyclic events in a system.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 11 / 28
Example
Consider the stochastic process Yt defined by
Yt =
u0 t = 0 with u0 ∼ N(0, σ2)Yt−1 t > 0
Then Yt is strictly stationary but not ergodic.Proof Obviously we have that Yt = u0 for all t ≥ 0. Stationarity follows from:
E [Yt ] = E [u0] = 0
E [Y 2t ] = E [u2
0 ] = σ2
E [YtYt−1] = E [u20 ] = σ2 (1)
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 12 / 28
Example
Thus we have µ = 0, γ(h) = σ2 ρ(h) = 1 are time invariant. Ergodicity for themean requires:
yT = T−1T−1∑t=o
Yt = T−1(Tu0)
yT = u0
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 13 / 28
Random walk
Yt = Yt−1 + ut
where ut ∼WN(0, σ2). By recursive substitution,
Yt = Yt−1 + ut
Yt = Yt−2 + ut−1 + ut
...
= Y0 + ut + ut−1 + ut−2 + . . .+ u1
The mean is time-invariant:
µ = E [Yt ] = E
[Y0 +
t∑s=1
us
]= Y0 +
t∑s=1
E [us ] = 0.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 14 / 28
Random walk
But the second moments are diverging. The variance is given by:
γt(0) = E [Y 2t ] = E
(Y0 +t∑
s=1
us
)2 = E
( t∑s=1
us
)2
= E
[t∑
s=1
t∑k=1
usuk
]= E
t∑s=1
u2s +
t∑s=1
t∑k=1,k 6=s
usuk
=
t∑s=1
E [u2s ] +
t∑s=1
t∑k=1,k 6=s
E [usuk ] =t∑
s=1
σ2 = tσ2.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 15 / 28
Random walk
The autocovariances are:
γt(h) = E [YtYt−h] = E
[(Y0 +
t∑s=1
us
)(Y0 +
t−h∑k=1
uk
)]
= E
[t∑
s=1
us
(t−h∑k=1
uk
)]
=t−h∑k=1
E [u2k ]
=t−h∑k=1
σ2 = (t − h)σ2 ∀h > 0.
Finally, the autocorrelation function ρt(h) for h > 0 is given by:
ρ2t (h) =γ2t (h)
γt(0)γt−h(0)=
[(t − h)σ2]2
[tσ2][(t − h)σ2]= 1− h
t∀h > 0
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 16 / 28
White Noise Process
A white-noise process is a weakly stationary process which has zero mean and isuncorrelated over time:
ut ∼WN(0, σ2)
Thus ut is WN process ∀t ∈ T :
E [ut ] = 0
E [u2t ] = σ2 <∞
E [utut−h] = 0 h 6= 0, t − h ∈ T
If the assumption of a constant variance is relaxed to E [u2t ] <∞, sometimes
ut is called a weak WN process.
If the WN process is normally distributed it is called a Gaussian white-noiseprocess:
ut ∼ NID(0, σ2). (2)
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 17 / 28
The assumption of normality implies strict stationarity and serial independence(unpredictability). A generalization of the NID is the IID process with constant,but unspecified higher moments.A process ut with independent, identically distributed variates is denoted IID:
ut ∼ IID(0, σ2) (3)
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 18 / 28
Martingale
Martingale The stochastic process xt is said to be martingale with respect to aninformation set (σ-field), It−1, of data realized by time t − 1 if
E [|xt |] < ∞E [xt |It−1] = xt−1 a.s. (4)
Martingale Difference Sequence The process ut = xt − xt−1 with E [|ut |] <∞ andE [ut |It−1] = 0 for all t is called a martingale difference sequence, MDS.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 19 / 28
Martingale difference
Let xt be an m.d. sequence and let gt−1 = σ(xt−1, xt−2, . . .) be anymeasurable, integrable function of the lagged values of the sequence. Then xtgt−1is a m.d., and
Cov [gt−1, xt ] = E [gt−1xt ]− E [gt−1]E [xt ] = 0
This implies that, putting gt−1 = xt−j ∀j > 0
Cov [xt , xt−j ] = 0
M.d. property implies uncorrelatedness of the sequence.White Noise processes may not be m.d. because the conditional expectation of anuncorrelated process can be a nonlinear.An example is the bilinear model (Granger and Newbold, 1986)
xt = βxt−2εt−1 + εt , εt ∼ i .i .d(0, 1)
The model is w.n. when 0 < β < 1√2
.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 20 / 28
The conditional expectations are the following nonlinear functions:
E [xt |It−1] = −∞∑i=1
(−β)i+1xt−i
i+1∏j=2
xt−j
Uncorrelated, zero mean processes:
White Noise Processes1 IID processes
1 Gaussian White Noise processes
Martingale Difference Processes
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 21 / 28
Innovation
Innovation
An innovation ut against an information set It−1 is a process whose densityf (ut |It−1) does not depend on It−1.
Mean Innovation
ut is a mean innovation with respect to an information set It−1 ifE [ut |It−1] = 0.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 22 / 28
The Wold Decomposition
If the zero-mean process Yt is wide sense stationary (implying E (Y 2t ) <∞) it has
the representation
Yt =∞∑j=0
ψjεt−j + vt
where ψ0 = 1,∑∞
j=0 ψ2j <∞
εt ∼WN(0, σ2)
εt = Yt − E [Yt |Yt−1,Yt−2, . . .]
E (vtεt−j) = 0,∀j
vt is perfectly predictable one-step ahead
vt = E [vt |Yt−1,Yt−2, . . .]
vt , linear deterministic component
If vt = 0, Yt is called a purely non-deterministic process.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 23 / 28
Causal process
Linear Filter
The linear filter with filter coefficients (ψj : j ∈ Z) is the operation that transformsa given time series Xt into the time series
Yt =∑j∈Z
ψjXt−j
A linear filter is a moving average of infinite order. All filters used in practice arefinite filters: only finitely many coefficients ψj are nonzero. Important examplesare
the difference filter ∆Yt
Seasonal filter ∆kYt for k the ’length of the season’ or ’period’.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 24 / 28
Causal process
Causal process
A linear process Yt is causal (strictly, a causal function of εt) if there is a
ψ(L) = ψ0 + ψ1L + ψ2L2 + . . .
with∑∞
j=0 |ψj | <∞
Yt = ψ0εt + ψ1εt−1 + ψ2εt−2 + . . .
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 25 / 28
Invertibility
Invertibility
A linear process Yt is invertible (strictly, an invertible function of εt) if thereis a
π(L) = π0 + π1L + π2L2 + . . .
with∞∑j=0
|ψj | <∞
andεt = π(L)Yt
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 26 / 28
Box-Jenkins approach to modelling time series
Approximate the infinite lag polynomial with the ratio of two finite-orderpolynomials φ(L) and θ(L):
ψ(L) =∞∑j=0
ψjLj ∼=
θ(L)
φ(L)=
1 + θ1L + θ2L2 + . . .+ θqLq
1− φ1L− φ2L2 . . .− φpLp
Time Series Models
p q Model Typep > 0 q = 0 φ(L)Yt = εt AR(P)p = 0 q > 0 Yt = θ(L)εt MA(Q)p > 0 q > 0 φ(L)Yt = θ(L)εt ARMA(p,q)
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 27 / 28
Invertible process
Yt has a causal ARMA process representation
Yt = ψ(L)εt =∑j
ψjYt−j
if ψj = 0, j < 0.
Yt has an invertible ARMA process representation
εt = π(L)Yt =∑j
πjYt−j
if the filter π(L) is causal, i.e. πj = 0, j < 0.
Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 28 / 28