Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Lesson 10: Covariances of causalARMA Processes
Umberto Triacca
Dipartimento di Ingegneria e Scienze dell’Informazione e MatematicaUniversita dell’Aquila,
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
A causal ARMA(p, q) process
Consider a causal ARMA(p, q) process
xt −φ1xt−1− ...−φpxt−p = ut + θ1ut−1 + ...+ θqut−q ∀t ∈ Z,
ut ∼ WN(0, σ2u)
φ(z) = 1− φ1z − ...− φpzp
andθ(z) = 1 + θ1z ... + θqz
q
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
A causal ARMA(p, q) process
The causality assumption implies that there exists constantsψ0, ψ1, ... such that
∞∑j=0
|ψj | <∞
and
xt =∞∑j=0
ψjut−j ∀t.
The sequence {ψ0, ψ1, ...} is determined by the identity
(1− φ1z − ...− φpzp) (ψ0 + ψ1z ...) = 1 + θ1z ... + θqz
q
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
A causal ARMA(p, q) process
Equating coefficients of z j , j=0,1,. . . , we obtain
ψ0 = 1
ψ1 = θ1 + ψ0φ1
ψ2 = θ2 + ψ1φ1 + ψ0φ2
and so on.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
A causal ARMA(p, q) process
We have
ψj = θj +
p∑k=1
φkψj−k , j = 0, 1...,
where θ0 = 1, θj = 0 for j > q and ψj = 0 for j < 0.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(p, q) process
Since
E (xt) =∞∑j=0
ψjE (ut−j) = 0 ∀t.
We have
γ(k) = E (xtxt−k)
= E(∑∞
j=0 ψjut−j∑∞
i=0 ψiut−k−i)
=∑∞
j=0
∑∞i=0 ψjψiE (ut−jut−k−i)
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(p, q) process
Since ut ∼ WN(0, σ2), we have that E (ut−jut−k−i) = σ2 ifi = j − k and 0 otherwise. Therefore
γ(k) = σ2∞∑j=0
ψjψj−k k = 0, 1, ...
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(p, q) process
It is possible to show that the sequence {γ(k)}, is absolutelysummable, that is
∞∑k=−∞
|γ(k)| <∞.
Further, we observe that the absolute summability of thesequence {γ(k)} implies that
limk→∞
γ(k) = 0.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(p, q) process
We can summarize our findings about the autocovariancefunction of a causal ARMA(p, q) process as follows:
The autocovariance function of a causal ARMA(p, q)process is absolutely summable;
The autocovariance function of a causal ARMA(p, q)process vanishes when the lag tends to infinity.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Ergodicity and ARMA process
Are the causal ARMA processes ergodic?
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Ergodic Theorems
Corollary 1 (Sufficient condition for mean-ergodicity). Let xtbe a stationary process with mean µ and autocovariancefunction γx(k). If
limk→∞γx(k) = 0,
then xt is mean-ergodic.
Corollary 2. (Sufficient condition for mean-ergodicity) Let xtbe a stationary process with mean µ and and autocovariancefunction γx(k). If
∞∑k=0
|γx(k)| <∞,
then xt is mean-ergodic.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Ergodicity under Gaussianity
Let {xt ; t ∈ Z} be a stationary process with mean µ andautocovariance function γx(k). If the process is Gaussian, then1. the condition of absolute summability of covariancefunction
∞∑k=0
|γx(k)| <∞
is sufficient to ensure ergodicity for all moments.2. the condition
limk→∞γx(k) = 0
is necessary and sufficient.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Ergodicity and ARMA process
Thus we have that
a causal ARMA process is ergodic for the mean.
a Gaussian causal ARMA process is ergodic for allmoments.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(1, 1) process
Consider a causal ARMA(1, 1) process:
xt − φxt−1 = ut + θut−1, ut ∼ WN(0, σ2)
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(1, 1) process
We find that
ψ0 = 1 and ψj = (φ + θ)φj−1 for j ≥ 1.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(1, 1) process
Thus we have
γ(0) = σ2∑∞
j=0 ψ2j
= σ2[
1 + (φ + θ)2∑∞j=0 φ
2j]
= σ2[
1 + (φ+θ)2
1−φ2
]
γ(1) = σ2∑∞
j=0 ψjψj−1
= σ2[φ + θ + (φ + θ)2 φ
∑∞j=0 φ
2j]
= σ2[φ + θ + (φ+θ)2φ
1−φ2
]and
γ(k) = φk−1γ(1), k ≥ 2.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(p, q) process
The autocovariance function, γ(k), of a causal ARMA processtends to zero, as k →∞, at an exponential rate, that is, thereexist constants, D and δ such that, as k →∞, γ(k) ≈ Dδk ,with −1 < δ < 1
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal AR(1)
process
Consider a causal AR(1) process:
xt − φxt−1 = ut , ut ∼ WN(0, σ2)
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal AR(1)
process
We haveγ(0) = σ2
(1
1−φ2
)and
γ(k) = φkγ(0), k ≥ 1.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal AR(1)
process
Consider a causal AR(1) process:
xt − 0.7xt−1 = ut , ut ∼ WN(0, 1)
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal AR(1)
process
Consider a causal AR(1) process:
xt + 0.7xt−1 = ut , ut ∼ WN(0, 1)
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal AR(1)
process
Thus the autocovariances decay exponentially in one of twoforms.
1 If 0 < φ < 1, the autocovariances are positive;
2 If −1 < φ < 0 they oscillate between positive andnegative values.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal AR(2)
process
Consider a causal AR(2) process:
xt = φ1xt−1 + φ2xt−2 + ut , ut ∼ WN(0, σ2u)
The autocovariance function of this process is given by thefollowing recursive formula:
γx(k) = φ1γx(k − 1) + φ2γx(k − 2) k = 2, 3, ...
with the initial conditions
γx(0) =(1− φ2)σ2
u
(1 + φ2)[
(1− φ2)2 − φ21
]and
γx(1) =φ1γx(0)
(1− φ2).
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal AR(2)
process
Consider a causal AR(2) process:
xt = 0.5xt−1 + 0.3xt−2 + ut , ut ∼ WN(0, 1)
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal AR(2)
process
In general, the autocovariance function of AR(p) processesdecays exponentially.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of an MA(1) process
Consider an MA(1) process:
xt = ut + θut−1, ut ∼ WN(0, σ2)
We haveγ(0) = σ2 (1 + θ2)
γ(1) = σ2θ
andγ(k) = 0, k ≥ 2.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal MA(q)
process
In general, the autocovariance function of MA(q) models iszero for all lags greater than q.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
The Autocovariance Function of a causal
ARMA(p, q) process
Causal ARMA processes are short memory processes. Thismeans that their autocovariance function dies out quickly, andhence
∞∑k=−∞
|γ(k)| <∞
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Long memory processes
A stationary process xt with an autocovariance function γ(k)is called a long memory process, if
∞∑k=−∞
|γ(k)| =∞
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Conclusion
In this lecture, we have considered the calculation of theautocovariance function γ(·) of a causal ARMA process.We remenber that the autocorrelation function is readily foundfrom the autocovariance function on dividing by γ(0).The partial autocorrelation function is also found from thefunction γ(·).
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Conclusion
We close this lecture by introducing the lag operator L.
Definition. Lxt = xt−1
Properties.
1 Lj = xt = xt−j2 L(φxt) = φLxt = φxt−1
3 LiLjxt = Lixt−j = xt−j−i = Li+jxt4 L(xt + yt) = Lxt + Lyt = xt−1 + yt−1
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Conclusion
Finally note that by convention we define the zeroth power ofL to be the identity operator, i. e.
L0xt = xt
Thus we can consider polynomials of degree n ≥ 0 in the lagoperator L, defined as
a(L) = a0L0 + a1L + ... + anL
n
where a0, a1, ..., an are real or complex coefficients. We have
a(L)xt = a0xt + a1xt−1 + ... + anxt−n
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Conclusion
It may shown that the algebra of the polynomials of degreen ≥ 0 in the lag operator L, over the field of the real orcomplex numbers (i.e. with real or complex coefficients) isisomorphic to the algebra of polynomials in the real or complexindeterminate, say z .
We can manipulate the polynomials a(L), b(L),... like thepolynomials a(z), b(z),...
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Conclusion
ARMA(1,1) in terms of the lag operator L.
Consider an ARMA(1,1) process defined by the equation
xt − φxt−1 = ut + θut−1 ut ∼ WN(0, σ2u)
By using the lag operator this equation can be rewritten as
φ(L)xt = θ(L)ut
where φ(L) = 1− φL and θ(L) = 1 + θL
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes
Conclusion
ARMA(p,q) in terms of the lag operator L.
We can writeφ(L)xt = θ(L)ut
whereφ(L) = 1− φ1L− φ2L
2 − ...− φ1Lp
andφ(L) = 1 + θ1L + θ2L
2 − ... + θqLq
.
Umberto Triacca Lesson 10: Covariances of causal ARMA Processes