De�nitions and Review MA processes AR processes ARMA Processes
Stationary ARMA processes
Helle Bunzel
ISU
January 30, 2009
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Introduction I
In this part of the lectures, we will start with some de�nitions, newand review.
After this we will focus on ARMA processes.
They are important because they are used VERY frequently to modelerror processes of time series.We will look at:
What they are and their properties.A bit about how to estimate them.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
White noise processes I
De�nition
A white noise process is a sequence fεtg∞t=�∞ , where
E (εt ) = 0
E�ε2t�= σ2
E (εi εj ) = 08i 6= j
De�nitionAn independent white noise process, is a white noise process where εi andεj are independent 8i 6= j
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
White noise processes II
De�nitionA Gaussian white noise process, is a white noise process where
εt � N�0, σ2
�Note that Hamilton de�nes the Gaussian process a little bit di¤erently.He starts with an independent white noise process. Does that matter?
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Expectations I
This part is mostly review.
We typically consider a sequence of random variables fYtg∞t=�∞ ,
from where we get a sample fytg∞t=�∞
In fact we usually only get a sample fytgTt=1 from fYtgTt=1 .
From one sequence of random variables fYtg∞t=�∞ we can imagine
getting many sample sequences:�y1t∞t=�∞ ,
�y2t∞t=�∞ , ...,
�y It∞t=�∞ .
The mean of Yt is
E (Yt ) �Z ∞
�∞yt fYt (yt ) dyt
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Expectations IIWe could estimate this mean by taking the average of the t 0thobservation from each of the I sequences above: 1I ∑I
i=1 yit
Note that
limI!∞
1I
I
∑i=1
y it = E (Yt )
Also note that this part of Hamilton is a little unclear.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Expectations IIITypical time series and their means:
Yt = µ+ εt , E (Yt ) = µ
Yt = βt + εt , E (Yt ) = βt
Note that the mean can depend on t!
In general
E (Yt ) = µt
Similarly
V (Yt ) �Z ∞
�∞(yt � µt )
2 fYt (yt ) dyt
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Autocovariance IDe�nition
The j 0th autocovariance of Yt is
γtj � Cov (Yt ,Yt�j ) = E�(Yt � µt )
�Yt�j � µt�j
��Recall that this de�nition implies that
γtj =Z ∞
�∞
Z ∞
�∞(yt � µt )
�yt�j � µt�j
�fYt ,Yt�j (yt , yt�j ) dytdyt�j
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Autocovariance II
This is also equivalent to the way Hamilton de�nes autocovariance:
γtj =Z ∞
�∞
Z ∞
�∞(yt � µt )
�yt�j � µt�j
�fYt ,Yt�1,...,Yt�j (yt , yt�1, ..., yt�j ) dytdyt�1...dyt�j
because
fYt ,Yt�j (yt , yt�j ) =Z ∞
�∞
Z ∞
�∞fYt ,Yt�1,...,Yt�j (yt , yt�1, ..., yt�j ) dyt�1...dyt�j+1
Note as a special case that
γt0 = Cov (Yt ,Yt�0) = V (Yt )
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Autocorrelation I
Recall that the correlation between two stochastic variables is
ρ =COV (X ,Y )pV (X )V (Y )
Similarly the autocorrelation function is de�ned as:
De�nition
The j 0th autocorrelation of Yt is
ρts =γtsp
V (yt )V (yt�s )
Note that by de�nition ρt0 = 1
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Stationarity IDe�nitionA process is said to be covariance-stationary or weakly stationary if
E (Yt ) = µ 8t,E [(Yt � µ) (Yt�j � µ)] = γj 8t, j
An important consequence of covariance-stationarity: Cov (Yt ,Yt�j )depends only on j , the time-distance between the two variables.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Stationarity IIDe�nitionA process is said to be strictly stationary if the joint distribution of(Yt ,Yt+j1 ,Yt+j2 , ....,Yt+jn ) depends only on j1, j2, ..., jn and not on t.
Note that a strictly stationary process is covariance-stationary. Why?
De�nitionA process is said to be Gaussian if the joint density fYt ,Yt+j1 ,Yt+j2 ,....,Yt+jn isGaussian for any choice of j1, j2, ..., jn.
Note that a covariance-stationary Gaussian process is strictlystationary. Why?
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Stationarity III
Why is stationarity important?
For most series we have only one observation at each point in time.Therefore, if µt is not the same every period, we have only oneobservation to estimate it with.If all observations share the same parameter, we can potentially usethem all to estimate the parameter.How many observations do we have available to estimate γs ?
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Ergodicity I
De�nitionA covariance stationary process is said to be ergodic for the mean if
plimT!∞1T
T
∑t=1yt = E (Yt )
It turns out that a su¢ cient condition for a covariance stationaryprocess to be ergodic for the mean is that
∞
∑j=0
��γj �� < ∞ (1)
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Ergodicity II
De�nitionA covariance stationary process is said to be ergodic for the secondmoments if
plimT!∞1
T � jT
∑t=1(yt � µ) (yt�j � µ) = γj 8j
Su¢ cient conditions for this to hold are given in Hamilton, Chapter 7.
It turns out that if Yt is a Gaussian process, (1) is also su¢ cient forergodicity of the second moment.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Stationarity and Ergodicity, Example I
Consider the following process:
Yt = µ̃+ εt
where both µ̃ and εt are random variables which are independent ofeach other and
µ̃ � N�0,λ2
�.
Furthermore εt is a Gaussian white noise process with mean 0 andvariance σ2.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Stationarity and Ergodicity, Example II
Note that
E (Yt ) = E (µ̃) + E (εt ) = 0
and
γtj = E [YtYt�j ] = E [(µ̃+ εt ) (µ̃+ εt�j )]
= E�µ̃2 + µ̃εt + µ̃εt�j + εt εt�j
�= E µ̃2 + E (εt εt�j ) =
�λ2 if j 6= 0
λ2 + σ2 if j = 0
This means that the series is covariance stationary.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Stationarity and Ergodicity, Example III
Note however that
∞
∑j=0
��γj �� = σ2 +∞
∑j=0
λ2 = ∞
and
1T
T
∑t=1Yt =
1T
T
∑t=1(µ̃+ εt ) = µ̃+
1T
T
∑t=1
εt ! µ 6= E (Yt ) = 0
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
First order MA processes
In this part of the course we�ll assume that everything is stable.
MA stands for "Moving Average".
Let fεtg be a white noise process and µ and θ be constants.
The process Yt de�ned as
Yt = µ+ εt + θεt�1
is a moving average process of order 1.
Notation: MA(1)
Note that we can also write it as
Yt = µ+ (1+ θL) εt
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Properties of MA(1) processes I
When looking at the properties of a timeseries process, �nding themean, variance, autocovariance and autocorrelation functions as wellas determining conditions for stationarity are the primary steps.
First the mean:
E (Yt ) = E (µ) + E ((1+ θL) εt ) = µ
Variance:
V (Yt ) = Eh(Yt � µ)2
i= E
h(µ+ εt + θεt�1 � µ)2
i= E
h(εt + θεt�1)
2i=�1+ θ2
�σ2
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Properties of MA(1) processes II
Autocovariances. Start with the �rst:
γ1 = E [(Yt � µ) (Yt�1 � µ)]
= E [(εt + θεt�1) (εt�1 + θεt�2)]
= θE�ε2t�1
�= θσ2
The second:
γ2 = E [(Yt � µ) (Yt�2 � µ)]
= E [(εt + θεt�1) (εt�2 + θεt�3)] = 0
All higher autocovariances are 0 as well.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Properties of MA(1) processes IIISummary:
E (Yt ) = µ
V (Yt ) =�1+ θ2
�σ2
γj =
�θσ2 if j = 10 otherwise
Clearly an MA(1) process is covariance stationary regardless of thevalues of the parameters!
The autocorrelation:
ρ1 =γ1p
V (Yt )V (Yt�1)=
θσ2�1+ θ2
�σ2=
θ�1+ θ2
�Note the mean parameter(s) has no in�uence on the autocovarianceand autocorrelation functions.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Properties of MA(1) processes IVAlso note that di¤erent processes can share the same autocorrelation.Consider the MA(1) process
Yt = µ+ εt + ψεt�1
where ψ = 1θ .
Then
ρ1 =ψ�
1+ ψ2� = 1
θ�1+
� 1θ
�2�=
θ2 1θ
θ2�1+
� 1θ
�2� = θ�θ2 + 1
�
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Properties of MA(1) processes V
Pictures of the autocorrelation functions:
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Higher order MA processesAn MA(q) process is given by
Yt = µ+ εt + θ1εt�1 + θ2εt�2 + ...+ θqεt�q ,
Yt = µ+q
∑j=0
θj εt�j , θ0 = 1,
Yt = µ+q
∑j=0
θjLj εt , θ0 = 1
where fεtg is white noise and (µ, θ1, ..., θq) are any real numbers.Again look at the properties of the process:
Mean:
E (Yt ) = E (µ) + E
"q
∑j=0
θjLj εt
#= µ
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Higher order MA processes
Variance:
V (Yt ) = γ0 = Eh(Yt � µ)2
i= E
h(εt + θ1εt�1 + θ2εt�2 + ...+ θqεt�q)
2i
= E
q
∑i=0
θi εt�iq
∑j=0
θj εt�j
!= E
q
∑j=0
θ2j ε2t�j
!, θ0 = 1
= σ2
1+
q
∑j=j
θ2j
!
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Higher order MA processes
Autocovariance:
γj = E [(Yt � µ) (Yt�j � µ)]
= E [(εt + θ1εt�1 + θ2εt�2 + ...+ θqεt�q)
� (εt�j + θ1εt�j�1 + θ2εt�j�2 + ...+ θqεt�j�q)]
First note that if j > q, γj = 0Hence assume that j � q.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Higher order MA processes
Then:
γj= E [(εt + θ1εt�1 + θ2εt�2 + ...+ θqεt�q)
� (εt�j + θ1εt�j�1 + θ2εt�j�2 + ...+ θqεt�j�q)]
= E�θj ε
2t�j + θj+1θ1ε
2t�j�1 + θj+2θ2ε
2t�j�2 + ...+ θqθq�j ε
2t�q�
= (θj + θj+1θ1 + θj+2θ2 + ...+ θqθq�j ) σ2
= σ2q
∑i=j
θi θi�j , θ0 = 1
There are q � j + 1 terms in the sum.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Higher order MA processes
In conclusion, we can write the autocovariance function as
γj =
�σ2 ∑q
i=j θi θi�j , θ0 = 1 if j � q0 if j > q
Again the process is covariance stationary as well as ergodic for themean.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Higher order MA processes
A picture of the autocorrelation function:
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
MA(oo) processes I
Consider the following MA(q) process:
Yt = µ+q
∑j=0
ψj εt�j
Consider the process we get if q ! ∞ :
Yt = µ+∞
∑j=0
ψj εt�j
This process is covariance stationary the coe¢ cients are squaresummable
∞
∑j=0
ψ2j < ∞
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
MA(oo) processes II
Sometimes absolute summability is assumed instead:
∞
∑j=0
���ψj ��� < ∞
The results are a bit harder to prove (we won�t), but they are asexpected:
E (Yt ) = µ
V (Yt ) = limq!∞
�ψ20 + ψ21 + ψ22 + ...+ ψ2q
�σ2 = σ2
∞
∑i=0
ψ2i
γj = limq!∞
σ2q
∑i=j
ψiψi�j = σ2∞
∑i=j
ψiψi�j
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
MA(oo) processes III
It turns out that an MA (∞) process with absolute summablecoe¢ cients also have absolutely summable autocovariances. Hencethe process is ergodic for the mean.
Finally, if the fεtg process is Gaussian, the process is ergodic for allmoments.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR processes
An AR(p) process is de�ned as
yt = c + φ1yt�1 + φ2yt�2 + ...+ φpyt�p + εt
where εt again is white noise.
This is exactly a q�th order di¤erence equation.
Let us consider the properties of an AR(1) process.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes I
We are considering
yt = c + φyt�1 + εt ,(1� φL) yt = c + εt
Recall that as long as jφj < 1, we can re-write this as
yt =c
1� φ+
∞
∑j=0
φj εt�j
Assume that jφj < 1 in what follows.We have now written yt as a MA (∞) process with ψj = φj .
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes II
When jφj < 1 we also know that
∞
∑j=0
���ψj ��� = ∞
∑j=0jφjj = 1
1� jφj < ∞
such that the AR(1) process is ergodic for the mean.
The mean of the AR(1) process is:
E (yt ) = E
"c
1� φ+
∞
∑j=0
φj εt�j
#=
c1� φ
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes III
The variance:
V (yt ) = E
"�yt �
c1� φ
�2#= E
24 ∞
∑j=0
φj εt�j
!235= E
"∞
∑j=0
φj εt�j∞
∑i=0
φi εt�i
#=
∞
∑j=0
φ2jσ2
= σ21
1� φ2
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes IVAnd the j 0th autocovariance is:
γj = E��yt �
c1� φ
��yt�j �
c1� φ
��= E
"∞
∑i=0
φi εt�i∞
∑l=0
φl εt�j�l
#
= E
"∞
∑i=j
φi εt�i∞
∑l=0
φl εt�j�l
#
= E
"∞
∑m=0
φm+j εt�(m+j)∞
∑l=0
φl εt�j�l
#
= E
"∞
∑m=0
φm+j εt�j�m∞
∑l=0
φl εt�j�l
#
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes V
γj = E
"∞
∑m=0
φm+j εt�j�m∞
∑l=0
φl εt�j�l
#
= E
"∞
∑l=0
φl+jφl ε2t�j�l
#= σ2φj
∞
∑l=0
φ2l
= σ2φj
1� φ2
Finally the autocorrelation is given by
ρj =σ2
φj
1�φ2
σ2 11�φ2
= φj
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes VI
Note that this is exactly the dynamic multiplier of the di¤erenceequation.
The stable AR(1) process is clearly covariance-stationary.
Graphs of AR(1) processes:
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes VII
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes
High serial correlation:
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes, alternative method I
An alternative way of �nding the moments of the process:
Assume that we are working with a stable and stationary process.
We can the use the di¤erence equation itself to determine themoments:
yt = c + φyt�1 + εt =)E (yt ) = E (c + φyt�1 + εt ) =)E (yt ) = c + φE (yt�1)
Because we assumed stationarity, we know that E (yt ) = E (yt�1) , so
E (yt ) = c + φE (yt�1) =)E (yt ) = c + φE (yt ),E (yt ) =
c1� φ
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes, alternative method II
To �nd the second moment, use the face that µ = c1�φ to re-write
the basic equation:
yt = c + φyt�1 + εt ,yt = µ (1� φ) + φyt�1 + εt ,
(yt � µ) = φ (yt�1 � µ) + εt
Now square both sides
(yt � µ)2 = φ2 (yt�1 � µ)2 + ε2t + 2φ (yt�1 � µ) εt
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes, alternative method III
And take expectations:
E (yt � µ)2 = φ2E (yt�1 � µ)2 + E�ε2t�+ 2φE [(yt�1 � µ) εt ]
γ0 = φ2γ0 + σ2 ,
γ0 =σ2
1� φ2
Finally, to �nd the autocovariances, we start from:
(yt � µ) = φ (yt�1 � µ) + εt
and multiply by (yt�j � µ) :
(yt � µ) (yt�j � µ) = φ (yt�1 � µ) (yt�j � µ) + (yt�j � µ) εt
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(1) processes, alternative method IV
Taking expectations:
E [(yt � µ) (yt�j � µ)] = φE [(yt�1 � µ) (yt�j � µ)]
+E [(yt�j � µ) εt ]
γj = φγj�1 )
γj = φjγ0 =φj
1� φ2σ2
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
Stationarity of Time Series ProcessesAR(1) Example
Consider
yt = a0 + a1yt�1 + εt
Assume that the process started at 0 with y0. Then
yt = a0t�1∑i=0ai1 + a
t1y0 +
t�1∑i=0ai1εt�1
Find the means of yt and yt�s :
E (yt ) = a0t�1∑i=0ai1 + a
t1y0
E (yt+s ) = a0t+s�1∑i=0
ai1 + at+s1 y0
These means are clearly not identical, so the series is NOT stationary.Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(p) processes I
A p0th order autoregressive process satis�es
yt = c + φ1yt�1 + φ2yt�2 + ...+ φpyt�p + εt
If the roots of
1� φ1z � φ2z1 � ...� φpz
p = 0
lie outside the unit circle we can write this process as acovariance-stationary MA (∞) process.
Quick aside: The wording "outside the unit circle".
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(p) processes II
We write
yt = µ+ ψ (L) εt
where
ψ (L) =�1� φ1L� φ2L
1 � ...� φpLp��1
and
∞
∑j=0
���ψj ��� < ∞
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(p) processes IIISince we know we are dealing with a stationary process, we can �ndthe moments from the equation itself, so
yt = c + φ1yt�1 + φ2yt�2 + ...+ φpyt�p + εt )µ = c + φ1µ+ φ2µ+ ...+ φpµ ,
µ =c
1� φ1 � φ2 � ...� φpAnd the autocovariances we get from:
yt = µ�1� φ1 � φ2 � ...� φp
�+ φ1yt�1 + ...+ φpyt�p + εt
(yt � µ) = φ1 (yt�1 � µ) + ...+ φp (yt�p � µ) + εt
(yt � µ) (yt�j � µ) = φ1 (yt�1 � µ) (yt�j � µ) + ...
+φp (yt�p � µ) (yt�j � µ) + (yt�j � µ) εt
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
AR(p) processes IV
If j = 0 :
γ0 = φ1γ�1 + ...+ φpγ�p + σ2
Because of stationarity
γ0 = φ1γ1 + ...+ φpγp + σ2
For j > 0 :
γj = φ1γj�1 + φ2γj�2 + ...+ φpγj�p
This system can be solved in a similar manner to the p0th orderdi¤erence equations.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA Processes IWe can combine AR processes with MA processes to getARMA (p, q) processes:
yt = a0 +p
∑i=1aiyt�i +
q
∑i=0bi εt�i
This too can be written as an MA (∞) process if the consitions forstability of the AR part of the process are met.:
yt �p
∑i=1aiyt�i = a0 +
q
∑i=0bi εt�i ,
1�p
∑i=1aiLi
!yt = a0 +
q
∑i=0bi εt�i ,
yt =a0 +∑q
i=0 bi εt�i∏pi=1 (L� αi )
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA Processes II
yt =a0 +∑q
i=0 bi εt�i∏pi=1 (L� αi )
Here αi , i = 1, ..., p are the roots of
1�p
∑i=1aiLi = 0
Note that
µ = E (yt ) =a0
∏pi=1 (L� αi )
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA Processes III
We can continue to re-write the process as
yt =a0 +∑q
i=0 bi εt�i∏pi=1 (L� αi )
,
yt � µ =∑qi=0 bi εt�i
∏pi=1 (L� αi )
,
yt � µ =∏qi=1 (L� βi )
∏pi=1 (L� αi )
εt
Where βi , i = 1, ..., q are the roots of
1+q
∑i=1biLi = 0
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA Processes IV
We got
yt � µ =∏qi=1 (L� βi )
∏pi=1 (L� αi )
εt
This formulation makes it clear that we can over-parametrize theprocess. If βi = αj for some i , j , we could write the same process asan ARMA(p � 1, q � 1), which has two fewer parameters.
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA(1,1) Example I
ARMA(1,1) Example:
yt = a1yt�1 + εt + β1εt�1
First we need to �nd the autocovariances.
Use Yule-Walker equations:E (ytyt ) = a1E (ytyt�1) + E (yt εt ) + β1E (yt εt�1)
E (yt�1yt ) = a1E (yt�1yt�1) + E (yt�1εt ) + β1E (yt�1εt�1)
E (yt�2yt ) = a1E (yt�2yt�1) + E (yt�2εt ) + β1E (yt�2εt�1)
::
E (yt�syt ) = a1E (yt�syt�1) + E (yt�s εt ) + β1E (yt�s εt�1)
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA(1,1) Example IIFrom the �rst equation we get
γ0 = a1γ1 + σ2 + β1E ((a1yt�1 + εt + β1εt�1) εt�1),γ0 = a1γ1 + σ2 + β1 (a1 + β1) σ2
Second equation:
γ1 = a1E (yt�1yt�1) + E (yt�1εt ) + β1E (yt�1εt�1),γ1 = a1γ0 + β1E ((a1yt�2 + εt�1 + β1εt�2) εt�1),γ1 = a1γ0 + β1σ
2
Plug into expression for γ0 :γ0 = a1
�a1γ0 + β1σ
2�+ σ2 + β1 (a1 + β1) σ2 ,γ0�1� a21
�= a1β1σ
2 + σ2 + β1 (a1 + β1) σ2
γ0 =1+ β21 + 2a1β1(1� a21)
σ2
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA(1,1) Example III
Then �nd γ1 :
γ1 = a1γ0 + β1σ2 = a1
1+ β21 + 2a1β1(1� a21)
σ2 + β1σ2
=a1 + a1β21 + 2a
21β1 + β1 � β1a
21
(1� a21)σ2
=a1 + β1 + a1β21 + a
21β1
(1� a21)σ2 =
(1+ a1β1) (a1 + β1)
(1� a21)σ2
Now look at the third equation:
E (yt�2yt ) = a1E (yt�2yt�1) + E (yt�2εt ) + β1E (yt�2εt�1),γ2 = a1γ1
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA(1,1) Example IV
Fourth:
E (yt�3yt ) = a1E (yt�3yt�1) + E (yt�3εt ) + β1E (yt�3εt�1),γ3 = a1γ2
And s�th:
E (yt�syt ) = a1E (yt�syt�1) + E (yt�s εt ) + β1E (yt�s εt�1),γs = a1γs�1
Note that only the �rst few are hard.
Now for the autocorrelations. First note:qV (yt )V (yt�s ) =
qV (yt )V (yt ) = V (yt ) = γ0
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA(1,1) Example VThen calculate the �rst autocorrelation:
ρ1 =γ1p
V (yt )V (yt�1)=
γ1γ0=
(1+a1β1)(a1+β1)
(1�a21)σ2
1+β21+2a1β1(1�a21)
σ2
=(1+ a1β1) (a1 + β1)
1+ β21 + 2a1β1The second:
ρ2 =γ2p
V (yt )V (yt�2)=
γ2γ0=a1γ1γ0
= a1ρ1
The third:
ρ3 =γ3p
V (yt )V (yt�3)=
γ3γ0=a1γ2γ0
= a1ρ2 = a21ρ1
Helle Bunzel ISU
Stationary ARMA processes
De�nitions and Review MA processes AR processes ARMA Processes
ARMA(1,1) Example VI
Similarly for all s � 2,
ρs = a1ρs�1 = as�11 ρ1
Helle Bunzel ISU
Stationary ARMA processes