Time-Varying Temporal Dependene in Autoregressive Models - Francisco Blasques, Siem Jan Koopman, Andre Lucas. June 2014

Time-Varying Temporal Dependen e

in Autoregressive Models

F. Blasques S.J. Koopman A. Lu as

VU University Amsterdam, Tinbergen Institute, CREATES

International Asso iation for Applied E onometri s

2014 Annual Conferen e

Queen Mary, University of London, 26-28 June 2014

1 / 1 Blasques, Koopman and Lu as Time-Varying Temporal Dependen e

Motivation

For an observable time series y1, . . . , yT , we onsider the

standard autoregressive model of order one, the AR(1) model

yt = ϕyt−1 + ut, ut ∼ pu(ut;λ), t = 1, . . . , T,

where ϕ is the autoregressive oe� ient with stationary

ondition −1 < ϕ < 1 and where ut is the random error with

density fun tion pu(ut;λ) and parameter ve tor λ.

In many appli ations in e onomi s and �nan e, there is a quest

for ϕ to be time-varying, we have

yt = ϕtyt−1 + ut, ut ∼ pu(ut;λ), t = 1, . . . , T,

but what is an appropriate dynami spe i� ation for ϕt ?


Motivation

The time-varying dependen e in an AR(1) model,

yt = ϕtyt−1 + ut, ut ∼ pu(ut;λ), t = 1, . . . , T,

an be modelled expli itly via the link fun tion

ϕt = h(αt),

where αt is spe i�ed as another dynami pro ess, say

αt = Φαt−1 + ηt, ηt ∼ pη(ηt, λ), t = 1, . . . , T.

We regard this system of dynami equations as a onditional,

and possibly nonlinear, state spa e model.

Kalman �lter and maximum likelihood methods an be used.


Motivation

The time-varying dependen e in an AR(1) pro ess :

yt = h(αt)yt−1 + ut, ut ∼ pu(ut;λ),

αt = Φαt−1 + ηt,

This framework is basi but inferen e an be involved.

As it is typi ally the ase for parameter-driven models.

One often relies on estimation within a Bayesian framework,

espe ially when one onsiders ve tor autoregressive models; see

the extensive Bayesian VAR literature, e.g. Kadiyala &

Karlsson, Koop & Korolibis, Banbura, Giannone & Rei hlin,

Clark & M Cra ken, Carriero Kapetanios & Mar ellino, et .


Our paper

We present an observation-driven model spe i� ation for the

time-varying dependen y in autoregressive models.

For the AR(1) ase we have

yt = h(ft;λ)yt−1 + ut, ut ∼ pu(ut;λ),

ft = φ(yt−1, f t−1;λ),

where both h() and φ() are �xed fun tions, both possibly

depending on a �xed parameter ve tor λ, with

xt = {xt, xt−1, xt−2, . . .} for x = f, y.

AR(1) model is now general and �exible but we need to spe ify

φ(yt−1, f t−1;λ), pu(ut;λ), h(ft;λ).


Time-varying temporal dependen e in AR(1) model

For the general and �exible AR(1) model


ft = φ(yt−1, f t−1;λ),

we take the linear updating equation

φ(yt−1, f t−1;λ) = ω + αs(yt−1, f t−1;λ) + βft−1,

where ω, α and β are �xed oe� ients and s(·, ·) is a

deterministi fun tion of past observations.

In parti ular, we take s(·, ·) as the s ore fun tion of the

onditional or predi tive log-density fun tion of yt,

log p(yt|ft, yt−1;λ) ≡ log pu(ut;λ),

as ut = yt − h(ft;λ)yt−1, with respe t to ft.6 / 1 Blasques, Koopman and Lu as Time-Varying Temporal Dependen e


Our time-varying temporal dependen e AR(1) model is given by


ft = ω + αs(yt−1, f t−1;λ) + βft−1,

with s ore fun tion

st ≡ s(yt, f t;λ) =∂ log p(yt|ft, y

t−1;λ)

∂ft.

In spirit of GAS model : Creal, Koopman & Lu as (2008,11,13).

Why the s ore ? It is optimal in Kullba k-Leibler sense, later !

But what about the hoi e for pu(ut;λ) and h(ft;λ) ?



Our time-varying temporal dependen e AR(1) model is given by


ft = ω + αst−1 + βft−1,

where also s ore fun tion

st =∂ log pu(ut;λ)

∂ft.

depends on hoi e

h(ft;λ) → ft logit(ft)

pu(ut;λ)

↓

Normal X ·

Student's t ·


Basi time-varying temporal dependen e

The linear Gaussian updating ase

yt = ft × yt−1 + ut, ut ∼ N(0, σ2u),

ft = ω + αst−1 + βft−1,

with s ore fun tion

st =∂[c− 0.5(yt − ftyt−1)

2/σ2u]

∂ft

= (yt − ftyt−1)(yt−1/σ2u) = utyt−1/σ

2u.

The time-varying autoregressive parameter updating equation is

ft = ω + αut−1yt−2

σ2u

+ βft−1.



We have the model

yt = ft × yt−1 + ut, ft = ω + αut−1yt−2

σ2u

+ βft−1.

Interesting interpretation :

update of ft rea ts to error ut−1 multiplied by yt−2 and

s aled by σ−2u .

role of yt−2 is to signal whether ft is below or above its

mean.

update distinguishes role of observed past data and of past

parameter value.

More interesting/intrinsi updating equations for other pu and h



−2 0 2 4−0.2

0.4

1

−2 0 2 4

−2 0 2 4

N-GAS (β = 0.5) N-GAS (β = 1) t-GAS (β = 0.5) t-GAS (β = 1)

yt−2yt−2 yt−2

yt−1 yt−1 yt−1

ft−1

ft

Figure: Updating for ft: h(f) = f and pu(u) = N , t.


Redu ed form

Our basi time-varying temporal dependen e model


σ2u

+ βft−1,

is e�e tively a nonlinear ARMA model

yt =

{

ω + β

[

yt−1 − ut−1

yt−1

]}

yt−1 + ut + α

[

yt−1

σ2u

]

ut−1.

It is a nonlinear ARMA(2, 1) !

Some minor algebra is required to obtain this result.

Similar results an be obtained for the other ases.


Nonlinear ARMA models

We have shown that our basi time-varying temporal

dependen e model is a nonlinear ARMA model.

But what is new ? So many other nonlinear ARMA models !

Threshold AR � Tong (1991)

yt = ϕt × yt−1 + ut, ϕt = ϕ+ ϕ∗I(yt−2 < γ),

Smooth Transition � Chan & Tong (1986), Teräsvirta (1994)

yt = ϕt × yt−1 + ut, ϕt = γ1xt−2 + γ2(1− xt−2),

where xt = [1 + exp(−γ3 yt)]−1.


Comparison with TAR and STAR


Comparison with our basi model


Properties

S ore fun tion : Familiar entity in e onometri s.

Maximum likelihood : Consisten y and Asymptoti

Normality, onditions an be established.

Optimality : Updating using s ore provides a step loser to

the true path of the time-varying parameter, optimality in the

Kullba k-Leibler sense.

Result 1: Only parameter updates based on the s ore always

redu e the lo al Kullba k-Leibler divergen e.

Result 2: The use of the s ore leads to onsiderably smaller

global KL divergen e in empiri ally relevant settings.

Noti e: These results hold generally for any DGP.


Optimal Observation-Driven Update

Key obje tive: Chara terize φ(·) that possess optimality

properties from information theoreti point of view.

Main Question: Is there an optimal form for the update

f̃t+1 = φ(

yt , f̃t ; θ)

, ∀ t ∈ N, f̃1 ∈ F ⊆ R,

Answer: This depends on the notion of optimality!

Result 1: Only parameter updates based on the s ore always

redu e the lo al Kullba k-Leibler divergen e p and p̃.

Result 2: The use of the s ore leads to onsiderably smaller

global KL divergen e in empiri ally relevant settings.

Note: Results hold for any DGP ( any p and {ft} )


De�nitions: Lo al GAS Updates

GAS-update:

f̃t+1 = φ(

yt , f̃t ; θ)

= ω + αs(yt, f̃t) + βf̃t, ∀ t ∈ N,

Newton-GAS update: ( ω = 0, α > 0, β = 1 )

f̃t+1 = αs(yt, f̃t) + f̃t, ∀ t ∈ N,

Lo al update: f̃t+1 in neighborhood of f̃t

Lo al optimality: Refers to lo al updates


De�nition I: Realized KL Divergen e

KL divergen e between p(·|ft) and p̃(

· |f̃t+1;θ)

is given by

DKL

(

p(·|ft) , p̃(

· |f̃t+1;θ)

)

=

∫

∞

−∞

p(yt|ft) lnp(yt|ft)

p̃(

yt|f̃t+1;θ) dyt.


De�nition I: Realized KL Divergen e

KL divergen e between p(·|ft) and p̃(

· |f̃t+1;θ)

is given by

DKL

(

p(·|ft) , p̃(

· |f̃t+1;θ)

)

=

∫

∞

−∞

p(yt|ft) lnp(yt|ft)

p̃(

yt|f̃t+1;θ) dyt.

The realized KL variation ∆t−1RKL

of a parameter update from f̃t

to f̃t+1 is de�ned as

∆t−1RKL

= DKL

(

p(·|ft) , p̃(

· |f̃t+1;θ)

)

−DKL

(

p(·|ft) , p̃(

· |f̃t;θ)

)


De�nition II: Conditionally Expe ted KL Divergen e

An optimal updating s heme, while subje t to randomness,

should have tenden y to move in orre t dire tion:

On average, the KL divergen e should redu e in expe tation.


De�nition II: Conditionally Expe ted KL Divergen e

An optimal updating s heme, while subje t to randomness,

should have tenden y to move in orre t dire tion:

On average, the KL divergen e should redu e in expe tation.

The onditionally expe ted KL (CKL) variation of a parameter

update from f̃t ∈ F̃ to f̃t+1 ∈ F̃ is given by

∆t−1CKL

=

∫

F

q(f̃t+1|f̃t, ft;θ)

[

∫

Y

p(y|ft) lnp̃(y|f̃t;θ)

p̃(y|f̃t+1;θ)dy

]

df̃t+1,

where q(f̃t+1|f̃t, ft;θ) denotes the density of f̃t+1 onditional on

both f̃t and ft. For a given pt, an update is CKL optimal if and

only if ∆t−1CKL

≤ 0.


Ba k to our basi model : ondition for RKL and CKL

Our basi time-varying temporal dependen e model


σ2u

+ βft−1,

we obtain RKL optimality under the ondition

α > σ2u

|ω + (β − 1)f̃t|

|(yt−1 − f̃t−1yt−2)yt−2|,

The new s ore information should have lo ally su� ient impa t

on the updating for ft.

A similar but di�erent ondition is derived for CKL optimality.



Empiri al illustration: Unemployment Insuran e Claims

We analyze the growth rate of US seasonally adjusted weekly

Unemployment Insuran e Claims (UIC) for roughly the last �ve

de ades.

Meyer (1995), Anderson & Meyer (1997, 2000), Hopenhayn &

Ni olini (1997) and Ashenfelter (2005) have studied the UIC

series.

The importan e of fore asting UIC has been highlighted by

Gavin & Kliesen (2002):

UIC is a leading indi ator for several labor market onditions:

how they an be used to fore asting GDP growth rates.

Here we onsider various models and do some omparisons

amongst them.


Empiri al illustration

Unemployment Insuran e Claims: Model Comparison

(N) AR-GAS TAR STAR AR(2) AR(5)

LL 6744 6736 6737 6439 6968

AIC -13478 -13462 -13464 -12870 -13921

RMSE 0.750 0.752 0.752 0.848 1.20


Empiri al results

2005 2006 2007 2008 2009 2010 2011 2012 2013

−0.05

0

0.05

Uemployment Insurance Claims

2005 2006 2007 2008 2009 2010 2011 2012 2013

0.4

0.5

0.6

Normal AR−GAS (Identity Link and Unit Scaling)


Con lusions

We have introdu ed time-varying temporal dependen e in the

AR(1) model


σ2u

+ βft−1,

an observation-driven approa h to time-varying

autoregressive oe� ient: GAS model is e�e tive !

redu ed form : nonlinear ARMA models

they an be ompared with TAR and STAR models

the �ltered estimate ft has optimality properties in the KL

sense when based on the s ore fun tion !

we provide some Monte Carlo eviden e

an empiri al illustration for UIC is presented


This project has received funding from the European Union’s

Seventh Framework Programme for research, technological

development and demonstration under grant agreement no° 320270

www.syrtoproject.eu

This document reflects only the author’s views.

The European Union is not liable for any use that may be made of the information contained therein.

Economy & Finance

Time-Varying Temporal Dependene in Autoregressive Models - Francisco Blasques, Siem Jan Koopman, Andre Lucas. June 2014