CRF-Filters: Discriminative Particle Filters for Sequential State Estimation

CRF-F: D P F S S E

B L, D F L L

Hannes Schulz

University of Freiburg, ACS

Feb 2008

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

Intro Transformation of Directed Model to CRF Application Experimental Results

C: S EC DM A S E

ut−2 ut−1

xt−1 xt

z1t−1 z2

t−1 znt−1 z1

t z2t zn

t

. . . . . .

xt−2

P(xt |u1:t−1, z1:t) = ηP(zt |xt)

∫P(xt |ut−1, xt−1)P(xt−1|u1:t−2, z1:t−1) dxt−1

Intro Transformation of Directed Model to CRF Application Experimental Results

D DM P

p(zt |xt) =∏n

i=1 p(z it |xt ) p(xt+1|xt , u)

u = (δrot1, δrot2, δtrans)executed with gaussiannoise

Intro Transformation of Directed Model to CRF Application Experimental Results

D DM P

p(zt |xt) =∏n

i=1 p(z it |xt )

P (zit|xt) zi

t zmax

zrand

p(xt+1|xt , u)

u = (δrot1, δrot2, δtrans)executed with gaussiannoise

Intro Transformation of Directed Model to CRF Application Experimental Results

D DM P

p(zt |xt) =∏n

i=1 p(z it |xt )

P (zit|xt) zi

t zmax

zrand

p(xt+1|xt , u)

u = (δrot1, δrot2, δtrans)executed with gaussiannoise

Intro Transformation of Directed Model to CRF Application Experimental Results

D DM P

p(zt |xt) =∏n

i=1 p(z it |xt )

P (zit|xt) zi

t zmax

zrand

p(xt+1|xt , u)

δrot1

δrot2

δtrans

xt−1

xt

u = (δrot1, δrot2, δtrans)executed with gaussiannoise

Intro Transformation of Directed Model to CRF Application Experimental Results

A P D A

xt

zt

p(z it |xt) are not cond. independent

Sensor models can only begenerated seperatly for each beamAssumption that measurementsare independent: “Workssurprisingly well”. . . if. . .

increasing uncertainty (tweaking)using every 10th measurement. . .

Intro Transformation of Directed Model to CRF Application Experimental Results

A P D A

ut−2 ut−1

xt−1 xt

z1t−1 z2

t−1 znt−1 z1

t z2t zn

t

. . . . . .

xt−2

P (zit|xt) zi

t zmax

zrand

p(z it |xt) are not cond. independent

Sensor models can only begenerated seperatly for each beam

Assumption that measurementsare independent: “Workssurprisingly well”. . . if. . .

increasing uncertainty (tweaking)using every 10th measurement. . .

Intro Transformation of Directed Model to CRF Application Experimental Results

A P D A

ut−2 ut−1

xt−1 xt

z1t−1 z2

t−1 znt−1 z1

t z2t zn

t

. . . . . .

xt−2

P (zit|xt) zi

t zmax

zrand

p(z it |xt) are not cond. independent

Sensor models can only begenerated seperatly for each beamAssumption that measurementsare independent: “Workssurprisingly well”. . . if. . .

increasing uncertainty (tweaking)using every 10th measurement. . .

Intro Transformation of Directed Model to CRF Application Experimental Results

A P D A

ut−2 ut−1

xt−1 xt

z1t−1 z2

t−1 znt−1 z1

t z2t zn

t

. . . . . .

xt−2

P (zit|xt) zi

t zmax

zrand

p(z it |xt) are not cond. independent

Sensor models can only begenerated seperatly for each beamAssumption that measurementsare independent: “Workssurprisingly well”. . . if. . .

increasing uncertainty (tweaking)using every 10th measurement. . .

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

Intro Transformation of Directed Model to CRF Application Experimental Results

I: CRF

ut−2 ut−1

xt−1 xt

zt−1 zt

xt−2

Undirected graphical models

Every (possible) dependencyrepresented by edge

Distribution defined over productsof functions over cliques

Functions are called cliquepotentials

Clique potentials representcompatibility of their variables

Intro Transformation of Directed Model to CRF Application Experimental Results

I: CRF

ut−2 ut−1

xt−1 xt

zt−1 zt

xt−2

Undirected graphical models

Every (possible) dependencyrepresented by edge

Distribution defined over productsof functions over cliques

Functions are called cliquepotentials

Clique potentials representcompatibility of their variables

Intro Transformation of Directed Model to CRF Application Experimental Results

I: CRF

ut−2 ut−1

xt−1 xt

zt−1 zt

xt−2

Undirected graphical models

Every (possible) dependencyrepresented by edge

Distribution defined over productsof functions over cliques

Functions are called cliquepotentials

Clique potentials representcompatibility of their variables

Intro Transformation of Directed Model to CRF Application Experimental Results

I: CRF

ut−2 ut−1

xt−1 xt

zt−1 zt

xt−2

Undirected graphical models

Every (possible) dependencyrepresented by edge

Distribution defined over productsof functions over cliques

Functions are called cliquepotentials

Clique potentials representcompatibility of their variables

Intro Transformation of Directed Model to CRF Application Experimental Results

I: CRF

ut−2 ut−1

xt−1 xt

zt−1 zt

xt−2

Undirected graphical models

Every (possible) dependencyrepresented by edge

Distribution defined over productsof functions over cliques

Functions are called cliquepotentials

Clique potentials representcompatibility of their variables

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

Intro Transformation of Directed Model to CRF Application Experimental Results

CRF-M S E

ut−2 ut−1

xt−1 xt

zt−1 zt

xt−2

p(x0:T |z1:T ,u0:T−1) =1

Z(z1:T ,u1:T−1)

T∏t=1

ϕp(xt , xt−1,ut−1)ϕm(xt , zt)

Z(·):∑

all trajectories∏ϕp(·)ϕm(·)

How to define ϕp(·) and ϕm(·)?

Intro Transformation of Directed Model to CRF Application Experimental Results

CRF-M S E

ut−2 ut−1

xt−1 xt

zt−1 zt

xt−2

p(x0:T |z1:T ,u0:T−1) =1

Z(z1:T ,u1:T−1)

T∏t=1

ϕp(xt , xt−1,ut−1)ϕm(xt , zt)

Z(·):∑

all trajectories∏ϕp(·)ϕm(·)

How to define ϕp(·) and ϕm(·)?

Intro Transformation of Directed Model to CRF Application Experimental Results

CRF-M S E

ut−2 ut−1

xt−1 xt

zt−1 zt

xt−2

p(x0:T |z1:T ,u0:T−1) =1

Z(z1:T ,u1:T−1)

T∏t=1

ϕp(xt , xt−1,ut−1)ϕm(xt , zt)

Z(·):∑

all trajectories∏ϕp(·)ϕm(·)

How to define ϕp(·) and ϕm(·)?

Intro Transformation of Directed Model to CRF Application Experimental Results

T P P φp

δrot1

δrot2

δtrans

xt−1

xt

ut−1 = (δrot1, δtrans , δrot2) odometry

ut−1 = (δrot1, δtrans , δrot2) derived odometry

. Before: Gaussian noise N(ui

t−1, σi2)

fp(xt , xt−1, ut−1) =

(δrot1 − δrot1)

2

(δtrans − δtrans)2

(δrot2 − δrot2)2

3 features

φp(xt , xt−1, ut−1) = exp{〈wp , fp(xt , xt−1, ut−1)〉

}N

(a,

1σ2

)= exp

(−

(a − a)2

2σ2

)

. Gaussian noise N(ui

t−1,1−2w i

p

)if w i

p < 0

Intro Transformation of Directed Model to CRF Application Experimental Results

T P P φp

δrot1

δrot2

δtrans

xt−1

xt

ut−1 = (δrot1, δtrans , δrot2) odometry

ut−1 = (δrot1, δtrans , δrot2) derived odometry

. Before: Gaussian noise N(ui

t−1, σi2)

fp(xt , xt−1, ut−1) =

(δrot1 − δrot1)

2

(δtrans − δtrans)2

(δrot2 − δrot2)2

3 features

φp(xt , xt−1, ut−1) = exp{〈wp , fp(xt , xt−1, ut−1)〉

}N

(a,

1σ2

)= exp

(−

(a − a)2

2σ2

)

. Gaussian noise N(ui

t−1,1−2w i

p

)if w i

p < 0

Intro Transformation of Directed Model to CRF Application Experimental Results

T P P φp

δrot1

δrot2

δtrans

xt−1

xt

ut−1 = (δrot1, δtrans , δrot2) odometry

ut−1 = (δrot1, δtrans , δrot2) derived odometry

. Before: Gaussian noise N(ui

t−1, σi2)

fp(xt , xt−1, ut−1) =

(δrot1 − δrot1)

2

(δtrans − δtrans)2

(δrot2 − δrot2)2

3 features

φp(xt , xt−1, ut−1) = exp{〈wp , fp(xt , xt−1, ut−1)〉

}

N

(a,

1σ2

)= exp

(−

(a − a)2

2σ2

)

. Gaussian noise N(ui

t−1,1−2w i

p

)if w i

p < 0

Intro Transformation of Directed Model to CRF Application Experimental Results

T P P φp

δrot1

δrot2

δtrans

xt−1

xt

ut−1 = (δrot1, δtrans , δrot2) odometry

ut−1 = (δrot1, δtrans , δrot2) derived odometry

. Before: Gaussian noise N(ui

t−1, σi2)

fp(xt , xt−1, ut−1) =

(δrot1 − δrot1)

2

(δtrans − δtrans)2

(δrot2 − δrot2)2

3 features

φp(xt , xt−1, ut−1) = exp{〈wp , fp(xt , xt−1, ut−1)〉

}N

(a,

1σ2

)= exp

(−

(a − a)2

2σ2

)

. Gaussian noise N(ui

t−1,1−2w i

p

)if w i

p < 0

Intro Transformation of Directed Model to CRF Application Experimental Results

R: SM N B A

P (zit|xt) zi

t zmax

zrand

p(zt |xt) =n∏

i=1

p(z it |xt)

Intro Transformation of Directed Model to CRF Application Experimental Results

M P φm

φm(xt , zt) = exp

⟨wm,

n∑i=0

f im(zt , xt)

⟩

f im(zt , xt) =

(¬mit ∧ ¬mi

t)cit(z

it − z i

t)2

(¬mit ∧ ¬mi

t)¬c it

(¬mit ∧ mi

t)

( mit ∧ ¬mi

t)

( mit ∧ mi

t)

mit ∈ {1, 0} measured zmax

mit ∈ {1, 0} expected zmax

c it ∈ {1, 0} z i

t − z it < 20cm

P (zit|xt) zi

t zmax

zrand

Intro Transformation of Directed Model to CRF Application Experimental Results

M P φm

φm(xt , zt) = exp

⟨wm,

n∑i=0

f im(zt , xt)

⟩

f im(zt , xt) =

(¬mit ∧ ¬mi

t)cit(z

it − z i

t)2

(¬mit ∧ ¬mi

t)¬c it

(¬mit ∧ mi

t)

( mit ∧ ¬mi

t)

( mit ∧ mi

t)

mi

t ∈ {1, 0} measured zmax

mit ∈ {1, 0} expected zmax

c it ∈ {1, 0} z i

t − z it < 20cm

P (zit|xt) zi

t zmax

zrand

Intro Transformation of Directed Model to CRF Application Experimental Results

M P φm

φm(xt , zt) = exp

⟨wm,

n∑i=0

f im(zt , xt)

⟩

f im(zt , xt) =

(¬mit ∧ ¬mi

t)cit(z

it − z i

t)2

(¬mit ∧ ¬mi

t)¬c it

(¬mit ∧ mi

t)

( mit ∧ ¬mi

t)

( mit ∧ mi

t)

mi

t ∈ {1, 0} measured zmax

mit ∈ {1, 0} expected zmax

c it ∈ {1, 0} z i

t − z it < 20cm

P (zit|xt) zi

t zmax

zrand

Intro Transformation of Directed Model to CRF Application Experimental Results

M P φm

φm(xt , zt) = exp

⟨wm,

n∑i=0

f im(zt , xt)

⟩

f im(zt , xt) =

(¬mit ∧ ¬mi

t)cit(z

it − z i

t)2

(¬mit ∧ ¬mi

t)¬c it

(¬mit ∧ mi

t)

( mit ∧ ¬mi

t)

( mit ∧ mi

t)

mi

t ∈ {1, 0} measured zmax

mit ∈ {1, 0} expected zmax

c it ∈ {1, 0} z i

t − z it < 20cm

P (zit|xt) zi

t zmax

zrand

Intro Transformation of Directed Model to CRF Application Experimental Results

M P φm

φm(xt , zt) = exp

⟨wm,

n∑i=0

f im(zt , xt)

⟩

f im(zt , xt) =

(¬mit ∧ ¬mi

t)cit(z

it − z i

t)2

(¬mit ∧ ¬mi

t)¬c it

(¬mit ∧ mi

t)

( mit ∧ ¬mi

t)

( mit ∧ mi

t)

mi

t ∈ {1, 0} measured zmax

mit ∈ {1, 0} expected zmax

c it ∈ {1, 0} z i

t − z it < 20cm

P (zit|xt) zi

t zmax

zrand

Intro Transformation of Directed Model to CRF Application Experimental Results

M P φm

φm(xt , zt) = exp

⟨wm,

n∑i=0

f im(zt , xt)

⟩

f im(zt , xt) =

(¬mit ∧ ¬mi

t)cit(z

it − z i

t)2

(¬mit ∧ ¬mi

t)¬c it

(¬mit ∧ mi

t)

( mit ∧ ¬mi

t)

( mit ∧ mi

t)

mi

t ∈ {1, 0} measured zmax

mit ∈ {1, 0} expected zmax

c it ∈ {1, 0} z i

t − z it < 20cm

P (zit|xt) zi

t zmax

zrand

Intro Transformation of Directed Model to CRF Application Experimental Results

M P φm

φm(xt , zt) = exp

⟨wm,

n∑i=0

f im(zt , xt)

⟩

f im(zt , xt) =

(¬mit ∧ ¬mi

t)cit(z

it − z i

t)2

(¬mit ∧ ¬mi

t)¬c it

(¬mit ∧ mi

t)

( mit ∧ ¬mi

t)

( mit ∧ mi

t)

mi

t ∈ {1, 0} measured zmax

mit ∈ {1, 0} expected zmax

c it ∈ {1, 0} z i

t − z it < 20cm

P (zit|xt) zi

t zmax

zrand

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

Intro Transformation of Directed Model to CRF Application Experimental Results

U CRF P F

At each time step t :Prediction

Move particles according to gaussian noisedetermined by wp

Same as sampling from N(ui

t−1,1−2w i

p

)Correction

Particle at xt gets weight φm(xt , zt)Resample (includes normalization)

Intro Transformation of Directed Model to CRF Application Experimental Results

U CRF P F

At each time step t :Prediction

Move particles according to gaussian noisedetermined by wp

Same as sampling from N(ui

t−1,1−2w i

p

)Correction

Particle at xt gets weight φm(xt , zt)Resample (includes normalization)

u

Intro Transformation of Directed Model to CRF Application Experimental Results

U CRF P F

At each time step t :Prediction

Move particles according to gaussian noisedetermined by wp

Same as sampling from N(ui

t−1,1−2w i

p

)Correction

Particle at xt gets weight φm(xt , zt)Resample (includes normalization)

movedparticles

Intro Transformation of Directed Model to CRF Application Experimental Results

U CRF P F

At each time step t :Prediction

Move particles according to gaussian noisedetermined by wp

Same as sampling from N(ui

t−1,1−2w i

p

)Correction

Particle at xt gets weight φm(xt , zt)Resample (includes normalization)

addednoise

Intro Transformation of Directed Model to CRF Application Experimental Results

U CRF P F

At each time step t :Prediction

Move particles according to gaussian noisedetermined by wp

Same as sampling from N(ui

t−1,1−2w i

p

)Correction

Particle at xt gets weight φm(xt , zt)Resample (includes normalization)

...sense...

Intro Transformation of Directed Model to CRF Application Experimental Results

U CRF P F

At each time step t :Prediction

Move particles according to gaussian noisedetermined by wp

Same as sampling from N(ui

t−1,1−2w i

p

)Correction

Particle at xt gets weight φm(xt , zt)Resample (includes normalization)

weights

Intro Transformation of Directed Model to CRF Application Experimental Results

U CRF P F

At each time step t :Prediction

Move particles according to gaussian noisedetermined by wp

Same as sampling from N(ui

t−1,1−2w i

p

)Correction

Particle at xt gets weight φm(xt , zt)Resample (includes normalization)

resample

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

Intro Transformation of Directed Model to CRF Application Experimental Results

D wp wm

Drive around in test area

Use high-quality scanmatcher to generate“ground truth” trajectory x∗

Using arbitrary weights, generate trajectory xwith CRF-filter

Use difference of summed features as weightupdate(−):wk = wk−1 + α (

∑f(x∗,u, z) −

∑f(x,u, z))

Decrease α if new Filter cannot track

loop

Adapts weights to task, sensor dependencies/environment,sensor noise, particle filter parameters

Intro Transformation of Directed Model to CRF Application Experimental Results

D wp wm

Drive around in test area

Use high-quality scanmatcher to generate“ground truth” trajectory x∗

Using arbitrary weights, generate trajectory xwith CRF-filter

Use difference of summed features as weightupdate(−):wk = wk−1 + α (

∑f(x∗,u, z) −

∑f(x,u, z))

Decrease α if new Filter cannot track

loop

Adapts weights to task, sensor dependencies/environment,sensor noise, particle filter parameters

Intro Transformation of Directed Model to CRF Application Experimental Results

D wp wm

Drive around in test area

Use high-quality scanmatcher to generate“ground truth” trajectory x∗

Using arbitrary weights, generate trajectory xwith CRF-filter

Use difference of summed features as weightupdate(−):wk = wk−1 + α (

∑f(x∗,u, z) −

∑f(x,u, z))

Decrease α if new Filter cannot track

loop

Adapts weights to task, sensor dependencies/environment,sensor noise, particle filter parameters

Intro Transformation of Directed Model to CRF Application Experimental Results

D wp wm

Drive around in test area

Use high-quality scanmatcher to generate“ground truth” trajectory x∗

Using arbitrary weights, generate trajectory xwith CRF-filter

Use difference of summed features as weightupdate(−):wk = wk−1 + α (

∑f(x∗,u, z) −

∑f(x,u, z))

Decrease α if new Filter cannot track

loop

Adapts weights to task, sensor dependencies/environment,sensor noise, particle filter parameters

Intro Transformation of Directed Model to CRF Application Experimental Results

D wp wm

Drive around in test area

Use high-quality scanmatcher to generate“ground truth” trajectory x∗

Using arbitrary weights, generate trajectory xwith CRF-filter

Use difference of summed features as weightupdate(−):wk = wk−1 + α (

∑f(x∗,u, z) −

∑f(x,u, z))

Decrease α if new Filter cannot track

loop

Adapts weights to task, sensor dependencies/environment,sensor noise, particle filter parameters

Intro Transformation of Directed Model to CRF Application Experimental Results

D wp wm

Drive around in test area

Use high-quality scanmatcher to generate“ground truth” trajectory x∗

Using arbitrary weights, generate trajectory xwith CRF-filter

Use difference of summed features as weightupdate(−):wk = wk−1 + α (

∑f(x∗,u, z) −

∑f(x,u, z))

Decrease α if new Filter cannot track

loop

Adapts weights to task, sensor dependencies/environment,sensor noise, particle filter parameters

Intro Transformation of Directed Model to CRF Application Experimental Results

L A

Averaged Perceptron Algorithm (Collins 2002) for tagging

wk = wk−1 + α(∑

f(x∗,u, z) −∑

f(x,u, z))

Proven to converge even in presence of errors in training dataIntuition of learning algorithm:

If PF works correctly, then∑f(x∗n , un−1, zn) =

∑f(xn, un−1, zn)

f i occurs less often in x∗ than in x→ decrease influence of f i

on particle filter by decreasing w i

O

1 I: S E U DM

2 T DM CRF

Short Introduction to CRF

CRF-Model for State Estimation

3 A

CRF-Filter Algorithm

Learning the Parameters

4 E R

Intro Transformation of Directed Model to CRF Application Experimental Results

E R

Properties of the learned weightsNorm of weight vector decreases withnumber of laser beams in z

. believes the features/measurements less

. equivalent to initially introduced“tweaking”?!

Two specialized CRF-filters compared togenerative particle filter trained usingexpectation maximization

TrackingError

GlobalLocalizationAccuracy

Generative 7.52 cm 30%CRF-Filter 7.07 cm 96%

Intro Transformation of Directed Model to CRF Application Experimental Results

E R

Properties of the learned weightsNorm of weight vector decreases withnumber of laser beams in z

. believes the features/measurements less

. equivalent to initially introduced“tweaking”?!

Two specialized CRF-filters compared togenerative particle filter trained usingexpectation maximization

TrackingError

GlobalLocalizationAccuracy

Generative 7.52 cm 30%CRF-Filter 7.07 cm 96%

Intro Transformation of Directed Model to CRF Application Experimental Results

C

1 A CRF is an alternative, undirected graphical model

2 CRF-Filters use a continuous CRF for recursive stateestimation

3 . . . can be trained to maximize filter performance dependingon the task

4 . . . can deal with correlated measurements5 . . . do not explicitly account for dependencies between sensor

data

Intro Transformation of Directed Model to CRF Application Experimental Results

C

1 A CRF is an alternative, undirected graphical model2 CRF-Filters use a continuous CRF for recursive state

estimation

3 . . . can be trained to maximize filter performance dependingon the task

4 . . . can deal with correlated measurements5 . . . do not explicitly account for dependencies between sensor

data

Intro Transformation of Directed Model to CRF Application Experimental Results

C

1 A CRF is an alternative, undirected graphical model2 CRF-Filters use a continuous CRF for recursive state

estimation3 . . . can be trained to maximize filter performance depending

on the task

4 . . . can deal with correlated measurements5 . . . do not explicitly account for dependencies between sensor

data

Intro Transformation of Directed Model to CRF Application Experimental Results

C

1 A CRF is an alternative, undirected graphical model2 CRF-Filters use a continuous CRF for recursive state

estimation3 . . . can be trained to maximize filter performance depending

on the task4 . . . can deal with correlated measurements

5 . . . do not explicitly account for dependencies between sensordata

Intro Transformation of Directed Model to CRF Application Experimental Results

C

1 A CRF is an alternative, undirected graphical model2 CRF-Filters use a continuous CRF for recursive state

estimation3 . . . can be trained to maximize filter performance depending

on the task4 . . . can deal with correlated measurements5 . . . do not explicitly account for dependencies between sensor

data