View
216
Download
0
Category
Preview:
Citation preview
Discrete Optimization Lecture 2 – Part 2
M. Pawan Kumar
pawan.kumar@ecp.fr
Slides available online http://cvn.ecp.fr/personnel/pawan/
• Integer Programming Formulation
• LP Relaxation
Outline
Integer Programming Formulation
Va Vb
Label l0
Label l12
5
4
2
0
1 1
0
2Unary Potentials
a;0 = 5
a;1 = 2
b;0 = 2
b;1 = 4
Labeling
f(a) = 1
f(b) = 0
ya;0 = 0 ya;1 = 1
yb;0 = 1 yb;1 = 0
Any f(.) has equivalent boolean variables ya;i
Integer Programming Formulation
Va Vb
2
5
4
2
0
1 1
0
2Unary Potentials
a;0 = 5
a;1 = 2
b;0 = 2
b;1 = 4
Labeling
f(a) = 1
f(b) = 0
ya;0 = 0 ya;1 = 1
yb;0 = 1 yb;1 = 0
Find the optimal variables ya;i
Label l0
Label l1
Integer Programming Formulation
Va Vb
2
5
4
2
0
1 1
0
2Unary Potentials
a;0 = 5
a;1 = 2
b;0 = 2
b;1 = 4
Sum of Unary Potentials
∑a ∑i a;i ya;i
ya;i {0,1}, for all Va, li∑i ya;i = 1, for all Va
Label l0
Label l1
Integer Programming Formulation
Va Vb
2
5
4
2
0
1 1
0
2Pairwise Potentials
ab;00 = 0
ab;10 = 1
ab;01 = 1
ab;11 = 0
Sum of Pairwise Potentials
∑(a,b) ∑ik ab;ik ya;iyb;k
ya;i {0,1}
∑i ya;i = 1
Label l0
Label l1
Integer Programming Formulation
Va Vb
2
5
4
2
0
1 1
0
2Pairwise Potentials
ab;00 = 0
ab;10 = 1
ab;01 = 1
ab;11 = 0
Sum of Pairwise Potentials
∑(a,b) ∑ik ab;ik yab;ik
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
Label l0
Label l1
Integer Programming Formulation
min ∑a ∑i a;i ya;i + ∑(a,b) ∑ik ab;ik yab;ik
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
Integer Programming Formulation
min Ty
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
= [ … a;i …. ; … ab;ik ….]
y = [ … ya;i …. ; … yab;ik ….]
One variable, two labels
ya;0
ya;1
ya;0 {0,1} ya;1 {0,1} ya;0 + ya;1 = 1
y = [ ya;0 ya;1] = [ a;0 a;1]
Two variables, two labels
q = [ a;0 a;1 b;0 b;1
ab;00 ab;01 ab;10 ab;11]y = [ ya;0 ya;1 yb;0 yb;1
yab;00 yab;01 yab;10 yab;11]
ya;0 {0,1} ya;1 {0,1} ya;0 + ya;1 = 1
yb;0 {0,1} yb;1 {0,1} yb;0 + yb;1 = 1
yab;00 = ya;0 yb;0 yab;01 = ya;0 yb;1
yab;10 = ya;1 yb;0 yab;11 = ya;1 yb;1
In General
Marginal Polytope
In General
q R(|V||L| + |E||L|2)
y {0,1}(|V||L| + |E||L|2)
Number of constraints
|V||L| + |V| + |E||L|2
ya;i {0,1} ∑i ya;i = 1 yab;ik = ya;i yb;k
Integer Programming Formulation
min Ty
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
= [ … a;i …. ; … ab;ik ….]
y = [ … ya;i …. ; … yab;ik ….]
Integer Programming Formulation
min Ty
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
Solve to obtain MAP labelling y*
Integer Programming Formulation
min Ty
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
But we can’t solve it in general
• Integer Programming Formulation
• LP Relaxation
Outline
Linear Programming Relaxation
min Ty
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
Two reasons why we can’t solve this
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
yab;ik = ya;i yb;k
One reason why we can’t solve this
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
∑k yab;ik = ∑kya;i yb;k
One reason why we can’t solve this
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
One reason why we can’t solve this
= 1∑k yab;ik = ya;i∑k yb;k
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
∑k yab;ik = ya;i
One reason why we can’t solve this
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
∑k yab;ik = ya;i
No reason why we can’t solve this *
*memory requirements, time complexity
One variable, two labels
ya;0
ya;1
ya;0 {0,1} ya;1 {0,1} ya;0 + ya;1 = 1
y = [ ya;0 ya;1] = [ a;0 a;1]
One variable, two labels
ya;0
ya;1
ya;0 [0,1] ya;1 [0,1] ya;0 + ya;1 = 1
y = [ ya;0 ya;1] = [ a;0 a;1]
Two variables, two labels
q = [ a;0 a;1 b;0 b;1
ab;00 ab;01 ab;10 ab;11]y = [ ya;0 ya;1 yb;0 yb;1
yab;00 yab;01 yab;10 yab;11]
ya;0 {0,1} ya;1 {0,1} ya;0 + ya;1 = 1
yb;0 {0,1} yb;1 {0,1} yb;0 + yb;1 = 1
yab;00 = ya;0 yb;0 yab;01 = ya;0 yb;1
yab;10 = ya;1 yb;0 yab;11 = ya;1 yb;1
Two variables, two labels
q = [ a;0 a;1 b;0 b;1
ab;00 ab;01 ab;10 ab;11]y = [ ya;0 ya;1 yb;0 yb;1
yab;00 yab;01 yab;10 yab;11]
ya;0 [0,1] ya;1 [0,1] ya;0 + ya;1 = 1
yb;0 [0,1] yb;1 [0,1] yb;0 + yb;1 = 1
yab;00 = ya;0 yb;0 yab;01 = ya;0 yb;1
yab;10 = ya;1 yb;0 yab;11 = ya;1 yb;1
Two variables, two labels
q = [ a;0 a;1 b;0 b;1
ab;00 ab;01 ab;10 ab;11]y = [ ya;0 ya;1 yb;0 yb;1
yab;00 yab;01 yab;10 yab;11]
ya;0 [0,1] ya;1 [0,1] ya;0 + ya;1 = 1
yb;0 [0,1] yb;1 [0,1] yb;0 + yb;1 = 1
yab;00 + yab;01 = ya;0
yab;10 = ya;1 yb;0 yab;11 = ya;1 yb;1
Two variables, two labels
q = [ a;0 a;1 b;0 b;1
ab;00 ab;01 ab;10 ab;11]y = [ ya;0 ya;1 yb;0 yb;1
yab;00 yab;01 yab;10 yab;11]
ya;0 [0,1] ya;1 [0,1] ya;0 + ya;1 = 1
yb;0 [0,1] yb;1 [0,1] yb;0 + yb;1 = 1
yab;00 + yab;01 = ya;0
yab;10 + yab;11 = ya;1
In General
Marginal Polytope
LocalPolytope
In General
q R(|V||L| + |E||L|2)
y [0,1](|V||L| + |E||L|2)
Number of constraints
|V||L| + |V| + |E||L|
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
∑k yab;ik = ya;i
No reason why we can’t solve this
Linear Programming Relaxation
Extensively studied
Optimization
Schlesinger, 1976
Koster, van Hoesel and Kolen, 1998
Theory
Chekuri et al, 2001 Archer et al, 2004
Machine Learning
Wainwright et al., 2001
Linear Programming Relaxation
Many interesting properties
• Global optimal MAP for trees
Wainwright et al., 2001
But we are interested in NP-hard cases
• Preserves solution for reparameterization
• Global optimal MAP for submodular energy
Chekuri et al., 2001
Linear Programming Relaxation
• Large class of problems
• Metric Labeling• Semi-metric Labeling
Many interesting properties - Integrality Gap
Manokaran et al., 2008
• Most likely, provides best possible integrality gap
Recommended