An Information Complexity Approach to Extended...

An Information Complexity Approach to

Extended Formulations

Ankur Moitra Institute for Advanced Study

joint work with Mark Braverman

The Permutahedron

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have? exponentially many!

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

e.g. S [n], Σi in S xi ≥ 1 + 2 + … + |S| = |S|(|S|+1)/2

exponentially many!

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

e.g. S [n], Σi in S xi ≥ 1 + 2 + … + |S| = |S|(|S|+1)/2

Let Q = {A| A is doubly-stochastic}

exponentially many!

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

e.g. S [n], Σi in S xi ≥ 1 + 2 + … + |S| = |S|(|S|+1)/2

Let Q = {A| A is doubly-stochastic}

Then P is the projection of Q: P = {A t | A in Q }

exponentially many!

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

e.g. S [n], Σi in S xi ≥ 1 + 2 + … + |S| = |S|(|S|+1)/2

Let Q = {A| A is doubly-stochastic}

Then P is the projection of Q: P = {A t | A in Q }

Yet Q has only O(n2) facets

exponentially many!

Extended Formulations

The extension complexity (xc) of a polytope P is the

minimum number of facets of Q so that P = proj(Q)

Extended Formulations

The extension complexity (xc) of a polytope P is the

minimum number of facets of Q so that P = proj(Q)

e.g. xc(P) = Θ(n logn)

for permutahedron

Extended Formulations

The extension complexity (xc) of a polytope P is the

minimum number of facets of Q so that P = proj(Q)

e.g. xc(P) = Θ(n logn)

for permutahedron

xc(P) = Θ(logn) for a

regular n-gon, but Ω(√n)

for its perturbation

Extended Formulations

The extension complexity (xc) of a polytope P is the

minimum number of facets of Q so that P = proj(Q)

e.g. xc(P) = Θ(n logn)

for permutahedron

xc(P) = Θ(logn) for a

regular n-gon, but Ω(√n)

for its perturbation

In general, P = {x | y, (x,y) in Q} E

Extended Formulations

The extension complexity (xc) of a polytope P is the

minimum number of facets of Q so that P = proj(Q)

e.g. xc(P) = Θ(n logn)

for permutahedron

xc(P) = Θ(logn) for a

regular n-gon, but Ω(√n)

for its perturbation

In general, P = {x | y, (x,y) in Q} E

…analogy with quantifiers in Boolean formulae

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Hence, we can run standard LP solvers instead of

the ellipsoid algorithm

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Hence, we can run standard LP solvers instead of

the ellipsoid algorithm

EFs often give, or are based on new combinatorial

insights

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Hence, we can run standard LP solvers instead of

the ellipsoid algorithm

EFs often give, or are based on new combinatorial

insights

e.g. Birkhoff-von Neumann Thm and permutahedron

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Hence, we can run standard LP solvers instead of

the ellipsoid algorithm

EFs often give, or are based on new combinatorial

insights

e.g. Birkhoff-von Neumann Thm and permutahedron

e.g. prove there is low-cost object, through its polytope

Explicit, Hard Polytopes?

Explicit, Hard Polytopes?

Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn}

Explicit, Hard Polytopes?

Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn}

(If we could optimize over this polytope, then P = NP)

Explicit, Hard Polytopes?

Can we prove unconditionally there is no small EF?

Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn}

(If we could optimize over this polytope, then P = NP)

Explicit, Hard Polytopes?

Can we prove unconditionally there is no small EF?

Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn}

Caveat: this is unrelated to proving complexity l.b.s

(If we could optimize over this polytope, then P = NP)

Explicit, Hard Polytopes?

Can we prove unconditionally there is no small EF?

[Yannakakis ’90]: Yes, through the nonnegative rank

Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn}

Caveat: this is unrelated to proving complexity l.b.s

(If we could optimize over this polytope, then P = NP)

Theorem [Yannakakis ’90]: Any symmetric LP for

TSP or matching has size 2Ω(n)

Theorem [Yannakakis ’90]: Any symmetric LP for

TSP or matching has size 2Ω(n)

Theorem [Fiorini et al ’12]: Any LP for TSP has size

2Ω(√n) (based on a 2Ω(n) lower bd for clique)

Theorem [Yannakakis ’90]: Any symmetric LP for

TSP or matching has size 2Ω(n)

Theorem [Fiorini et al ’12]: Any LP for TSP has size

2Ω(√n) (based on a 2Ω(n) lower bd for clique)

Theorem [Braun et al ’12]: Any LP that approximates

clique within n1/2-eps has size exp(neps)

Hastad’s proved an n1-o(1) hardness of approx. for

clique, can we prove the analogue for EFs?

Theorem [Yannakakis ’90]: Any symmetric LP for

TSP or matching has size 2Ω(n)

Theorem [Fiorini et al ’12]: Any LP for TSP has size

2Ω(√n) (based on a 2Ω(n) lower bd for clique)

Theorem [Braun et al ’12]: Any LP that approximates

clique within n1/2-eps has size exp(neps)

Hastad’s proved an n1-o(1) hardness of approx. for

clique, can we prove the analogue for EFs?

Theorem [Yannakakis ’90]: Any symmetric LP for

TSP or matching has size 2Ω(n)

Theorem [Fiorini et al ’12]: Any LP for TSP has size

2Ω(√n) (based on a 2Ω(n) lower bd for clique)

Theorem [Braun et al ’12]: Any LP that approximates

clique within n1/2-eps has size exp(neps)

Theorem [Braverman, Moitra ’13]: Any LP that

approximates clique within n1-eps has size exp(neps)

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

The Factorization Theorem

The Factorization Theorem

How can we prove lower bounds on EFs?

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric

Parameter

Algebraic

Parameter

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric

Parameter

Algebraic

Parameter

Definition of the slack matrix…

The Slack Matrix

The Slack Matrix

P

The Slack Matrix

P S(P)

vertex

face

t

The Slack Matrix

P S(P)

vertex

face

t

vj

The Slack Matrix

P S(P)

vertex

face

t

vj

<ai,x> ≤ bi

The Slack Matrix

The entry in row i, column j is how slack the jth vertex

is on the ith constraint

P S(P)

vertex

face

t

vj

<ai,x> ≤ bi

The Slack Matrix

The entry in row i, column j is how slack the jth vertex

is on the ith constraint

bi - <ai, vj>

P S(P)

vertex

face

t

vj

<ai,x> ≤ bi

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric

Parameter

Algebraic

Parameter

Definition of the slack matrix…

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric

Parameter

Algebraic

Parameter

Definition of the slack matrix…

Definition of the nonnegative rank…

Nonnegative Rank

S =

Nonnegative Rank

S M1 Mr = + + …

rank one, nonnegative

Nonnegative Rank

S M1 Mr = + + …

Definition: rank+(S) is the smallest r s.t. S can be

written as the sum of r rank one, nonneg. matrices

rank one, nonnegative

Nonnegative Rank

S M1 Mr = + + …

Definition: rank+(S) is the smallest r s.t. S can be

written as the sum of r rank one, nonneg. matrices

rank one, nonnegative

Note: rank+(S) ≥ rank(S), but can be much larger too!

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric

Parameter

Algebraic

Parameter

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric

Parameter

Algebraic

Parameter

xc(P) = rank+(S(P))

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric

Parameter

Algebraic

Parameter

xc(P) = rank+(S(P))

Intuition: the factorization gives a change of variables

that preserves the slack matrix!

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric

Parameter

Algebraic

Parameter

xc(P) = rank+(S(P))

Intuition: the factorization gives a change of variables

that preserves the slack matrix!

We will give a new way to lower bound nonnegative

rank via information theory…

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

The Rectangle Bound

=

rank one, nonnegative

S M1 Mr + + …

The Rectangle Bound

=

rank one, nonnegative

M1 Mr + + …

The Rectangle Bound

= + + …

rank one, nonnegative

Mr

The Rectangle Bound

= + + …

rank one, nonnegative

Mr

The Rectangle Bound

= + + …

rank one, nonnegative

The Rectangle Bound

= + + …

rank one, nonnegative

The support of each Mi is a combinatorial rectangle

The Rectangle Bound

= + + …

rank one, nonnegative

The support of each Mi is a combinatorial rectangle

rank+(S) is at least # rectangles needed to cover supp of S

The Rectangle Bound

= + + …

rank one, nonnegative

rank+(S) is at least # rectangles needed to cover supp of S

The Rectangle Bound

= + + …

rank one, nonnegative

rank+(S) is at least # rectangles needed to cover supp of S

Non-deterministic Comm. Complexity

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

A Sampling Argument

= + + …

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

A Sampling Argument

= + + …

If r is too small, this procedure uses too little entropy!

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

The Construction of [Fiorini et al]

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

The Construction of [Fiorini et al]

S

constraints:

vertices:

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

The Construction of [Fiorini et al]

S

constraints:

vertices: a in {0,1}n

b in {0,1}n

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

The Construction of [Fiorini et al]

S

constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

The Construction of [Fiorini et al]

S

constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

UNIQUE DISJ.

Output ‘YES’ if a

and b as sets

are disjoint, and

‘NO’ if a and b

have one index

in common

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

The Reconstruction Principle

The Reconstruction Principle

Let T = {(a,b) | aTb = 0}, |T| = 3n

The Reconstruction Principle

Let T = {(a,b) | aTb = 0}, |T| = 3n

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

The Reconstruction Principle

Let T = {(a,b) | aTb = 0}, |T| = 3n

How does the sampling procedure specialize to

this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

The Reconstruction Principle

Let T = {(a,b) | aTb = 0}, |T| = 3n

Sampling Procedure:

How does the sampling procedure specialize to

this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

The Reconstruction Principle

Let T = {(a,b) | aTb = 0}, |T| = 3n

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

How does the sampling procedure specialize to

this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

The Reconstruction Principle

Let T = {(a,b) | aTb = 0}, |T| = 3n

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

How does the sampling procedure specialize to

this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

The Reconstruction Principle

Let T = {(a,b) | aTb = 0}, |T| = 3n

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

How does the sampling procedure specialize to

this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

Entropy Accounting 101

Entropy Accounting 101

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

Entropy Accounting 101

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

≤ n log23

Entropy Accounting 101

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

+

choose i

choose (a,b)

conditioned on i

≤ n log23

Entropy Accounting 101

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r +

choose i

choose (a,b)

conditioned on i

≤ n log23

Entropy Accounting 101

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r (1-δ)n log23 +

choose i

choose (a,b)

conditioned on i

≤ n log23

Entropy Accounting 101

Sampling Procedure:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r (1-δ)n log23 +

choose i

choose (a,b)

conditioned on i

≤ n log23 (?)

Suppose that a-j and b-j are fixed

a b bj

aj

Suppose that a-j and b-j are fixed

a b bj

aj

Mi restricted to (a-j,b-j)

Suppose that a-j and b-j are fixed

a b bj

aj

Mi restricted to (a-j,b-j)

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

zero

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

But rank(Mi)=1, hence there must be another

zero in either the same row or column

zero

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b) Mi(a,b)

(…bj=0…) (…bj=1…)

But rank(Mi)=1, hence there must be another

zero in either the same row or column

zero zero

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b) Mi(a,b)

(…bj=0…) (…bj=1…)

But rank(Mi)=1, hence there must be another

zero in either the same row or column

zero zero

H(aj,bj| i, a-j, b-j) ≤ 1 < log23

Entropy Accounting 101

Generate uniformly random (a,b) in T:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r +

choose i

choose (a,b)

conditioned on i

≤ n log23

Entropy Accounting 101

Generate uniformly random (a,b) in T:

Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

Choose i with probability Ri/R

Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r n +

choose i

choose (a,b)

conditioned on i

≤ n log23

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

Approximate EFs [Braun et al]

S

constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

Approximate EFs [Braun et al]

S

constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

Is there a K (with small xc) s.t. Pcorr K (C+1)Pcorr?

Approximate EFs [Braun et al]

S

constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

Is there a K (with small xc) s.t. Pcorr K (C+1)Pcorr?

+ C

Approximate EFs [Braun et al]

S

constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

New Goal:

Output the answer to

UDISJ with prob. at

least ½ + 1/2(C+1)

Is there a K (with small xc) s.t. Pcorr K (C+1)Pcorr?

+ C

Is the correlation polytope hard to approximate for

large values of C?

Analogy: Is UDISJ hard to compute with prob.

½+1/2(C+1) for large values of C?

Is the correlation polytope hard to approximate for

large values of C?

Analogy: Is UDISJ hard to compute with prob.

½+1/2(C+1) for large values of C?

There is a natural barrier at C = √n for proving l.b.s:

Is the correlation polytope hard to approximate for

large values of C?

Analogy: Is UDISJ hard to compute with prob.

½+1/2(C+1) for large values of C?

Claim: If UDISJ can be computed with prob.

½+1/2(C+1) using o(n/C2) bits, then UDISJ can be

computed with prob. ¾ using o(n) bits

There is a natural barrier at C = √n for proving l.b.s:

Is the correlation polytope hard to approximate for

large values of C?

Analogy: Is UDISJ hard to compute with prob.

½+1/2(C+1) for large values of C?

Claim: If UDISJ can be computed with prob.

½+1/2(C+1) using o(n/C2) bits, then UDISJ can be

computed with prob. ¾ using o(n) bits

Proof: Run the protocol O(C2) times and take the

majority vote

There is a natural barrier at C = √n for proving l.b.s:

Information Complexity

Information Complexity

In fact, a more technical barrier is:

Information Complexity

[Bar-Yossef et al ’04]:

# Bits exchanged ≥ information revealed

In fact, a more technical barrier is:

Information Complexity

[Bar-Yossef et al ’04]:

# Bits exchanged ≥ information revealed

In fact, a more technical barrier is:

≥ n x information revealed

for a one-bit problem

Information Complexity

[Bar-Yossef et al ’04]:

# Bits exchanged ≥ information revealed

In fact, a more technical barrier is:

≥ n x information revealed

for a one-bit problem Direct Sum Theorem

Information Complexity

[Bar-Yossef et al ’04]:

# Bits exchanged ≥ information revealed

In fact, a more technical barrier is:

≥ n x information revealed

for a one-bit problem Direct Sum Theorem

Problem: AND has a protocol with advantage 1/C

that reveals only 1/C2 bits…

Problem: AND has a protocol with advantage 1/C

that reveals only 1/C2 bits…

Problem: AND has a protocol with advantage 1/C

that reveals only 1/C2 bits…

Send her bit with prob.

½ + 1/C, else send

the complement

Problem: AND has a protocol with advantage 1/C

that reveals only 1/C2 bits…

Send her bit with prob.

½ + 1/C, else send

the complement

Send his bit with prob.

½ + 1/C, else send

the complement

Send her bit with prob.

½ + 1/C, else send

the complement

Send his bit with prob.

½ + 1/C, else send

the complement

Is there a stricter one bit problem that we could

reduce to instead?

Send her bit with prob.

½ + 1/C, else send

the complement

Send his bit with prob.

½ + 1/C, else send

the complement

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

Outline

Part I: Tools for Extended Formulations

Yannakakis’s Factorization Theorem

The Rectangle Bound

A Sampling Argument

Part II: Applications

Correlation Polytope

Approximating the Correlation Polytope

A Better Lower Bound for Disjointness

Definition: Output matrix is the prob. of outputting

“one” for each pair of inputs

Definition: Output matrix is the prob. of outputting

“one” for each pair of inputs

For the previous protocol:

½ + 5/C ½ + 1/C

½ + 1/C ½ - 3/C

A = 0

A = 1

B = 0 B = 1

For the previous protocol:

½ + 5/C ½ + 1/C

½ + 1/C ½ - 3/C

The constraint that a protocol achieves advantage at

least 1/C is a set of linear constraints on this matrix

A = 0

A = 1

B = 0 B = 1

For the previous protocol:

½ + 5/C ½ + 1/C

½ + 1/C ½ - 3/C

The constraint that a protocol achieves advantage at

least 1/C is a set of linear constraints on this matrix

(Using Hellinger): bits revealed ≥ (max– min)2 = Ω(1/C2)

A = 0

A = 1

B = 0 B = 1

For the previous protocol:

½ + 5/C ½ + 1/C

½ + 1/C ½ - 3/C

The constraint that a protocol achieves advantage at

least 1/C is a set of linear constraints on this matrix

(Using Hellinger): bits revealed ≥ (max– min)2 = Ω(1/C2)

(New): bits revealed ≥ |diagonal – anti-diagonal| = 0

A = 0

A = 1

B = 0 B = 1

For the previous protocol:

½ + 5/C ½ + 1/C

½ + 1/C ½ - 3/C

A = 0

A = 1

B = 0 B = 1

What if we also require the output distribution to be

the same for inputs {0,0}, {0,1}, {1,0}?

For the previous protocol:

½ + 5/C ½ + 1/C

½ + 1/C ½ - 3/C

A = 0

A = 1

B = 0 B = 1

What if we also require the output distribution to be

the same for inputs {0,0}, {0,1}, {1,0}?

For the previous new protocol:

½ + 1/C ½ + 1/C

½ + 1/C ½ - 3/C

A = 0

A = 1

B = 0 B = 1

What if we also require the output distribution to be

the same for inputs {0,0}, {0,1}, {1,0}?

(Using Hellinger): bits revealed ≥ (max – min)2 = Ω(1/C2)

For the previous new protocol:

½ + 1/C ½ + 1/C

½ + 1/C ½ - 3/C

A = 0

A = 1

B = 0 B = 1

What if we also require the output distribution to be

the same for inputs {0,0}, {0,1}, {1,0}?

(Using Hellinger): bits revealed ≥ (max – min)2 = Ω(1/C2)

(New): bits revealed ≥ |diagonal – anti-diagonal| = Ω(1/C)

For the previous new protocol:

½ + 1/C ½ + 1/C

½ + 1/C ½ - 3/C

A = 0

A = 1

B = 0 B = 1

How can we reduce to such a one-bit problem?

n/2 bits n/2 bits

a

b

How can we reduce to such a one-bit problem?

n/2 bits n/2 bits

a

b

Symmetrized Protocol T’:

How can we reduce to such a one-bit problem?

n/2 bits n/2 bits

a

b

Symmetrized Protocol T’:

Generate a random partition (x, y, z) of n/2 and

fill in x (0,0), y (0,1) and z (1,0) pairs

How can we reduce to such a one-bit problem?

n/2 bits n/2 bits

0

0

0

1

1

0

…

…

…

…

…

…

a

b

Symmetrized Protocol T’:

Generate a random partition (x, y, z) of n/2 and

fill in x (0,0), y (0,1) and z (1,0) pairs

How can we reduce to such a one-bit problem?

n/2 bits n/2 bits

0

0

0

1

1

0

…

…

…

…

…

…

a

b

Symmetrized Protocol T’:

Generate a random partition (x, y, z) of n/2 and

fill in x (0,0), y (0,1) and z (1,0) pairs

Permute the n bits uniformly at random

How can we reduce to such a one-bit problem?

n/2 bits n/2 bits

0

0

0

1

1

0

…

…

…

…

…

…

a

b

Symmetrized Protocol T’:

Generate a random partition (x, y, z) of n/2 and

fill in x (0,0), y (0,1) and z (1,0) pairs

Permute the n bits uniformly at random

Run T on the n bit string, return the output

Symmetrized Protocol T’:

Generate a random partition (x, y, z) of n/2 and

fill in x (0,0), y (0,1) and z (1,0) pairs

Permute the n bits uniformly at random

Run T on the n bit string, return the output

Symmetrized Protocol T’:

Generate a random partition (x, y, z) of n/2 and

fill in x (0,0), y (0,1) and z (1,0) pairs

Permute the n bits uniformly at random

Run T on the n bit string, return the output

Claim: The protocol T’ has bias 1/C for DISJ.

Symmetrized Protocol T’:

Generate a random partition (x, y, z) of n/2 and

fill in x (0,0), y (0,1) and z (1,0) pairs

Permute the n bits uniformly at random

Run T on the n bit string, return the output

Claim: The protocol T’ has bias 1/C for DISJ.

Claim: Yet flipping a pair (ai, bi) between (0,0), (0,1) or

(1,0) results in two distributions p,q (on inputs to T) with

|p-q|1 ≤ 1/n

Symmetrized Protocol T’:

Generate a random partition (x, y, z) of n/2 and

fill in x (0,0), y (0,1) and z (1,0) pairs

Permute the n bits uniformly at random

Run T on the n bit string, return the output

Claim: The protocol T’ has bias 1/C for DISJ.

Claim: Yet flipping a pair (ai, bi) between (0,0), (0,1) or

(1,0) results in two distributions p,q (on inputs to T) with

|p-q|1 ≤ 1/n

Proof:

Theorem: any protocol for UDISJ with adv. 1/C must

reveal Ω(n/C) bits (because it can be symmetrized)

Theorem: any protocol for UDISJ with adv. 1/C must

reveal Ω(n/C) bits (because it can be symmetrized)

Similarly sampling a pair (aj, bj) needs entropy log23 – δ,

where δ is the difference of diagonals

Theorem: any protocol for UDISJ with adv. 1/C must

reveal Ω(n/C) bits (because it can be symmetrized)

Similarly sampling a pair (aj, bj) needs entropy log23 – δ,

where δ is the difference of diagonals

Theorem: For any K with Pcorr K (C+1)Pcorr, the

extension complexity of K is at least exp(Ω(n/C))

Theorem: any protocol for UDISJ with adv. 1/C must

reveal Ω(n/C) bits (because it can be symmetrized)

Can our framework be used to prove further lower

bounds for extension complexity?

Similarly sampling a pair (aj, bj) needs entropy log23 – δ,

where δ is the difference of diagonals

Theorem: For any K with Pcorr K (C+1)Pcorr, the

extension complexity of K is at least exp(Ω(n/C))

Theorem: any protocol for UDISJ with adv. 1/C must

reveal Ω(n/C) bits (because it can be symmetrized)

Can our framework be used to prove further lower

bounds for extension complexity?

Similarly sampling a pair (aj, bj) needs entropy log23 – δ,

where δ is the difference of diagonals

Theorem: For any K with Pcorr K (C+1)Pcorr, the

extension complexity of K is at least exp(Ω(n/C))

For average case instances?

Theorem: any protocol for UDISJ with adv. 1/C must

reveal Ω(n/C) bits (because it can be symmetrized)

Can our framework be used to prove further lower

bounds for extension complexity?

Similarly sampling a pair (aj, bj) needs entropy log23 – δ,

where δ is the difference of diagonals

Theorem: For any K with Pcorr K (C+1)Pcorr, the

extension complexity of K is at least exp(Ω(n/C))

For average case instances? For SDPs?

Thanks!

Any Questions?

P vj