View
15
Download
0
Category
Preview:
Citation preview
Near-Optimal Erasure List-Decodable Codes
Dean Doron (Tel-Aviv University)Workshop on Coding and Information Theory, CMSA
Joint work withAvi Ben-AroyaAmnon Ta-Shma
• A code C : {0,1}n → {0,1}n̄ is (1-ε,L) erasure list-decodable, if
• Given a codeword
Erasure List-Decodable Codes
0 1 1 0 0 0 1 0 1 1
n̄
• A code C : {0,1}n → {0,1}n̄ is (1-ε,L) erasure list-decodable, if
• Given a codeword where all but ε fraction of its coordinates were (adversarially) erased,
Erasure List-Decodable Codes
0 1 1 0 0 0 1 0 1 1
n̄
• A code C : {0,1}n → {0,1}n̄ is (1-ε,L) erasure list-decodable, if
• Given a codeword where all but ε fraction of its coordinates were (adversarially) erased,
Erasure List-Decodable Codes
? 1 ? ? ? 0 1 ? ? 1
n̄
• A code C : {0,1}n → {0,1}n̄ is (1-ε,L) erasure list-decodable, if
• Given a codeword where all but ε fraction of its coordinates were (adversarially) erased,
Erasure List-Decodable Codes
? 1 ? ? ? 0 1 ? ? 1
1 1 1 0 1 0 1 0 0 1
0 1 1 0 0 0 1 0 1 1
0 1 0 0 0 0 1 1 1 1
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
0 1 1 1 1 0 1 1 0 1
n̄there exists a list of size at most L that contains the original codeword.
• A code C : {0,1}n → {0,1}n̄ is (1-ε,L) erasure list-decodable, if
• Given a codeword where all but ε fraction of its coordinates were (adversarially) erased,
Erasure List-Decodable Codes
? 1 ? ? ? 0 1 ? ? 1
1 1 1 0 1 0 1 0 0 1
0 1 1 0 0 0 1 0 1 1
0 1 0 0 0 0 1 1 1 1
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
0 1 1 1 1 0 1 1 0 1
n̄there exists a list of size at most L that contains the original codeword.
• We work over the binary field.
• The two key parameters:
• The code’s rate R = n̄/n.
• The list-size L.
Bounds for the binary case
Bounds on the rate
• The code’s rate — R.
Bounds on the rate
• The code’s rate — R.
• Can be as large as ε [Guruswami 03] (recall that ε is the fraction of coordinates we keep).
• Compare with the errors model, where ~ ε2 is necessary for a code of distance ½ - ε.
Bounds on the rate
Bounds on the list-size
• The list-size — L.
Bounds on the list-size
• The list-size — L.
• Can be as tiny as log(1/ε) [Guruswami 03].
• Compare with the errors model, where poly(1/ε) is necessary.
Bounds on the list-size
• The list-size — L.
• Can be as tiny as log(1/ε) [Guruswami 03].
• Compare with the errors model, where poly(1/ε) is necessary.
• For linear codes, the list-size is at least 1/ε [Guruswami 03].
Bounds on the list-size
Previous Results
• Any binary code of distance ½ - ε can handle 1-2ε erasures with polynomial list size.
Previous Results
• Any binary code of distance ½ - ε can handle 1-2ε erasures with polynomial list size.
• Drawback: We have upper bounds. Cannot break the rate-ε2 bound that way [MRRW,…].
Previous Results
• Any binary code of distance ½ - ε can handle 1-2ε erasures with polynomial list size.
• Drawback: We have upper bounds. Cannot break the rate-ε2 bound that way [MRRW,…].
• Few explicit constructions (especially for binary codes).
Previous Results
Previous Results
Rate R List-Size L
Non-explicit ε log(1/ε)
Optimal codes with distance ½-ε ε2 poly(1/ε)
Guruswami 01 (constant ε) ε2/log(1/ε) 1/ε
Guruswami-Indyk 03 (constant ε) ε2/log(1/ε) log(1/ε)
• In the small ε regime, we break the ε2 barrier, and manage to do this with nearly optimal parameters.
Our Result
• In the small ε regime, we break the ε2 barrier, and manage to do this with nearly optimal parameters.
Our Result
For every small enough ε and a constant γ there exists an explicit binary erasure list-decodable code with rate
R = ε1+ γ
and list-size
L = poly(log(1/ε))
Previous Results
Rate R List-Size L
Non-explicit ε log(1/ε)
Optimal codes with distance ½-ε ε2 poly(1/ε)
Guruswami 01 (constant ε) ε2/log(1/ε) 1/ε
Guruswami-Indyk 03 (constant ε) ε2/log(1/ε) log(1/ε)
Our Result(small ε) ε1+γ poly(log(1/ε))
Different Perspectives
Erasure List-
Decodable Codes
Ramsey Graphs
Strong 1-bit
Dispersers
[Guruswami 04]
[Gradwohl, Kindler, Reingold, Ta-Shma 05]
Different Perspectives
Erasure List-
Decodable Codes
Ramsey Graphs
Strong 1-bit
Dispersers
[Guruswami 04]
[Gradwohl, Kindler, Reingold, Ta-Shma 05]
Different Perspectives
Erasure List-
Decodable Codes
Ramsey Graphs
Strong 1-bit
Dispersers
[Guruswami 04]
[Gradwohl, Kindler, Reingold, Ta-Shma 05]
Different Perspectives
Erasure List-
Decodable Codes
Ramsey Graphs
Strong 1-bit
Dispersers
[Guruswami 04]
[Gradwohl, Kindler, Reingold, Ta-Shma 05]
Different Perspectives
Erasure List-
Decodable Codes
Ramsey Graphs
Strong 1-bit
Dispersers
[Guruswami 04]
[Gradwohl, Kindler, Reingold, Ta-Shma 05]
1-bit Strong Dispersers
Disp: {0,1}n × {0,1}d →{0,1}
{0,1}n
0
1
Disp(x,y)
{0,1}
D=2dx
• For every (n,k)-source X, or a set X⊆{0,1}n of size at least K=2k,
1-bit Strong Dispersers
Disp: {0,1}n × {0,1}d →{0,1}
{0,1}n
0
1
X Disp(x,y)
{0,1}
D=2dx
• For every (n,k)-source X, or a set X⊆{0,1}n of size at least K=2k,
1-bit Strong Dispersers
Disp: {0,1}n × {0,1}d →{0,1}
{0,1}n
0
1
X
For every x, Pr[X = x] ≤ 2-k.
Disp(x,y)
{0,1}
D=2dx
• For every (n,k)-source X, or a set X⊆{0,1}n of size at least K=2k,
1-bit Strong Dispersers
Disp: {0,1}n × {0,1}d →{0,1}
{0,1}n
0
1
X Disp(x,y)
{0,1}
D=2dx
• For every (n,k)-source X, or a set X⊆{0,1}n of size at least K=2k,
• For all but ε-fraction of the seeds y∈{0,1}d,
Disp(X,y) = {0,1}
1-bit Strong Dispersers
Disp: {0,1}n × {0,1}d →{0,1}
{0,1}n
0
1
X Disp(x,y)
{0,1}
D=2dx
• For every (n,k)-source X, or a set X⊆{0,1}n of size at least K=2k,
• For all but ε-fraction of the seeds y∈{0,1}d,
Disp(X,y) = {0,1}
1-bit Strong Dispersers
Disp: {0,1}n × {0,1}d →{0,1}
{0,1}n
0
1
XDisp(x1,ybad)=1
Disp(xK,ybad)=1
⋮
{0,1}
• For every (n,k)-source X, or a set X⊆{0,1}n of size at least K=2k,
• For all but ε-fraction of the seeds y∈{0,1}d,
Disp(X,y) = {0,1}
1-bit Strong Dispersers
Disp: {0,1}n × {0,1}d →{0,1}
{0,1}n
0
1
X
⋮
Disp(x1,y)=0
Disp(xK,y)=1
{0,1}
Bounds for1-bit Strong Dispersers
• Dispersers are an important tool in derandomization.
Bounds for1-bit Strong Dispersers
• Dispersers are an important tool in derandomization.
• The two key parameters:
Bounds for1-bit Strong Dispersers
• Dispersers are an important tool in derandomization.
• The two key parameters:
• The disperser’s seed length — d.
• The disperser’s entropy requirement — k (or, how small can X be).
Bounds for1-bit Strong Dispersers
Bounds on the seed length
• The disperser’s seed length — d.
Bounds on the seed length
• The disperser’s seed length — d.
• Can be as small as log(n)+log(1/ε) [Radhakrishnan,Ta-Shma 00, Meka,Reingold,Zhou 14].
Bounds on the seed length
Bounds on the entropy
• The disperser’s entropy requirement — k (or, how small can X be).
• Can be as tiny as loglog(1/ε). That is, an optimal disperser works for extremely small sets — of size O(log(1/ε)) [Radhakrishnan,Ta-Shma 00, Meka,Reingold,Zhou 14].
Bounds on the entropy
Comparison with extractors
• Dispersers’ parameters outperform seeded extractors, in which we don’t only require that the output covers the entire codomain, but also require doing it uniformly.
• The seed length must be at least log(n)+2log(1/ε) [Radhakrishnan,Ta-Shma 00] (compare with log(n)+log(1/ε) for dispersers).
• The entropy requirement must be at least 2log(1/ε) [Radhakrishnan,Ta-Shma 00] (compare with loglog(1/ε) for dispersers).
Comparison with extractors
Different Perspectives
Erasure List-
Decodable Codes
Ramsey Graphs
Strong 1-bit
Dispersers
[Gradwohl, Kindler, Reingold, Ta-Shma 05]
[Guruswami 04]
Codes from Dispersers
Codes from DispersersIf Disp: {0,1}n × {0,1}d →{0,1} has error ε and entropy requirement k, we can explicitly construct a code C: {0,1}n → {0,1}D for D=2d that is erasure list-decodable from 1-2ε erasures with list-size L = 2k and rate R=n/D.
Codes from DispersersIf Disp: {0,1}n × {0,1}d →{0,1} has error ε and entropy requirement k, we can explicitly construct a code C: {0,1}n → {0,1}D for D=2d that is erasure list-decodable from 1-2ε erasures with list-size L = 2k and rate R=n/D.
Taking an optimal disperser with d=log(n)+log(1/ε) and k=loglog(1/ε), we get an optimal erasure list-decodable code with rate R=n/D=ε and list-size L=2k=log(1/ε).
Constructing Dispersers
• How can we construct small-error 1-bit strong dispersers?
Constructing Dispersers
• How can we construct small-error 1-bit strong dispersers?
• We need all but ε-fraction of the seeds to cover {0,1}.
• We don’t want to do it uniformly (this would give an extractor).
• Say, typically, Disp(X,y) is ε0-close to uniform for ε0 ≫ ε (say, ε0 = ¼).
Constructing Dispersers
Different Perspectives
Erasure List-
Decodable Codes
Ramsey Graphs
Strong 1-bit
Dispersers
[Gradwohl, Kindler, Reingold, Ta-Shma 05]
[Guruswami 04]
2-Source Extractors
Two-Source Extractors
{0,1}n
X
{0,1}d
Y
edge ↔ 2Ext = 1
2Ext: {0,1}n × {0,1}d →{0,1}
Two-Source Extractors
{0,1}n
X
• For every (n,k)-source X,
• For every (d,d’)-source Y, independent of X,
• 2Ext(X,Y) is ε0-close to uniform (think ε0 of as constant). {0,1}d
Y
edge ↔ 2Ext = 1
2Ext: {0,1}n × {0,1}d →{0,1}
Two-Source Extractors
{0,1}n
X
[2d]
|Y|=2d’
edge ↔ 2Ext = 1
2Ext: {0,1}n × {0,1}d →{0,1}
Two-Source Extractors
{0,1}n
X
• The error of the corresponding strong disperser (the faction of bad seeds) is given by ε = 2d’-d.
[2d]
|Y|=2d’
edge ↔ 2Ext = 1
2Ext: {0,1}n × {0,1}d →{0,1}
Implications of Two-Source Extractors
• By constructing two-source extractors for an (n,k)-source and a (d,d’=γd) source, we get:
Implications of Two-Source Extractors
• By constructing two-source extractors for an (n,k)-source and a (d,d’=γd) source, we get:
• A strong disperser for entropy k and seed-length d=(1/(1-γ))log(1/ε).
Implications of Two-Source Extractors
• By constructing two-source extractors for an (n,k)-source and a (d,d’=γd) source, we get:
• A strong disperser for entropy k and seed-length d=(1/(1-γ))log(1/ε).
• An erasure list-decodable code of list-size L=2k and rate R=ε1/(1-γ).
Implications of Two-Source Extractors
• By constructing two-source extractors for an (n,k)-source and a (d,d’=γd) source, we get:
• A strong disperser for entropy k and seed-length d=(1/(1-γ))log(1/ε).
• An erasure list-decodable code of list-size L=2k and rate R=ε1/(1-γ).
Implications of Two-Source Extractors
So we must achieve γ<½ to break the ε2 bound!
• By constructing two-source extractors for an (n,k)-source and a (d,d’=γd) source, we get:
• A strong disperser for entropy k and seed-length d=(1/(1-γ))log(1/ε).
• An erasure list-decodable code of list-size L=2k and rate R=ε1/(1-γ).
Implications of Two-Source Extractors
Our Goal
• We are given an (n,k)-source X and an independent (d,d’=γd)-source Y. We want to extract one unbiased bit.
Our Goal
γdY
kX
• We are given an (n,k)-source X and an independent (d,d’=γd)-source Y. We want to extract one unbiased bit.
• For γ>½, the problem was already solved.
Our Goal
γdY
kX
Our Goal
• We are given an (n,k)-source X and an independent (d,d’=γd)-source Y. We want to extract one unbiased bit.
• For γ>½, the problem was already solved.
• Raz [Raz 05] gave a construction that supports, say, γ=0.6. In such a case, Raz(X,Y) is very close to uniform.
Our Goal
• We are given an (n,k)-source X and an independent (d,d’=γd)-source Y. We want to extract one unbiased bit.
• For γ>½, the problem was already solved.
• Raz [Raz 05] gave a construction that supports, say, γ=0.6. In such a case, Raz(X,Y) is very close to uniform.
• In the code sense, it gives a list-size of poly(log(1/ε)).
Our Goal
• We are given an (n,k)-source X and an independent (d,d’=γd)-source Y. We want to extract one unbiased bit.
• For γ>½, the problem was already solved.
• Raz [Raz 05] gave a construction that supports, say, γ=0.6. In such a case, Raz(X,Y) is very close to uniform.
• In the code sense, it gives a list-size of poly(log(1/ε)).
• But, γ>½ so we cannot break the rate ε2 bound.
Our Goal
Our Goal
• We are given an (n,k)-source X and an independent (d,d’=γd)-source Y. We want to extract one unbiased bit.
Our Goal
γdY
kX
Condensing Y
• We want to:
• Condense Y. From a (d,γd)-source to an (m,0.6m)-source.
Condensing Y
γdY
kX
Condenser
• We want to:
• Condense Y. From a (d,γd)-source to an (m,0.6m)-source.
• To apply Raz’s extractor.
Condensing Y
γdY
kX
Condenser
Seeded Condensers
• We need Cond: {0,1}d × {0,1}s →{0,1}m that increases the density of the source to rate 0.6.
Seeded Condensers
• We need Cond: {0,1}d × {0,1}s →{0,1}m that increases the density of the source to rate 0.6.
• To apply Cond, we need a uniform seed Us. How do we get it?
Seeded Condensers
• We need Cond: {0,1}d × {0,1}s →{0,1}m that increases the density of the source to rate 0.6.
• To apply Cond, we need a uniform seed Us. How do we get it?
• Option 1 — Use another independent source.
Seeded Condensers
• We need Cond: {0,1}d × {0,1}s →{0,1}m that increases the density of the source to rate 0.6.
• To apply Cond, we need a uniform seed Us. How do we get it?
• Option 1 — Use another independent source.
• Option 2 — Enumerate over all possible seeds. This gives us a table Y’.
Seeded Condensers
Condensing Y
γdY
kX
Condenser
Cond(Y,s1)Cond(Y,s2)
Cond(Y,si)
Cond(Y,sA)
Y’
Seeded Condensers
• The number of rows of Y’ is the number of possible seeds of the condenser.
Seeded Condensers
• The number of rows of Y’ is the number of possible seeds of the condenser.
• Composing existing constructions, we have an explicit condenser having a constant number of seeds! This fact is crucial for us [Dvir,Kopparty,Saraf,Sudan 13, Zuckerman 06].
Seeded Condensers
• The number of rows of Y’ is the number of possible seeds of the condenser.
• Composing existing constructions, we have an explicit condenser having a constant number of seeds! This fact is crucial for us [Dvir,Kopparty,Saraf,Sudan 13, Zuckerman 06].
• More specifically, the seed length of the condenser is s=O(log(1/εcond)) and we apply it with a constant εcond.
Seeded Condensers
• The number of rows of Y’ is the number of possible seeds of the condenser.
• Composing existing constructions, we have an explicit condenser having a constant number of seeds! This fact is crucial for us [Dvir,Kopparty,Saraf,Sudan 13, Zuckerman 06].
• More specifically, the seed length of the condenser is s=O(log(1/εcond)) and we apply it with a constant εcond.
• Y’ is still only a function of Y.
Seeded Condensers
Condensing Y
γdY
kX
Condenser
~0.6d
Cond(Y,si)
Y’
Applying Raz’s extractor
• Most of the rows of Y’ are good — they have large density.
• We can now apply Raz’s extractor.
• X as the source.
• Every row of Y’ as the seed.
Applying Raz’s extractor
Applying Raz’s extractor
γdY
kX
Condenser
~0.6d
Cond(Y,si)
Y’
Extractor
~U
Raz(X,Y’i)
Y’’
Applying Raz’s extractor
• We now have that most of the rows of Y’’ are uniform and there is a small set of bad rows.
Applying Raz’s extractor
• We now have that most of the rows of Y’’ are uniform and there is a small set of bad rows.
• That’s not enough for us — we need to output a single good value.
Applying Raz’s extractor
• We now have that most of the rows of Y’’ are uniform and there is a small set of bad rows.
• That’s not enough for us — we need to output a single good value.
• The solution (in the spirit of [Chattopadhyay,Zuckerman 16])
• Apply a resilient function on the rows of Y’’.
Applying Raz’s extractor
Resilient Functions
• What do we need from a resilient function?
Resilient Functions
• What do we need from a resilient function?
• Nearly balanced.
• Its output cannot be heavily influenced by any small set of bad rows.
Resilient Functions
• What do we need from a resilient function?
• Nearly balanced.
• Its output cannot be heavily influenced by any small set of bad rows.
• For example, the Majority function is a resilient function.
Resilient Functions
Resilient Functions
Resilient Functions
• That approach is problematic…
Resilient Functions
• That approach is problematic…
• Even if there is only one bad row, the Majority function can still fail!
~U~U~U~U~U~U
Y’’1 = ¬Y’’2
Ext(X,Y’i)Y’’3 = ¬Y’’4
Resilient Functions
• How can we fix this?
Resilient Functions
• How can we fix this?
• The Majority function will work if the good rows are not only uniform, but uniform and 10-wise independent [Viola 14].
Resilient Functions
• How can we fix this?
• The Majority function will work if the good rows are not only uniform, but uniform and 10-wise independent [Viola 14].
• So, before applying a resilient function, we need to (partially) break the correlations in Y’’.
Resilient Functions
• How can we fix this?
• The Majority function will work if the good rows are not only uniform, but uniform and 10-wise independent [Viola 14].
• So, before applying a resilient function, we need to (partially) break the correlations in Y’’.
• For that, we use a correlation breaker.
Resilient Functions
Correlation Breakers
• We think of a 10-wise correlation breaker as an extractor, with the added property that its output on 10 distinct good seeds is (very close to) 10-wise independent.
Correlation Breakers
• We think of a 10-wise correlation breaker as an extractor, with the added property that its output on 10 distinct good seeds is (very close to) 10-wise independent.
• Cohen’s correlation breaker [Cohen 16] allows for almost optimal parameters when the overall number of rows is constant.
Correlation Breakers
Correlation Breakers
• A correlation breaker requires a source and an independent seed.
Correlation Breakers
• A correlation breaker requires a source and an independent seed.
• We use every row of Y’’, and for the seed we reuse our Y.
Correlation Breakers
• A correlation breaker requires a source and an independent seed.
• We use every row of Y’’, and for the seed we reuse our Y.
• The source and the seed are not independent… Y’’ is a function of both X and Y.
Correlation Breakers
• A correlation breaker requires a source and an independent seed.
• We use every row of Y’’, and for the seed we reuse our Y.
• The source and the seed are not independent… Y’’ is a function of both X and Y.
• A main part of the analysis — show that we’re still OK.
Correlation Breakers
• A correlation breaker requires a source and an independent seed.
• We use every row of Y’’, and for the seed we reuse our Y.
• The source and the seed are not independent… Y’’ is a function of both X and Y.
• A main part of the analysis — show that we’re still OK.
• This gives a large set of good rows, every 10 of them are jointly close to uniform.
Correlation Breakers
• The correlation breaker outputs one bit.
• Every 10 good rows are jointly very close to uniform.
Applying a correlation breaker
~U
Ext(X,Y’i)
Y’’ = f(X,Y)
γdY
Correlation
CB(Y,Y’’i)
Y’’’
Breaker
• We are now ready to apply a resilient function!
Applying a resilient function
~U
Ext(X,Y’i)
Y’’ = f(X,Y)
γdY
Correlation
CB(Y,Y’’i)
Y’’’
Breaker
• We are now ready to apply a resilient function!
Applying a resilient function
~U
Ext(X,Y’i)
Y’’ = f(X,Y)
γdY
Correlation
CB(Y,Y’’i)
Y’’’
Breaker
Resilient
FunctionU1
Overview
γdY
kX
Condenser
~0.6d
Cond(Y,si)
Y’
Extractor
~U
Raz(X,Y’i)
Y’’
Correlation
CB(Y,Y’’i)Breaker
Y’’’
U1
ResilientFunction
Constantbias ε0
Wrapping-Up
• Given an (n,k)-source X and a (d,d’=γd) source Y, we extracted one bit with an arbitrarily small constant error.
Wrapping-Up
• Given an (n,k)-source X and a (d,d’=γd) source Y, we extracted one bit with an arbitrarily small constant error.
• k can be extremely small — Ω(loglog(n)). Corresponds to a small list-size of our erasure list-decodable code.
Wrapping-Up
• Given an (n,k)-source X and a (d,d’=γd) source Y, we extracted one bit with an arbitrarily small constant error.
• k can be extremely small — Ω(loglog(n)). Corresponds to a small list-size of our erasure list-decodable code.
• d can be small — Ω(log(n)), and the entropy-rate γ is an arbitrary constant. Corresponds to the large rate of our code.
Wrapping-Up
• Given an (n,k)-source X and a (d,d’=γd) source Y, we extracted one bit with an arbitrarily small constant error.
• k can be extremely small — Ω(loglog(n)). Corresponds to a small list-size of our erasure list-decodable code.
• d can be small — Ω(log(n)), and the entropy-rate γ is an arbitrary constant. Corresponds to the large rate of our code.
• In this, unbalanced, regime of parameters, our two-source construction is close to optimal.
Wrapping-Up
Our Result
Our Result
For every small enough ε and a constant γ there exists an explicit binary erasure list-decodable code with rate
R = ε1+ γ
and list-size
L = poly(log(1/ε))
• We skipped over many delicate details which are crucial for our construction to work. Examples are:
• Reusing the source Y.
• Errors cannot be aggregated, but must be handled with care (e.g., separating errors that depend on the number of rows and errors that depend on the row’s length).
• Handling a set of bad rows that depend on the samples and not only on the distributions.
• The resilient function cannot be Majority and we need to achieve t-wise independence for a large t.
Wrapping-Up
• Efficient list-decoding?
• Achieving linear erasure list-decodable codes that beat the rate ε2 bound (they would necessarily have a larger list-size).
• Getting even closer to optimal codes…
• Obtaining strong dispersers outputting many bits.
Open Questions
Thanks for listening.
Recommended