Simple extractors for all min-entropies and a new
pseudo-random generator
Ronen ShaltielChris Umans
Pseudo-Random Generators
pseudo-random
bits
PRG seed
Use a short “seed” of very few truly random bits to generate a long string of pseudo-random bits.
Pseudo-Randomness: No small circuit can distinguish truly random bits from pseudo-random bits.
few truly random bits
many “pseudo-random” bits
Nisan-Wigderson setting: The generator is more powerful than the circuit. (i.e., PRG runs in time n5 for circuits of size n3).
Hardness vs. Randomness paradigm: [BM,Y,S] Construct PRGs assuming hard functions. fEXP hard (on worst case) for small circuits. [NW88,BFNW93,I95,IW97,STV99,ISW99,ISW00]
Randomness Extractors [NZ]
random bits
Ext imperfect randomness
Extractors extract many random bits from arbitrary distributions which contain sufficient randomness.
A sample from a physical source of randomness. A high (min)-entropy distribution.
statistically close to uniform distribution.
Impossible for deterministic procedures!
Randomness Extractors [NZ]
random bits
Ext short seed
Extractors use a short seed of truly random bits extract many random bits from arbitrary distributions which contain sufficient randomness.
Extractors have many applications!
A lot of work on explicit constructions [vN53,B84, SV86,Z91,NZ93,SZ94,Z96,T96,T99,RRV99,ISW00, RSW00,TUZ01,TZS02].
Survey available from my homepage.
imperfect randomness
Trevisan’s argument
PRGs Extractors
Pseudo-random bits
PRG short seed hard function random
bits Ext short seed imperfect
randomness
Trevisan’s argument: Every PRG construction with certain relativization properties is also an extractor.
Extractors using the Nisan-Wigderson generator: [Tre99,RRV99,ISW00,TUZ01].
The method of Ta-Shma, Zuckerman and Safra [TZS01] Use Trevisan’s argument to give a new
method for constructing extractors. Extractors by solving a “generalized list-
decoding” problem. (List-decoding already played a role in this area [Tre99,STV99]).
Solution inspired by list-decoding algorithms for Reed-Muller codes [AS,STV99].
Simple and direct construction.
Our results Use the ideas of [TZS01] in an improved way:
Simple and direct extractors for all min-entropies. (For every a>0, seed=(1+a)(log n), output=k/(log n)O(a) .)
New list-decoding algorithm for Reed-Muller codes [AS97,STV99].
Trevisan’s argument “the other way”: New PRG construction. (Does not use Nisan-Wigderson
PRG). Optimal conversion of hardness into pseudo-randomness.
(HSG construction using only “necessary” assumptions). Improved PRG's for nondeterministic circuits
(Consequence: better derandomization of AM). Subsequent paper [Uma02] gives quantitive improvements
for PRGs.
The construction
Goal: Construct pseudo-random generators We’re given a hard function f on n
bits. We want to construct a PRG.
pseudo-random bits PRG short seed
n bits
n10 bits
Truth table of f
f(1)f(2)f(3)
…f(x)
…f(2n)
A naive idea
x
f(x)..f(x+n10)
G outputs n10 successive values of fG(x)=f(x),f(x+1),..,f(x+n10)
Previous: Make positions as independent as possible.[TZS01]: Make positions as dependent as possible.
Want to prove
f isn’t hard
G isn’t pseudo-random
f is hard
G is pseudo-random
Outline of Prooff isn’t hard
Use P to compute f
Exists next-bit predictor P for G
G isn’t pseudo-random
f is hard
G is pseudo-random
Next-Bit Predictorsf isn’t hard
Use P to compute f
Exists next-bit predictor P for G
G isn’t pseudo-random
f(x)..f(x+i-1)
f(x+i)
By the hybrid argument, there’s a small circuit P which predicts the next bit given the previous bits.
P(prefix)=next bit with probability ½+ε.
To show that f is easy we’ll use P to construct a small circuit for f.
Circuits can use “non-uniform advice”.
We can choose nO(1) inputs and query f on these inputs.
Showing that f is easyf isn’t hard
Use P to compute f
Exists next-bit predictor P for G
G isn’t pseudo-random
Rules of the gameWe need to design an
algorithm that: Queries f at few
positions. (poly(n)). Uses the next-bit
predictor P. Computes f
everywhere. (on all 2n positions).
f isn’t hard
Use P to compute f
Exists next-bit predictor P for G
G isn’t pseudo-random
Computing f using few queries
Simplifying assumption: P(prefix)=next bit with probability 1.
Queries (non-uniform advice) f(0),..,f(i-1) - n10 bits
Use P to compute f(i),f(i+1),f(i+2)…
f isn’t hard
Use P to compute f
Exists next-bit predictor P for G
G isn’t pseudo-random
f(0)…f(i-1)
f(i)
f(1)……f(i)
f(i+1)
f(2)..f(i+1)
f(i+2)
Compute f everywhere
Rules of the gameWe need to design an
algorithm that: Queries f at few
positions. (poly(n)). Uses the next-bit
predictor P. Computes f
everywhere. (on all 2n positions).
f isn’t hard
Use P to compute f
Exists next-bit predictor P for G
G isn’t pseudo-random
*To get a small circuit we also need that for every x, f(x) can be computed in time nO(1) given the non-uniform advice.
A Problem: The predictor makes errors
We’ve made a simplifying assumption that:
Prx[P(prefix)=next bit] = 1
We are only guaranteed that:
Prx[P(prefix)=next bit] > ½+ε
f(x)..f(x+i-1)
f(x+i)
vXvvXXXvXXvvvXvvXXVXvXXvXf(0)…f(i-1)f(1)……f(i)
Error: cannot
Continue!
Use Error-Correcting
techniques to recover from
errors!
Prefix
Using multivariate polynomials
The function f
2n
A line:One
Dimension
Using multivariate polynomials
f(1,1)f(1,2)
f(2,1)
2n/2
2n/2
A cube:many
dimensions
f(x1,x2)
*Low degree extension [BF]: We take a field F with about 2n/d elements and extend f to a degree about 2n/d polynomial in d variables.
w.l.o.g f(x1,..,xd) is a low degree polynomial in d
variables*
x1
x2
j
ji
iji xxaxxf 2
,1,21 ),(
Adjusting to Many Dimensions
Problem: No natural meaning to successive in many dimensions.
Successive in [TZS01]: move one point right.
The Generator: G(x1,x2)=f(x1,x2)..f(x1,x2+n10)
2n/2
f(x1,x2)..f(x1,x2+n10)
X1
X
2
Decoding ErrorsApply the Predictor in
parallel along a random line.
With high probability we get (½+ε)-fraction of correct predictions.*
Apply error correction:Learn all points on line
2n/2
*By pairwise independence properties of random lines.
v
x
v
v
x
x
v
v
x
v
v
v
v
v
v
v
v
v
A restriction of fto a line:
A univariate polynomial!
v
v
v
v
v
v
v
v
v
v
x
v
v
x
x
v
v
x
Low degree univariate polynomials have error-correcting properties!
Basic idea: Use decoding algorithms for Reed-Solomon
codes to decode and continue.
If #errors is small (<25%) then it is possible to recover
the correct values.
The predictor is only correct with probability ½+ε . May make almost 50% errors.
Too many errors Coding Theory: Not
enough information on on the line to uniquely decode.
It is possible to List-Decode to get few polynomials one of which is correct [S97].
[TZS01]: Use additional queries to pin down the correct polynomial.
2n/2
We also have the information we
previously computed!
v
x
v
v
x
x
v
v
x
Curves Instead of LinesLines: deg. 1
polynomials: L(t)=at+bCurves: higher deg. (nO(1))
C(t)=artr+ar-1tr-1..+a0
2n/2
Observation: f restricted to a low-degree curve is still a low-degree univariate polynomial.
Points on degree r curve are r-wise independent. (crucial for analysis).
A special curve with intersection properties.
Curve passes through: Few (random) points Successive points.
2n/2
This curve intersects itself when moved!
Recovering From Errors2n/2
No errors!
Previously computed.
(½+ε)-fraction of correct
predictions.
Just like before:
Query n10 successive curves.
Apply the predictor in parallel.
Recovering From Errors2n/2
No errors!
Previously computed.
(½+ε)-fraction of correct
predictions.
Lemma:
+ =
Given: - “Noisy” predicted values. - Few correct values.
We can correct!
Given: - “Noisy” predicted values. - Few correct values.
We can correct!
Recovering From Errors2n/2
Lemma:
+ =
We implemented an errorless Predictor!
Warning: This presentation is oversimplified. The lemma works only for randomly placed points.
Actual solution is slightly more complicated and uses two
“interleaved” curves.
Story so far… We can “error-correct” a predictor
that makes errors. Coding Theory: Our strategy gives
a new list-decoding algorithm for Reed-Muller codes [AS97,STV99].
Short version
List decoding
Given a corrupted message p: Pr[p(x)=f(x)]>ε
Output f1,..,ft s.t. f in list.
Our setup: List decoding with predictor
Given a predictor P: Pr[P(f(x-1),f(x-2),..,f(x-i))=f(x)]>ε
Use k queries to compute f everywhere.
Our setup: List decoding with predictor
Given a predictor P: Pr[P(x,f(x-1),f(x-2),..,f(x-i))=f(x)]>ε
Use k queries to compute f everywhere.
The decoding scenario is a special case when i=0 (predictor from empty prefix).
Our setup: List decoding with predictor
Given a predictor P: Pr[P(x,f(x-1),f(x-2),..,f(x-i))=f(x)]>ε
Use k queries to compute f everywhere.
To list-decode output all possible f’s for all 2k possible answers to queries.
Reducing the number of queries
Want: nO(1)
Make: n10 · |Curve|
How many queries?2n/2
2n/2
n10
Want to use short curves.
Using many dimensions1 dimension:
2n
2 dimensions: 2n/2
3 dimensions: 2n/3
d dimensions: 2n/d
d=Ω(n/log(n)) => length = nO(1)
Conflict?
Many Dimensions One Dimension
Error correction.Few queries.
Natural meaningto successive.
We’d like to have both!
A different Successor Function
Fd Vector-Space.
Base Field F.
Fd Extension Field of F.
Multiplicative group has a generator g.
Fd \ 0={1,g,g2,g3,…}
Successor(v)=g·v
Covers the space.
Many Dimensions One Dimension
1 g g2 g3 ……. gi …………………….
We compute f Everywhere!
A New Successor Function
Many Dimensions One Dimension
1 g g2 g3 ……. gi …………………….
Successor(v)=g·v
Covers the space.
We compute f Everywhere!
Invertible linear transform.
Maps curves to curves!
We use our decoding algorithm succesively.
Choice of successor function guarantees that we learn f at every point!
Nothing Changes!2n/2
Lemma:
+ =
The final Construction
Ingredients: f(x1,..,xd): a d-variate polynomial. g: generator of the extension field
Fd.Pseudo-Random Generator: )(),...,(),(),()(
102 vgfvgfvgfvfvG n
This is essentially the naive idea we started from.
*The actual construction is a little bit more complicated.
Query f at few short successive “special curves”.
Use predictor to learn the next curve with errors.
Use intersection properties of the special curve to error correct the current curve.
Successive curves cover the space and so we compute f everywhere.
Summary of prooff isn’t hard
Use P to compute f
Exists next-bit predictor P for G
G isn’t pseudo-random
Conclusion A simple construction of PRG’s.
(Almost all the complications we talked about are in the proof, not the construction!)
This construction and proof are very versatile and have many applications: Randomness extractors, (list)-decoding,
hardness amplification, derandomizing Arthur-Merlin games, unbalanced expander graphs.
Further research: Other uses for the naive approach for PRG’s. Other uses for the error-correcting technique.
That’s it…
What I didn’t show Next step: Use error corrected predictor to
compute f everywhere. The cost of “error-correction”:
We’re using too many queries just to get started.
We’re using many dimensions. (f is a polynomial in many variables).
It’s not clear how to implement the naive strategy in many dimensions!
More details from the paper/survey: www.wisdom.weizmann.ac.il/~ronens
Conclusion A simple construction of PRG’s.
(Almost all the complications we talked about are in the proof, not the construction!)
This construction and proof are very versatile and have many applications: Randomness extractors, (list)-decoding,
hardness amplification, derandomizing Arthur-Merlin games, unbalanced expander graphs.
Further research: Other uses for the naive approach for PRG’s. Other uses for the error-correcting technique.
That’s it…