Embed Size (px)
Citation preview
Randomized Algorithms
南京大学
尹一通
Course Info
• 尹一通
[email protected] [email protected]
• office hour:
804, Wednesday 2-5pm
• course homepage: http://114.212.87.200
Textbooks
Rajeev Motwani and Prabhakar Raghavan.Randomized Algorithms.
Cambridge University Press, 1995.
Michael Mitzenmacher and Eli Upfal.Probability and Computing:
Randomized Algorithms and Probabilistic Analysis.
Cambridge University Press, 2005.
ReferencesCLRS
Feller An Introduction to Probability Theory and Its Applications
Häggström Finite Markov Chains and Algorithmic Applications
Alon and SpencerThe Probabilistic Method
References
• Markov Chain Monte Carlo.
http://www.cs.berkeley.edu/~sinclair/cs294/f09.html
• Expander Graphs and Their Applications.
http://www.cs.huji.ac.il/~nati/PAPERS/expander_survey.pdf
• Pseudorandomness.
http://people.seas.harvard.edu/~salil/pseudorandomness/
“algorithms which employ a degree of randomness as part of its logic” —— wikipedia
“algorithms that flip coins”
Randomized Algorithms
How? Why?
Quicksort
Qsort(π):choose a pivot x = π[1];
•construct π1 with all π1[i ] < x ,
π2 with all π2[i ] > x ;Qsort(π1) and Qsort(π2);
input: permutation π of n numbers
Complexity: number of comparisonst(!)
worst-case: max!
t(!) = !(n2)
average-case: E[t(!)] = !(n log n)
random Π
Asymptotic Notationsf IVL g IZM�JW\P�N]VK\QWV[�WN�XW[Q\Q^M�QV\MOMZ["
• g � 7 (f)" �C > 0 []KP�\PI\g(n) � Cf(n) NWZ�ITT�[]NNQKQMV\Ta�TIZOM n�
• g � � (f)" f = 7(g)�
• g � � (f)" g � 7 (f) IVL g � � (f)�
• g � W (f)" g � 7 (f) J]\ g �� � (f)�
• g � � (f)" g � � (f) J]\ g �� 7 (f)
Quicksortinput: permutation π of n numbers
n
|π1| |π2|
Recursion tree:
balanced tree: !(n logn)
complexity: sum of labels
Qsort(π):choose a pivot x = π[1];
•construct π1 with all π1[i ] < x ,
π2 with all π2[i ] > x ;Qsort(π1) and Qsort(π2);
choose median as pivot^
rough
Randomized Quicksortinput: permutation π of n numbers
Qsort(π):
choose uniform random r ∈[n];let pivot x = π[r];
•construct π1 with all π1[i ] < x ,
π2 with all π2[i ] > x ;Qsort(π1) and Qsort(π2);
Complexity: T (!) random variable
!!, E[T (!)] = E[t(!)]
!!, E[T (!)] = E[t(!)]
Qsort(Π):
choose a pivot x = Π[1];
•construct Π1 with all Π1[i ] < x ,
Π2 with all Π2[i ] > x ;Qsort(Π1) and Qsort(Π2);
n
|Π1| |Π2|
Principle of deferred decisionThe decision of random choice in the random input
is deferred to the running time of the algorithm.
decision of Π[1] is deferred to the running time
![1] ! ![r] random position in a fixed permutation
fixed position in a random permutation
!!, E[T (!)] = E[t(!)]
deterministic algorithm:
randomized algorithm:
worst-case:average-case:
max!
t(!)
E[t(!)]
max!
Er[time(Alg(r,!))]
worst-case or average-case?
average (typical) behavior
worst-case input
random bits r
max!
E[T (!)] = E[t(!)] = O(n log n)
Fooling an Adversaryinput
input
input
input
input
Any deterministic algorithm may work poorly on some bad input.
A
A
A
A
A
A
A
Fooling an Adversaryinput
input
input
input input
A
Any deterministic algorithm may work poorly on some bad input.
Fooling an Adversaryinput
input
input
input
A randomized algorithm is a probability distribution over lots of deterministic algorithms.
No adversary can win for sure (neither can algorithm!).
A is adistribution
over
A
A
A
Primality TestInput: positive integer n
Output: “yes” if n is prime“no” if n is composite
efficient algorithm: running time is polynomialof the length of input
(log n)O(1) time for primality test
for k = 1, 2, . . . ,�
n
check whether n�� k
efficient?
an!1 ! 1 (mod n)
n is
Fermat’s little theorem
Pierre de Fermat amateur mathematician
a � {1, 2, . . . , n � 1}�
prime
an!1 ! 1 (mod n)
a � {1, 2, . . . , n � 1}
n is composite
�
Primality test:compositeness
find such a proof a • How to find it?
• Does it exist?
Fermat Test• Choose a random a � {1, 2, . . . , n � 1}.
• If an�1 �� 1 (mod n), return composite.
• Else return probably prime.
Carmichael numbern is composite, and
an!1 ! 1 (mod n)a � {1, 2, . . . , n � 1}�
fools the Fermat test
or Carmichael
non-Carmichael
^
Fermat Test• Choose a random a � {1, 2, . . . , n � 1}.
• If an�1 �� 1 (mod n), return composite.
• Else return probably prime.
n is prime:
n is non-Carmichael composite:
n is Carmichael:
return probably prime
incorrectly return probably prime
good a bad a an!1 ! 1an!1 !" 1
Fermat Test• Choose a random a � {1, 2, . . . , n � 1}.
• If an�1 �� 1 (mod n), return composite.
• Else return probably prime.
Z!n = {a | 1 ! a ! n" 1 # gcd(a, n) = 1}
multiplicative group modulo n
B = {a ! Z!n | an"1 " 1 (mod n)} closed under
multiplication mod n
B is subgroup of Z�n |Z�
n|�� |B|
Fermat Test• Choose a random a � {1, 2, . . . , n � 1}.
• If an�1 �� 1 (mod n), return composite.
• Else return probably prime.
n is prime:
n is non-Carmichael composite:
n is Carmichael:
return composite
return probably prime
with probability ≥1/2
incorrectly return probably prime
nontrivial square root of 1
a2 � 1 (mod n)
Fermat’s proof of compositeness:
an�1 �� 1 (mod n)
Another proof of compositeness:
incomplete
a �� ±1 (mod n)
If n is prime, 1 does not have nontrivial square root.Theorem
a2 � 1 (mod n) (a + 1)(a � 1) � 0 (mod n)
(a + 1)(a � 1)�� n
choose a random a � {1, 2, . . . , n � 1};
decompose n � 1 as n � 1 = 2tm with odd m;
compute am, a2m, . . . , a2im, . . . , a2tm (mod n);
Miller-Rabin test
Fermat test
nontrivial square root of 1
n is prime: n is non-Carmichael composite:
n is Carmichael:return composite
return probably prime
with probability ≥1/2Fermat test
return composite with probability ≥1/2
if �i, a2im � 1 but a2i�1m �� ±1, return composite;
else return probably prime;
if an�1 = a2tm �� 1 (mod n), return composite;
all bad a ∈ a proper subgroup
Nondeterminism
For any composite n, there exist a ! {1, 2, . . . , n" 1}
an!1 !" 1 (mod n) or
where m is odd and n � 1 = 2tm.
�i, a2im � 1 (mod n) but a2i�1m �� ±1 (mod n)
^over half
certificate (proof, witness) of compositeness of n efficiently verifiable
PRIME ! co-NP
COMPOSITE ! NP
Randomization: redundancy of certificates
in 2002, PRIME ∈ P(AKS test)
Miller-Rabin test
(time) Complexity
P
NP co-NP
complexity of algorithm
complexity of problem
TIME(A) = maxx
TIME(A(x))
TIME(P ) = minA for P
TIME(A)
complexity class: P, NP, ...
Performance
Hardness
worst-case running time
complexity of optimal Alg
poly-time computable decision problems
poly-time verifiabledecision problems
P
NPco-NP
Computation vs Verification
poly-timeAlg
f: input {yes, no}
x f(x)
poly-timeverifier
yes x yes
∃ proof yNP
P
Randomized Complexity
Miller-Rabin prime n “prime”
composite n “composite”with probability
≥1/2
Monte Carlo vs Las Vegas
Monte Carlo Las Vegas
running time is fixedcorrectness is random
always correctrunning time is random
Two types of randomized algorithms:
Monte Carlo vs Las Vegas
A
A
A
input
input
input
input
input
Adeterm
random
Las Vegas : on which Alg is slow
Monte Carlo : on which Alg is wrong
Randomized Complexityf: input {yes, no}
x
complexity class: ZPP
Zero-error Probabilistic Poly-time
expectedpoly-time
f(x)
Las Vegas:
poly-time
Randomized Complexity
one-sided error
two-sided error
< 1
< 1/2
ε
f: input {yes, no}
yes x yes
yes x yes with prob 1-εε
no x
no x no with prob 1-ε
no
with prob 1-ε
complexity class: RP co-RP
complexity class: BPP
poly-time
RandomizedPoly-time
Bounded-error Probabilistic Poly-time
Monte Carlo:constant
constant
Randomized Complexitydeterministic Alg is randomized Alg:
RP ! BPP, co-RP ! BPPone-sided error is two-sided error:
redundancy of certificates ⇒ existence of certificates:RP ! NP, co-RP ! co-NP
Open problem: BPP vs. P
Does randomness help computation?
P ! ZPP,P ! RP,P ! co-RP,P ! BPP
truncate Las Vegas ⇒ Monte CarloZPP ! RP,ZPP ! co-RP
Graph Coloringgraph G(V,E)
coloring ! : V ! [q]
!uv " E, !(u) #= !(v)
q-colorabilityNP-hard in general
max degree Δq >Δ
q <Δ NP-hard
q =ΔtrivialBrook’s theorem
Sampling Graph Colorings
sampling is at least as hard as existenceq >Δ
start with an arbitrary coloring σ;Repeat for t steps:
pick v ∈V and c ∈[q] uniformly at random;let σ(v)=c if the resulting σ is a coloring;
Conjecture: σ is very close to uniform random for t =O(n log n)
Random walk
Countingcount the number of colorings of a graph
f(G) = # of colorings of G
#P-hard
#P
NP existence of certificate
number of certificates
analog of NP for counting
poly-time randomized algorithm for approximate counting colorings
sampling coloringin poly-time
Leslie Valiant
Checking Identity
database 1
database 2
Are they identical?
北京
南京
Communication Complexity(Yao 1979)
Li Lei Han Meimei
x y
f(x, y)
EQ(x, y) =
!1 if x = y,
0 otherwise.
EQ : {0, 1}n ! {0, 1}n " {0, 1}
# of bitscommunicated
Fingerprinting
Li Lei Han Meimei
x y! [2n] ! [2n]p
y mod p
uniformly pick arandom primep ! [k]
Communication Complexity: = O(log n)O(log k)
k = poly(n)
x = yif
if x != y
x ! y (mod p)
x ! y? (mod p)
for some
always correct
error probability = 1poly(n)
Public Coin
Li Lei Han Meimei
x y! !{0, 1}n {0, 1}n
x = y?
r ! {0, 1}nrandom
error probability 1/106 in O(1) communication
