Embed Size (px)
Citation preview
Randomized Algorithms
南京大学
尹一通
Course Info
• 尹一通
[email protected] [email protected]
• office hour:
804, Wednesday 2-4pm
• course homepage: http://tcs.nju.edu.cn/wiki
Textbooks
Rajeev Motwani and Prabhakar Raghavan.Randomized Algorithms.
Cambridge University Press, 1995.
Michael Mitzenmacher and Eli Upfal.Probability and Computing:
Randomized Algorithms and Probabilistic Analysis.
Cambridge University Press, 2005.
ReferencesCLRS
Feller An Introduction to Probability Theory and Its Applications
Häggström Finite Markov Chains and Algorithmic Applications
Alon and SpencerThe Probabilistic Method
“algorithms which use randomness in computation”
Randomized Algorithms
Why?
• Simpler.
• Faster.
• Can do impossibles.
• Leads to clever deterministic algorithms.
• Random input.
• Deterministic problem with random nature.
• ... ...
“algorithms which use randomness in computation”
Randomized Algorithms
How?
• To hit a witness.
• To fool an adversary.
• To simulate random samples.
• To construct a solution.
• To break symmetry.
• ... ...
Probability Principles• Basics: probability space, events, independence,
conditional probability;
• Moments and deviation: random variables, expectation, Markov’s inequality, variance, second moment method;
• The probabilistic method: threshold phenomena, Lovász local lemma;
• Concentration: Chernoff bound, martingales, Azuma’s inequality, bounded difference method;
• Random walks: Markov chain, hitting/cover time, mixing time, coupling, conductance.
Probability Space
�Sample space: Ω
Probability measure:
Pr : � � [0, 1]�
e��
Pr(e) = 1s.t.
event A � �
Pr(A) =�
e�A
Pr(e)probability
Probability SpaceKolmogorov (1933)
Sample space Ω: set of all elementary events (samples)
Set of events Σ: each event is a subset of Ω
� is closed under �, �, \.
�, � � �.
Probability measure
Pr(�) = 1
A � B = � � Pr(A � B) = Pr(A) + Pr(B)
limn��
Pr(An) = 0
for A1 � · · · � An � · · · with�
n An = �
impossible event, certain eventσ-algebra
(K1)(K2)
(K3)(K4)
(K5*)
Pr : � � [0, 1]
Probability Space
�
E : the outcome is even.Pr[E ]
= Pr[ ⇥ ⇥ ]= Pr[ ] + Pr[ ] + Pr[ ]
Notation:
predicate E
Pr[E ] = Pr(A)
event:
A = {x � � | E(x)}set
probability:
�, �¬ �\
��
�, �
� is closed under �, �, \.
�, � � �.
Pr(�) = 1
A � B = � � Pr(A � B) = Pr(A) + Pr(B)
(K1)(K2)(K3)(K4)
Pr[¬E ] = 1 � Pr[E ]
Pr[E1 � E2] = Pr[E1] + Pr[E2] � Pr[E1 � E2]
If E1 � E2, then Pr[E1] � Pr[E2].
� is closed under �, �, \.
�, � � �.
Pr(�) = 1
A � B = � � Pr(A � B) = Pr(A) + Pr(B)
(K1)(K2)(K3)(K4)
Pr(� \ A) = 1 � Pr(A)
If A � B, then Pr(A) � Pr(B).
Pr(A � B) = Pr(A) + Pr(B) � Pr(A � B)
Independence
Definition:Events E1, E2, . . . , En are mutually independentif and only if, for any subset I � {1, 2, . . . , n},
Pr
��
i�I
Ei
�=
�
i�I
Pr[Ei].
Definition:Two events E1 and E2 are independent if and only if
Pr [E1 � E2] = Pr[E1] · Pr[E2].
Model of Computation
Turing machine,random-access machine (RAM)
with access to a perfect coin
(a sequence of uniform and independent random bits)
standard model:
Monte Carlo vs Las Vegas
Monte Carlo Las Vegas
running time is fixedcorrectness is random
always correctrunning time is random
Two types of randomized algorithms:
Monte Carlo algorithmsrunning time is boundedcorrectness is random
• MC algorithm with one-sided error:
if answer=yes: always returns “yes”;if answer= no: Pr[returns “no”]≥ε;
for decision problems (with “yes-or-no” answer):(false positive)
repeat independently for t times“yes” if all t trials return “yes”;“no” if otherwise;return {
if answer=yes: always returns “yes”;if answer= no: Pr[returns “yes”] � (1 � �)t = �
t = O( 1� log 1
� )
Monte Carlo algorithmsrunning time is boundedcorrectness is random
• MC algorithm with two-sided error:
if answer=yes: Pr[returns “yes”]≥1/2+ε;if answer= no: Pr[returns “no”]≥1/2+ε;
for decision problems (with “yes-or-no” answer):
repeat independently for t times
return majority answer;
Monte Carlo algorithms
• MC algorithm with two-sided error:
Pr[correct]≥1/2+ε;
for decision problems (with “yes-or-no” answer):
repeat independently for t times
Pr[exact i trials are correct]
Pr[≤t/2 trials are correct]
=
�t/2��
i=0
�t
i
� �1
2+ �
�i �1
2� �
�t�i
��t/2��
i=0
�t
i
��1
2+ �
�t/2 �1
2� �
�t/2
=
�1
4� �2
�t/2 �t/2��
i=0
�t
i
��
�1
4� �2
�t/2
2t = (1 � 4�2)t/2 = �
t = O( 1�2 log 1
� )
=�t
i
� �12 + �
�i �12 � �
�t�i
Las Vegas algorithmsalways correct
running time is random
LV algorithm MC algorithm
truncation
?
Checking Matrix Multiplication
A B C× =?
three n×n matrices A, B, C:
best matrix multiplication algorithm: O(n2.376)
O(n2.3727)
Checking Matrix Multiplication
A B C× =?
three n×n matrices A, B, C:
best matrix multiplication algorithm: O(n2.376)
O(n2.3727)
O(n�)
best matrix multiplication algorithm:
Checking Matrix Multiplication
A B C× =?
three n×n matrices A, B, C:
Freivald’s Algorithmpick a uniform random r ∈{0,1}n;check whether A(Br) = Cr ;
time: O(n2) if AB = C, always correct
O(n2.3727)
Freivald’s Algorithmpick a uniform random r ∈{0,1}n;check whether A(Br) = Cr ;
if AB = C, always correct
if AB ≠ C,
Pr[ABr = Cr] = Pr[Dr = 0]
D = AB � C �= 0n�nlet
D ri
(Dr)i =n�
k=1
Dikrk =0
rj = � 1
Dij
n�
k �=j
Dikrk
2n
2n�1
=1
2
Dij �= 0say
�
Freivald’s Algorithmpick a uniform random r ∈{0,1}n;check whether A(Br) = Cr ;
if AB = C, always correct
If AB ⇥= C, for a uniformly random r � {0, 1}n,
Pr[ABr = Cr] � 12.
Theorem (Freivald, 1979)
repeat independently for 100 times
time: O(n2) if AB ≠ C, error probability ≤ 2�100
Polynomial Identity Testing (PIT)
Input: two polynomials f, g � F[x] of degree dOutput: f � g?
Input: a polynomial of degree dOutput:
f � F[x]
f � 0?
f is given in product form or as black-box
simple deterministic algorithm:
check whether f(x)=0 for all x � {1, 2, . . . , d + 1}
A degree d polynomial has at most d roots. Fundamental Theorem of Algebra:
Randomized Algorithmpick a uniform random r ;check whether f(r) = 0 ;
∈S S � F
Input: a polynomial of degree dOutput:
f � F[x]
f � 0?
A degree d polynomial has at most d roots. Fundamental Theorem of Algebra:
Randomized Algorithmpick a uniform random r ;check whether f(r) = 0 ;
∈S S � F
f �� 0if
Pr[f(r) = 0] � |S|d
|S| = 2d
=1
2
Checking Identity
database 1
database 2
Are they identical?
北京
南京
Communication Complexity(Yao 1979)
Li LeiHan Meimei
EQ : {0, 1}n ! {0, 1}n " {0, 1}
# of bitscommunicated
a b
f(a, b)
EQ(a, b) =
!1 a = b
0 a != bThere is no deterministic communication protocol solving EQ with less than n bits in the worst-case.
Theorem (Yao, 1979)
Communication Complexity
Li LeiHan Meimei
a b!{0, 1}n !{0, 1}n
f =n�1�
i=0
aixi g =
n�1�
i=0
aixi
pick uniform random r ∈[2n]
r, g(r)
f(r)=g(r) ?
one-sided error � 1
2
by PIT:
# of bit communicated: too large!
Communication Complexity
Li LeiHan Meimei
a b!{0, 1}n !{0, 1}n
f =n�1�
i=0
aixi g =
n�1�
i=0
aixi
pick uniform random r ∈[p]
r, g(r)
f(r)=g(r) ?
k = �log2(2n)�
p � [2k, 2k+1]choose a prime f, g � Zp[x]let
