MitzenmacherPROBABILITY REFRESHER upfal

I H T

zrFfield setof Lcf SbeventsLH3 2T

r

E EzEz

A collection F of subsets of 52 is called G field ifi DEFCir A EF ACE F

nCiii Aa Az An E F W A E Fi L

4maynot beUnion bound pairwisedisjoint

Inclusion exclusion principle E E2

chain rule Pr II Ei Pr Ei l j EjPro Ei Ez IED Pro I E n Ez i Pr Ek l ni Ei

Repeatedly choosing random numbers acc toa given distribution samplingwith replacement simple to codewithout replacement gives slightly betterbounds

TLIP

Discrete Random Variables Can2and Expectation M O

Defn A random variable X is a functionR V X I IR A discrete R.V is a R V

that takes finite or countably infinitenumber of values

Defn Expectation E X3 i lPCX i

Properties of expectationthem 2.1 C linearity of ExpectationsFor any finite collection of discrete RVsXa Xz Xn with finite expectations

IE IE Xi IE El XiNote we don't need these Rvs to be independent

Lem 2 2 For any constant and discreteRV X E CX C IEEXT

Defn 2.4 convex function A function f IR IRis said to be convex if for any sea Nz and0 E A E I s f case I a Nz E a fGee t I a fCazn

Fun Intense Is 9.5e g fCa x 24 N x3 is

et HelP for PZ 1 partlyconvex

Tkm 2.4 Jensen's Inequality If f is a convex

function then E f x 3 f E x

corollary E x 233 IEEX

Bernoulli Binomial and Geometric RV

Bernoulli Y be a RV s E Y I w P PO w p y PIE Y p I l P O p

e g one coin toss can be modeled by BernoulliAlso called indicator random variable

Binomial X Bin n p is a random variable

taking the values 0,1 2 N andIP X K Ya pk l P

n k where Ocp 21e g n coin toss How many headsuseful in sampling it x E EhXi EH i

ripGeometric X Geom p is a geometric RVif takes values 1 2 3 with IP X K p l p K

e g numbers of coin flips till you get a first treadIEEXT Hp

BE uIE.ommeenntts and EDeviation 94Variance and moments of a random variable

like KthderivativeK the moment of a RV X is F XK forfunctionsvariance of vars X EfcX E 72 ECx2 CECxDcovariance of Rus X YCor X Y IE CX E X Y EY

Th m 3.2 Van EX Y Var X Var Y t 2 CorCXY

Sf X Y are indep then Cor CX Y O and Thm 3.2is like linearity of expectationThem 3.3 IE X Y E X E Y for indep X Y

Theorem 3 I Markov's inequalityFor a nonnegative random variable X t t 0

pm x t E EI or Fn x E EX E Lt E

G G Define fee q for a Ctro fCx for a t

Define gcse Mt1Fact t gGe Z f Cr

Fact 2 ECfcxD O Procx ET t 2 the x t ppCx t

Prs x t E f CX C Fact 27E E g Cx Fact I

E XIE1 Efx scalingE

Theorem 3 2 Chebyshev's inequalityFor any a O p Ix Ex 1 a s Var

of9th w log assume EX 0

_g qVars X L By scalingSo It XZ L

Define fGet 0 for Kl Ct g se I1 for bet t T2

Pro 1 1 t IE Cf xDE IE Eg xD As 9 a z feelE X E xD 1

ApplicationXi 10 if i'th coin flip is head

X Xi denote headselse in a coin flips

E EX up Mz Var X E var Xi n

Markov IP X 374 E F q 3Msihzeusq.netefx

Chebyshev IP X 3374 E P IX EX 1374Tm.IEaT.ndEYn z Yn4q Inriffed

Chernoff and IHoeffding BoundsChernoff bounds are studied for the tail distributionof a sum of independent O 1 random variableswhich are also known as Poisson trials

Note Bernoulli trials are special cases of Poissontrials where independent O 2 RVs have same distr

O 1 RVsTheorem 4.4 4.5 9Let Xi Xn be independent Poisson trials s tP Xi 1 Pi Let X Xi and µ EX ThenPoisson trials deviations above mean

V 8 O P X Clt 8 te s f g ote orE 0,11 IP X 3 Clt d te s e te 3

for 1 E S g e148 3

for O E S g e StgCI187142

for 1236 te p x R g z R s e04 2 87

Poisson trials deviation below mean

to C Co 1 P C x E l 8 te f f y E et

Combined deviation around mean

it 8 E Q1 IP I x te l 3 orte s 2 e M 13

Additive bounds when s are identical i f E o

a 1pct EX i 7 Pt E e e III P E

b PC's Exists E e GEE

III J

Special cases Not 0 1 RvXa Xz Xn be independent Rvs with X Xi and

Known asIP Xi 1 IP Xi 1 Iz s then RademacherV a 0 IP X a e a42n distr

IP X E a g e a42n

IP 1 1 za E zeaten

IP Xi 1 P Xi O L te EX Zthen

I t a O P Y z te ta s e Zak Additive

T 8 o P y 1 8 µ g e Epe Multiplicative43missingunlike

Ciii t a ECO MI IP y E te a E e Zak AdditiveD A JE CO 17 IPC x e l S te s e 82M multiple

Hoeffding Bounds General RVs w BoundedRangeet X Xn be independent random variabless t V I E i E n E Xi µ and IP a Xie b 1

ThenP Int Xi te e e z e 2nE4Cb

as

more general subsumes most boundsLet X Xn be independent random variabless t V I E i E n E Xi te and IP 9 Exists 1

nn ntheIP IE x E tea I e s ze 2EY.IECbi Ail1 1 i i

There are many more tailors made conc inedsurvey by McDiarmid

Independent Bounded Difference InequalityLet X Xa Xz Xn be a family of independentrandom variables with XjE Aj for j 1 2 n

and 0 Ijn Aj R be a function such that

Cse 0 Can e g whenever the vectors se se

differs only in the j th coordinate Then f t O

IP 4 Cx E Cola t E e HYEin g

Example coin flipFrom Chebyshev IP l X 2 I 3 E 4h

Chernoff IP I X 2 I 3 2 s 2e te843

2eMd 2 exp L y ng 2 3 E 2e n124 tighten

In fact we can show the deviations from the meanP I x 2 13 ziF E 2 exp f 2 64

Yn

We will later use Martingales to obtainconcentration for Rvs that are not independent

Conditional ExpectationDefn E LY I Z z E y P Y y 1 Z 3

yIE Y IZ is a RV taking value IECTI Z Z when Z zFor RVs X and Y E X PmCT y E X Y y

conditional variancevars CX1Y E x21 Y E EXIT

Properties of conditional expectationsLet x Y Z be random variables a b E IR g IR IR then

linearity IE xx BY l Z a E Xl Z BE YI Zmonotonicity X E Y it X I Z E E T I Z

In particulars if X 30 E x I 2 3 O

independence if x z are independent thenE X I Z E X

conditional Jensen's ineg If R IR is convexE 4CX IZ 3 E XIZ

rule of average conditional Law of total expectationEy It XI YI E x used crucially inMartingalesE E x g Y Y g Y E EXITIn particular it g Y ly g y

E I Y gCY E XI Y

E it XI Y Z f Y IE XI Y

Law of Total VarianceVar X Ey Var XI Y Vary IECX1YAbove we assume all the expectations exist