Discrete Probability

1

Lecture 14

Discrete Probability

The study of probability uses special jargon

• A sample space in a nonempty set• An experiment is a process that produces a point in a sample space

2

sample space• An event is a subset of sample space• The event space is the power set of the sample space

Experiment: Toss a coin

Sample space: U={H,T}Event Space: P(U)={Ø, {H}, {T}, {H,T}}

Discrete Probability

Remark: Often some of the more interesting events have special name

Experiment: Toss Two Coin

3

Sample space: U={HH, HT, TH, TT}Event Space: Subsets of U

Named events: Match ={HH, TT}At least one head = {HT, TH, HH}

Uniform Probability Measure

Def: The uniform probability measure on a finite sample space S assigns to each event E the probability

||

||)(

S

EEp =

4

Sample space: U={H,T}Event Ø, {H}, {T}, {H,T}Probability 0 ½ ½ 1

|| S

Uniform Probability Measure

Def: The uniform probability measure on a finite sample space S assigns to each event E the probability

||

||)(

S

EEp =

5

Experiment: Toss Two CoinSample space: U={HH, HT, TH, TT}

Event Name At least one head one of eachEvent {HT, TH, HH} {HT, TH}Probability ¾ 2/4

|| S

Uniform Probability Measure

Def: The uniform probability measure on a finite sample space S assigns to each event E the probability

||

||)(

S

EEp =

6

Experiment: Roll a dieSample space: U={1,2,3,4,5,6}Event Name odd even 2 mod 3Even {1,3,5} {2,4,6} {2,5}Probability 3/6 3/6 2/6

|| S

Uniform Probability Measure

Def: The uniform probability measure on a finite sample space S assigns to each event E the probability

||

||)(

S

EEp =

7

Experiment: Roll two dieSample space: U={11,….,16,……,61,………66}Event Name double sum is 9Even {11, 22,….,66} {36,45,54,63} Probability 6/36 4/36

|| S

Uniform Probability Measure

8

Experiment: deal one card from deckSample space: standard 52 card deckEvent Name heart sevenEvenProbability 13/52 4/52

Probability of Complementary Event

Let E be an event in finite sample space S, under the uniform probability distribution. Then

)(1)( EpEp −=

9

Pf:1. |S| = |E| +|E|

)(1)( −=

Probability of a Union Event

Let E1 and E2 be an events in a finite sample space S, under the uniform probability distribution. Then

)()()()( 212121 EEpEpEpEEp ∩−+=∪

10

Pf:1. An integer is chosen from the internal [1,2,….,100]. Find

the probability that it is divisible either by 6 or by 15.

P(6\n v 15\n) = p(6\n) +p(15\n)-p(30\n)= 16/100 + 6/100 -3/100= 19/100

212121

Non-uniform Probability Measure

Here are two non-uniform probability measure for some familiar sample spaces.

The standard loaded coin has probability p(H)=0.8 and p(T)=0.2

11

The standard loaded die has sample space U={1,2,3,4,5,6}

Then assign the singleton {j} the probability j/21

Bernoulli distribution

A Bernoulli distribution is a probability measure on a sample space with exactly two points

Flip standard loaded coinSample space = {H,T}

12

Sample space = {H,T}Event Space = {Ø {H}, {T}, {H,T}}

pr({H}) = 4/5 pr ({T}) = 1/5

Bernoulli distribution

A Bernoulli distribution is a probability measure on a sample space with exactly two points

The standard loaded die induces a Bernoulli distribution with

13

with

pr(even) = 4/7 pr ({odd}) = 3/7

Conditional probability

)()|(

YEprYEpr

∩=

Let p be a probability distribution on a sample space U and let Y be an event. Then the conditional probability of event E given that event Y has occurred is

1414

)(

)()|(

Ypr

YEprYEpr

∩=

3

14/3

4/1

)(

)(

)(

)()|(

=

=

¬=

¬¬∧=¬

TTP

HHp

TTP

TTHHpTTHHp

Conditional Independence

)()|( EprYEpr =

Let pr be a probability distribution on a sample space U.Event E is probabilistically independent of event Y if andonly if

15

)|(

)(/)(

)(/)()Pr(

)(/)()()|(

EYpr

EprEYpr

EprYEprY

YprYEprEprYEpr

=∩=

∩=∩==

Conditional Independence

Roll two fair dice. Let Y be the event that the first die is a one and the event that the sum is odd. Then

pr(E) = 1.1/2.1/2=1/2

)()|(

∩= YEprYEpr

16

2

16

36/1

36/36/1

)16,14,12(

)()|(

=

=

=

=

=

p

YprYEpr

Random Variable

A random variable is a real valued function on the domain of a probability sapce.

Flip a coin three times. Let the X(t) be the number of heads that occurs. Then

17

heads that occurs. Then

X(TTT) = 0X(TTH)=X(THT)=X(HTT) =1X(THH)=X(HHT)=X(HTH)=2X(HHH)=3

Since a random variable is a function, it is not a variable and it is not random.

Bayes Theorem

• Start with the joint probability distribution:

toothache ¬¬¬¬toothache

catch ¬catch catch ¬catch

18

cavity 0.108 0.012 0.072 0.008

¬cavity 0.016 0.064 0.144 0.576

Inference by enumeration

• Start with the joint probability distribution:

toothache ¬¬¬¬toothache

catch ¬catch catch ¬catch

19

• P(toothache) = 0.108 + 0.012 + 0.016 + 0.064 = 0.2

cavity 0.108 0.012 0.072 0.008

¬cavity 0.016 0.064 0.144 0.576

Inference by enumeration• Start with the joint probability distribution:

toothache ¬¬¬¬toothache

catch ¬catch catch ¬catch

cavity 0.108 0.012 0.072 0.008

20

• Can also compute conditional probabilities:P(¬cavity | toothache) = P(¬cavity ∧ toothache)

P(toothache)= 0.016+0.064

0.108 + 0.012 + 0.016 + 0.064= 0.4

¬cavity 0.016 0.064 0.144 0.576

Normalization

toothache ¬¬¬¬toothache

catch ¬catch catch ¬catch

cavity 0.108 0.012 0.072 0.008

¬cavity 0.016 0.064 0.144 0.576

21

• Denominator can be viewed as a normalization constant α

P(Cavity | toothache) = α, P(Cavity,toothache) = α, [P(Cavity,toothache,catch) + P(Cavity,toothache,¬ catch)]= α, [<0.108,0.016> + <0.012,0.064>] = α, <0.12,0.08> = <0.6,0.4>

Thank You

22