24
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected]) 5-1 1036: Probability & Statistics 1036: Probability & 1036: Probability & Statistics Statistics Lecture 5 Lecture 5 Some Discrete Some Discrete Probability Distributions Probability Distributions

1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

  • Upload
    others

  • View
    13

  • Download
    0

Embed Size (px)

Citation preview

Page 1: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-1

1036: Probability & Statistics

1036: Probability & 1036: Probability & StatisticsStatistics

Lecture 5 Lecture 5 –– Some Discrete Some Discrete Probability Distributions Probability Distributions

Page 2: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-2

Discrete Uniform Distribution• If a Random variable X have values of x1, x2,… xk, with equal

probabilities, then the discrete uniform distribution is given by

• the discrete uniform distribution depends on the parameter k.

• The mean and variance of the discrete uniform distribution f (x;k) are

kxxxxk

kxf ,,, ,1);( 21 K==

k

xk

ii∑

== 1µ( )

k

xk

ii∑

=

−= 1

2

σ

Page 3: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-3

Bernoulli Process• The experiment consists of n repeated trials• Each trial results in an outcome that may be classified as a

success or a failure (2 possible outcomes)• The probability of success, denoted by p, remains constant

from trial to trial• The repeated trials are independent

• The number X of successes in n Bernoulli trials is called a binomial random variable.

• The probability distribution of the binomial random variable X is

nxqpqpxn

pnxb xnx ,,2,1,0 ,1 where,),;( K==+⎟⎟⎠

⎞⎜⎜⎝

⎛= −

Identically, independently, distribution (i.i.d.)

Page 4: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-4

Binomial Distribution • The binomial distribution derives it name form the fact

that the n+1 terms in the binomial expansion of (p+q)n

correspond to the various values of b(x; n, p) for x=0,1,2,…,n. That is,

• For simplicity, we define

1 ),;(),;2(),;1(),;0(

210)( 221

=++++=

⎟⎟⎠

⎞⎜⎜⎝

⎛++⎟⎟

⎞⎜⎜⎝

⎛+⎟⎟

⎞⎜⎜⎝

⎛+⎟⎟

⎞⎜⎜⎝

⎛=+ −−

pnnbpnbpnbpnb

pnn

qpn

pqn

qn

qp nnnnn

L

L

∑=

=r

xpnxbpnrB

0),;(),;(

Page 5: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-5

Example 5.5• The probability that a patient recovers from a rare blood

disease is 0.4. If 15 persons infected, then the probability of (a). at least 10 recovery, (b). from 3 to 8 survive, and (c). exactly 5 survive?

Sol: Let X be the number of persons that recover

( ) ( ) )40159(14.0,15;10 .15

10.,;BxbXPa

x−==≥ ∑

=

( ) ( ) )4.0,15;2()4.0,15;8(4.0,15;83 .8

3BBxbXPb

x−==≤≤ ∑

=

( ) )4.0,15;4()4.0,15;5()4.0,15;5(5 . BBbXPc −===

Page 6: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-6

Example 5.6• A retailer purchases an electronic device with a defective

rate of 3%. The inspector randomly picks 20 items. What is the probability of at least 1 defective item among these 20?

• Suppose that the retailer receives 10 shipments in a month and the inspector randomly tests 20 devices per shipment. What is the probability that there will be 3 shipments containing at least one defective device?

4562.0)03.0,20;0(1)0(1)1( =−==−=≥ bXPXP

73 )4562.01(4562.03

10)3( −⎟⎟

⎞⎜⎜⎝

⎛==YP

)4562.0,10;(yb

Page 7: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-7

µ, σ of the Binomial Distribution

• The mean and variance of the binomial distribution b(x;n,p)are µ=np, σ2=npq.

Proof in a binomial experiment, the number of successes can be written as the sum of the n independent indicator variables. Then

nIIIX +++= L21

npIEIEIEIIIEXE nn =+++=+++== )()()()()( 2121 LLµ

pqpIE j =×+×= 01)(

npqIII n =+++= )()()( 22

21

22 σσσσ L

pqpppqpuIEIjIJj =−=−×+×=−= 2222222 01)()(σ

We have

Page 8: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-8

Multinomial Distribution• The binomial experiment becomes a multinomial experiment

if we let each trial have more than 2 possible outcomes.• Suppose that there are k possible outcomes E1, …, Ek with

probabilities p1, …, pk. For n experiments, we have that E1occurs x1 times; E2 occurs x2 times; …; Ek occurs xk times.

• Let rvs. X1, X2, …, Xk denote the number of occurrences for E1, …, Ek. Then, we have the multinomial distribution

k

k

xk

xx

k

xk

xx

kkk

pppxxx

n

pppxxx

nnpppxxxf

L

LKK

21

21

2121

21,..,21

2121

!!...!!

,),,,,;,,,(

=

⎟⎟⎠

⎞⎜⎜⎝

⎛=

∑∑==

==k

ii

k

ii pnx

11

1 and where

nkppp )( 21 +++ L

Page 9: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-9

Remarks• If the binomial is applied to sampling from a lot of

items, for example deck of cards, the sampling must be done with replacement of each item after it is observed.

• What the distribution we have if the sampling done without replacement?

• In general, this is known as a hyper-geometric experiment– A random sample of size n is selected without

replacement from N items– k of N items may be classified as successes and N−k are

classified as failures

Page 10: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-10

Hypergeometric Distribution• The probability distribution of the hypergeometric random variable

X, the number of successes in a random sample of size n selected from N items of which k are labeled success and N-k labeled failure, is

• Example: lots of 40 components each are called unacceptable if they contain as many as 3 defectives or more. The procedure for sampling the lot is to select 5 components at random and to reject the lot if a defective is found. What is the probability that exactly 1 defective is found in the sample if there are 3 defective in the entire lot?

• Sol:

},min{)}(,0max{ ,),,;( nkxkNn

nN

xnkN

xk

knNxh ≤≤−−

⎟⎟⎠

⎞⎜⎜⎝

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

=

)3,5,40;1(h

Page 11: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-11

Mean of the HypergeometricDistribution

Nnk

nN

xnkN

xk

Nkn

nN

xnkN

xk

k

nN

xnkN

xk

xknNxxhXE

n

x

n

x

n

x

n

x

=

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

=

⎟⎟⎠

⎞⎜⎜⎝

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

=

⎟⎟⎠

⎞⎜⎜⎝

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

==

∑ ∑

∑ ∑

= =

= =

11

11

11

),,;()(

1 1

0 0

1

)1,1,1;(

111

)1()1(11

0

1

0

=

−−−=

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

−−−⎟⎟⎠

⎞⎜⎜⎝

⎛ −

∑∑−

=

=

n

y

n

y

knNyh

nN

ynkN

yk

Let y=x-1

Page 12: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-12

Variance of the HypergeometricDistribution

1)1(

1

1

11

1

11

1

11

11

11

),,;()(

00

00

1

0 1

22

⎟⎠⎞

⎜⎝⎛

−−−

−⋅=

⎟⎟⎟⎟⎟

⎜⎜⎜⎜⎜

⎟⎟⎠

⎞⎜⎜⎝

⎛ −

⎟⎟⎠

⎞⎜⎜⎝

⎛ −⎟⎟⎠

⎞⎜⎜⎝

⎛−−

−−

⎟⎟⎠

⎞⎜⎜⎝

⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛−−

=

⎟⎟⎟⎟⎟

⎜⎜⎜⎜⎜

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛ −⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛−−

=

⎭⎬⎫

⎩⎨⎧

⎟⎟⎠

⎞⎜⎜⎝

⎛ −−⎟⎟⎠

⎞⎜⎜⎝

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

=

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

==

∑∑

∑∑

∑ ∑

==

==

=

= =

Nnk

nnN

Nkn

nN

Nkn

nN

xk

xnkN

xn

nN

nN

xk

xnkN

xnN

Nkn

nN

xk

xnkN

x

nN

xk

xnkN

xNkn

xk

xk

nN

xnkN

xNkn

nN

xnkN

xk

xNknknNxhxXE

n

x

n

x

n

x

n

x

n

x

n

x

n

x

⎟⎟⎠

⎞⎜⎜⎝

⎛−⎟⎟⎠

⎞⎜⎜⎝

⎛ +=⎟⎟

⎞⎜⎜⎝

⎛− r

nr

nr

n 11 ( )

NkN

Nkn

NnN

NNkNnN

Nnk

Nnk

Nnk

NknnkN

Nnk

XEX X

−⋅⋅⋅

−−

=

⎟⎟⎠

⎞⎜⎜⎝

⎛−

−−=

−−

−−+=

−=

1

1))((

1

)()( 222 µσ

Page 13: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-13

Remarks

• If n<<N (The good approximation is n/N≤0.05 ), then the hypergeometric distribution approaches to the binomial distribution. That is

( )NnkXE ==µ

⎟⎠⎞

⎜⎝⎛ −⋅⋅

−−

=Nk

Nkn

NnN 11

( ) npNnkXE ===µ ⎟

⎠⎞

⎜⎝⎛ −⋅=

Nk

Nkn 12σ

Page 14: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-14

Multinomial Hypergeometric Distribution

• If N items can be partitioned into k cells A1, …, Ak with elements a1, …, ak, respectively, then the probability distribution of the random variables X1, …, Xk representing the number of elements selected from A1, …, Ak in a random sample of size n is

⎟⎟⎠

⎞⎜⎜⎝

⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

=

nN

xa

xa

xa

nNaaaxxxf k

k

kk

L

KK 2

2

1

1

2121 ),,,,,;,,,(

∑∑==

==k

ii

k

ii Nanx

11

and with

Page 15: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-15

Negative Binomial Distribution• Consider the binomial experiment. Instead of finding k

successes among n trials, we continue the trials until a fixed number of successes occur.

• n is not a fixed number now, but a random variable, X• we are now interested in the probability that the kth success

occurs on the xth trial.• This is called negative binomial experiments• If repeated independent trials can result in a success with

probability p and a failure with probability q, then the probability distribution of the random variable, X, the number of the trial on which the kth success occurs, is

,...1, ,11

11

),;(* 1 +=⎟⎟⎠

⎞⎜⎜⎝

⎛−−

=⎟⎟⎠

⎞⎜⎜⎝

⎛−−

= −−− kkxqpkx

pqpkx

pkxb kxkkxk

Page 16: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-16

Example 5.16Playoff rule: team wins 4 games in 7 games a winnerTeam A has probability 55% of winning over team B

a. Probability that team A win in 6th gameX: number of games to win, a RV, x=6, p=0.55k=4, min. number to win

( ) 464*

1416

55.0,4;6 −⎟⎟⎠

⎞⎜⎜⎝

⎛−−

= qpb =18.5%

b. Probability that team A wins

( ) ( ) ( ) ( ) ( )55.0,4;755.0,4;655.0,4;555.0,4;44 **** bbbbXP +++=≥

=60.8%

Page 17: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-17

Geometric Distribution• Consider the special case of the negative binomial

distribution with k=1, we have a probability distribution for the number of trials required for a single success.

• The function is just the probability distribution of the random variable X, the number of the trial on which the first success occurs.

• The mean and variance of the geometric distribution are

,...3,2,1 ,);(),1;(* 1 === − xpqpxgpxb x

22 1 ,1

pp

p−

== σµ

Proof ….

Page 18: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-18

Geometric Distribution1

11

1 =−

== ∑∑∞

=

qq

qpq

qppq

x

x

x

x

pupqxpqxpq

qxpqqxpq

x

x

x

x

x

x

x

x

x

x

===−

=⇔=

∑∑∑

∑∑

=

−∞

=

=

−−

1

have wesides,both gsubtractin

1

1

11

1

11 µµ

pu 1=right shift

right shift

L

L

++++=

++++=432

32

432 432

pqpqpqpqqpqpqpqp

µ

µ

Page 19: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-19

σ2 of Geometric Distribution• Instead of considering E(X2), we consider the following

2222

2

2

32

432

32

32

)(

)()(21)]1([

)32(2 642)]1([)]1([

34231201 )]1([34231201 )]1([

34231201)]1([

pqXE

XEXEpq

pXXE

pqpqqqpqpqpqXXqEXXE

pqpqpqpqXXqEpqpqpqpXXE

pqpqpqpXXE

=−=

−=×=−

+++=

+++=−−−

+⋅⋅+⋅⋅+⋅⋅+⋅⋅=−

+⋅⋅+⋅⋅+⋅⋅+⋅⋅=−

+⋅⋅+⋅⋅+⋅⋅+⋅⋅=−

µσ

L

L

L

L

L

Page 20: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-20

Poisson process• Poisson experiments : Number of outcomes X

occurs during a given time interval• Properties of Poisson process:

– Number of outcomes occurring in one time interval isindependent of the number that occurs in any other time interval no memory

– Probability that a single outcome occurs in a short timeinterval is proportional to the length of the time interval and does not depend on the number of outcomes occurring outsides this time interval

– Probability that > 1 outcomes occur in such a short timeinterval is negligible

Page 21: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-21

Poisson Distribution• The probability distribution of the Poisson random variable X,

representing the number of outcomes occurring in a given time interval or specified region denoted by t, is

where λ is the average number of outcomes per unit

• The mean and variance of the Poisson distribution both have the value λt

( ) ( ) ,...2,1,0 ,!

; ==−

xx

tetxpxt λλ

λ

Proof….

rate !!

Page 22: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-22

Poisson Distribution

µµµ

µµµµµ

µ

µµµµ

==

−=

−===

∑∑∑∑∞

=

=

−−∞

=

−∞

=

−∞

=

0

1

1

110

!

!)1(!)1(!!)(

y

y

x

x

x

x

x

x

x

x

ye

xe

xe

xex

xexXE

µµσ

µµµ

µµµµµ

µ

µµµµ

=−=∴

==

−=

−=−=−=−

∑∑∑∑∞

=

=

−−∞

=

−∞

=

−∞

=

222

2

0

2

2

22

220

)(

!

!)2(!)2(!)1(

!)1()]1([

XE

ye

xe

xe

xexx

xexxXXE

y

y

x

x

x

x

x

x

x

x

Page 23: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-23

Theorem 5.6• Let X be a binomial random variable with

probability distribution b(x; n, p). When n→∞, p→0, and µ=np remains constant, we have

);(),;( µxppnxb →

Proof….

Page 24: 1036: Probability & Statisticstwins.ee.nctu.edu.tw/courses/prob06/lectures/Lecture 05-discrete... · 1036: Probability & Statistics Lecture 5 – Some Discrete Probability Distributions

Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])

5-24

Proof

);( !

1lim!

11lim!

),;(lim

have weconstant,remain and while, As

1!

)11()11(1

1!

)1()1(),;(

µ

µµµµµµ

µ

µµ

µµ

xpxe

nxnnxpnxb

xnnxn

xn

nnxxnnnqp

xn

pnxb

uxn

n

xxn

n

x

n

xnx

xnxxnx

=

=⎟⎠⎞

⎜⎝⎛ −=⎟

⎠⎞

⎜⎝⎛ −⎟

⎠⎞

⎜⎝⎛ −=

∞→

⎟⎠⎞

⎜⎝⎛ −

−−−=

⎟⎠⎞

⎜⎝⎛ −⎟

⎠⎞

⎜⎝⎛+−−

=⎟⎟⎠

⎞⎜⎜⎝

⎛=

∞→

∞→∞→

−−

L

L