확률및공학통계(Probability and Engineering Statistics)

이시웅

교재

• 주교재– 서명 : Probability, Random Variables and Rando

m Signal Principles– 저자 : P. Z. Peebles, 역자 : 강훈외 공역

• 보조교재– 서명 : Probability, Random Variables and Stochas

tic Processes, 4th Ed.– 저자 : A. Papoulis, S. U. Pillai

Introduction to Book

• Goal– Introduction to the principles of random signals

– Tools for dealing with systems involving such signals

• Random Signal– A time waveform that can be characterized only in

some probabilistic manner

– Desired or undesired waveform(noise)

1.1 Set Definition

• Set : a collection of objects - A• Objects: Elements of the set - a• If a is an element of set A :• If a is not an element of set A :• Methods for specifying a set

1. Tabular method2. Rule method

• Set– Countable, uncountable– Finite, infinite– Null set(=empty) : Ø : a subset of all other sets– Countably infinite

AaAa

• A is a subset of B : : If every element of a set A is also an element in another set B, A

is said to be contained in B

• A is a proper subset of B : : If at least one element exists in B which is not in A,

• Two sets, A and B, are called disjoint or mutually exclusive if they have no common elements

• A : Tabularly specified, countable• B : Tabularly specified, countable, and infinite• C : Rule-specified, uncountable, and infinite• D and E : Countably finite• F : Uncountably infinite• D is the null set?• A is contained in B, C, and F• • B and F are not subsets of any of the other sets or of each other• A, D, and E are mutually exclusive of each other

}5.85.0{

},3,2,1{

}7,5,3,1{

cC

B

A

}0.120.5{

}14,12,10,8,6,4,2{

}0.0{

fF

E

D

BEandFDFC ,

• Universal set : S– The largest set or all -encompassing set of objects under

discussion in a given situation

• Example 1.1-2– Rolling a die

• S = {1,2,3,4,5,6}• A person wins if the number comes up odd : A ={1,3,5}• Another person wins if the number shows four or less : B =

{1,2,3,4}• Both A and B are subsets of S

– For any universal set with N elements, there are 2N possible subsets of S

• Example : Token– S = {T, H}– {}, {T}, {H}, {T,H}

1.2 Set Operations

• Venn Diagram

– C is disjoint from both A and B– B is a subset of A

• Equality : A = B– Two sets are equal if all elements in A are present in B and

all elements in B are present in A; that is, if A B and B A.

S

AB

C

• Difference : A - B– The difference of two sets A and B is the set containing all

elements of A that are not present in B– Example: A = {0.6< a 1.6}, B = {1.0b2.5}

• A-B = {0.6 < c < 1.0}

• B-A = {1.6 < d 2.5}

• Union (Sum): C = AB– The union (call it C) of two sets A and B– The set of all elements of A or B or both

• Intersection (Product) : D = AB– The intersection (call it D) of two sets A or B– The set of all elements common to both A and B– For mutually exclusive sets A and B, AB = Ø

• The union and intersection of N sets An, n = 1,2,…,N :

,1

21 N

nnN AAAAC

N

nnN AAAAD

121

• Complement : – The complement of the set A is the set of all elements not in A– –

ASA AAandSAASS ,,,

• Example

– Applicable unions and intersections

– Complements

}8,7,6,4,3,1{

}11,10,9,8,7,6,2{

}12,5,3,1{

}12integers1{

C

B

A

S

}11,10,9,8,7,6,4,3,2,1{

}12,8,7,6,5,4,3,1{

}12,11,10,9,8,7,6,5,3,2,1{

CB

CA

BA

}8,7,6{

}3,1{

CB

CA

BA

}12,11,10,9,5,2{

}12,5,4,3,1{

}11,10,9,8,7,6,4,2{

C

B

A B2,9,10,11

6,7,8

5,12A

1,3

4C

S

• Algebra of Sets– Commutative law:

– Distributive law

– Associative law

ABBA

ABBA

)()()(

)()()(

CABACBA

CABACBA

CBACBACBA

CBACBACBA

)()(

)()(

• De Morgan’s Law– The complement of a union (intersection) of two sets A and

B equals the intersection (union) of the complements and

BABA

BABA

)(

)(

A B

• Example 1.2-2

}2416,52{

}165{

}225{},162{

}242{

ccBAC

cBABAC

bBaA

sS

}2416,52{

}2422,52{

},2416{

ccBAC

aaBSB

aASA

BABA )(

• Example 1.2-3

}43{

}10,8,6,2{

}6,4,2,1{

cC

B

A

}6,4,2{)()(

}6,4,2{)(

CABA

CBA

}4{

}6,2{

}10,8,6,43,2{

CA

BA

cCB

)()()( CABACBA

1.3 Probability Introduced Through Sets and Relative Frequency

• Definition of probability1. Set theory and fundamental axioms

2. Relative frequency

• Experiment : Rolling a single die– Six numbers : 1/6

• Sample space (S)– The set of all possible outcomes in any experiments

Universal set– Discrete, continuous– Finite, infinite

All possible outcomes

likelihood

• Mathematical model of experiments1. Sample space

2. Events

3. Probability

• Events– Example : Draw a card from a deck of 52 cards -> “draw a spade”

– Definition : A subset of the sample space

– Mutually exclusive : two events have no common outcomes

– Card experiment• Spades : 13 of the 52 possible outputs• events

– Discrete or continuous)10(5.422 1552 N

– Events defined on a countably infinite sample space do not have to be countably infinite

• Sample space: {1, 3, 5, 7, …} event: {1,3,5,7}

– Sample space: , event: A= {7.4<a<7.6}• Continuous sample space and continuous event

– Sample space: , event A = {6.1392}• Continuous sample space and discrete event

}136{ sS

}136{ sS

• Probability Definition and Axioms– Probability

• To each event defined on a sample space S, we shall assign a nonnegative number

• Probability is a function• It is a function of the events defined• P(A): the probability of event A• The assigned probabilities are chosen so as to satisfy three

axioms1. 2. S:certain event, Ø: impossible event3.

for all m n = 1, 2, …, N with N possibly infinite The probability of the event equal to the union of any number

of mutually exclusive events is equal to the sum of the individual event probabilities

0)( AP1)( SP

nm

N

nn

N

nn AAifAPAP

11

)(U

• Obtaining a number x by spinning the pointer on a “fair” wheel of chance that is labeled from 0 to 100 points– Sample space – The probability of the pointer falling between any two

numbers : – Consider events

• Axiom 1:

• Axiom 2:

• Axiom 3: Break the wheel’s periphery into N continuous segments, n=1,2,…,N with x0=0

, for any N,

}1000{ xS

12 xx 100/)( 12 xx

}{ 21 xxxA 0100 12 xandx

},{ 1 nnn xxxA NAP n /1)(

)(11

)(111

SPN

APAPN

n

N

nn

N

nn

UNnxn /100)(

– If the interval is allowed to approach to zero (->0), the probability

• Since in this situation,

• Thus, the probability of a discrete event defined on a continuous sample space is 0

• Events can occur even if their probability is 0

• Not the same as the impossible event

1 nn xx

)()( nn xPAP

N 0)( nAP

• Mathematical Model of Experiments– A real experiment is defined mathematically by

three things1.Assignment of a sample space

2.Definition of events of interest

3.Making probability assignments to the events such that the axioms are satisfied

• Observing the sum of the numbers showing up when two dice are thrown– Sample space : 62=36 points

– Each possible outcome: a sum having values from 2 to 12– Interested in three events defined by

}10{},118{},7{ sumCsumBsumA

– Assigning probabilities to these events• Define 36 elementary event, i = row, j = column

• • Aij: Mutually exclusive events-> axiom 3• The events A, B, and C are simply the unions of appropriate events

}),({ jijioutcomeforsumAij 36/1)( ijAP

12

1

36

13)(,

4

1

36

19)(

6

1

36

16)()(

6

17,

6

17,

CPBP

APAPAPi

iii

ii

18

5

36

110)(,

18

1

36

12)(

CBPCBP U

• Probability as a Relative Frequency– Flip a coin: heads shows up nH times out of the n flips

– Probability of the event “heads”:

– Relative frequency:

– Statistical regularity: relative frequencies approach a fixed value(a probability) as n becomes large

)(lim HPn

nHn

n

nH

• Example 1.3-3– 80 resistors in a box:10-18, 22-12, 27-33, 47-17, draw out on

e resistor, equally likely

– Suppose a 22 is drawn and not replaced. What are now the probabilities of drawing a resistor of any one of four values?

80/17)47(80/33)27(

80/12)22(80/18)10(

drawPdrawP

drawPdrawP

79/17)22|47(

79/33)22|27(

79/11)22|22(

79/18)22|10(

drawP

drawP

drawP

drawP