Functions of Random Variables

Method of Distribution Functions

• X1,…,Xn ~ f(x1,…,xn)

• U=g(X1,…,Xn) – Want to obtain fU(u)

• Find values in (x1,…,xn) space where U=u• Find region where U≤u • Obtain FU(u)=P(U≤u) by integrating f(x1,

…,xn) over the region where U≤u

• fU(u) = dFU(u)/du

Example – Uniform X• Stores located on a linear city with density

f(x)=0.05 -10 ≤ x ≤ 10, 0 otherwise• Courier incurs a cost of U=16X2 when she delivers to a

store located at X (her office is located at 0)

1600080

)()(

160004044

05.005.0)()(

44

416

2/1

4

4

2

uudu

udFuf

uuuudxuUPuF

uXuuU

uXuXuU

UU

u

uU

Example – Sum of Exponentials• X1, X2 independent Exponential()

• f(xi)=-1e-xi/xi>0, >0, i=1,2

• f(x1,x2)= -2e-(x1+x2)/ x1,x2>0

• U=X1+X2

),2(~01

11)(11

1111

11)(

,

/2

/2

////

2/)(

02/

02/)(/

0

20

//

021//

0 0 2

21221

2121

22222

21221

2

GammaUuue

eueeufuee

dxedxedxee

dxeedxdxeeuUP

XuXuXuXXuUxuXuXXuU

u

uuuU

uu

xuxuxuxuxu

xuxxuxxu xu

Method of Transformations• X~fX(x)• U=h(X) is either increasing or decreasing in X• fU(u) = fX(x)|dx/du| where x=h-1(u)

• Can be extended to functions of more than one random variable:• U1=h1(X1,X2), U2=h2(X1,X2), X1=h1

-1(U1,U2), X2=h2-1(U1,U2)

22112121

1

2

2

1

2

2

1

1

2

2

1

2

2

1

1

1

),()(||),(),(

||

1duuufufJxxfuuf

dUdX

dUdX

dUdX

dUdX

dUdX

dUdX

dUdX

dUdX

J

U

Example• fX(x) = 2x 0≤ x ≤ 1, 0 otherwise• U=10+500X (increasing in x)• x=(u-10)/500• fX(x) = 2x = 2(u-10)/500 = (u-10)/250• dx/du = d((u-10)/500)/du = 1/500• fU(u) = [(u-10)/250]|1/500| = (u-10)/125000

10 ≤ u ≤ 510, 0 otherwise

Method of Conditioning

• U=h(X1,X2)

• Find f(u|x2) by transformations (Fixing X2=x2)

• Obtain the joint density of U, X2: • f(u,x2) = f(u|x2)f(x2)

• Obtain the marginal distribution of U by integrating joint density over X2

222 )()|()( dxxfxufufU

Example (Problem 6.11)• X1~Beta( X2~Beta(Independent

• U=X1X2

• Fix X2=x2 and get f(u|x2)

10))ln(1(18

)ln(1818018)ln(18181818)()|()(

1011831)/1)(/(6)()|(),(

01)/1)(/(6)|(

/1/

103)(10)1(6)(

221

22

22

1

2

21

222

22

22

222222

22

222

21

2121

22221111

uuuuu

uuuuxuuxdxxuudxxfxufuf

xuxuux

xxuxuxfxufxuf

xux

xuxuxuf

xdUdXxUXxXU

xxxfxxxxf

uuuU

Problem 6.11

0

1

2

3

4

5

6

7

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

u

Den

sity

of U

=X1X

2

f(u)f(u|x2=.25)f(u|x2=.5)f(u|x2=.75)

Method of Moment-Generating Functions

• X,Y are two random variables• CDF’s: FX(x) and FY(y)

• MGF’s: MX(t) and MY(t) exist and equal for |t|<h,h>0

• Then the CDF’s FX(x) and FY(y) are equal• Three Properties:

– Y=aX+b MY(t)=E(etY)=E(et(aX+b))=ebtE(e(at)X)=ebtMX(at)

– X,Y independent MX+Y(t)=MX(t)MY(t)

– MX1,X2(t1,t2) = E[et1X1+t2X2] =MX1(t1)MX2(t2) if X1,X2 are indep.

Sum of Independent Gammas

,~

)1()1()1(

)()()(

,...,1)1()(

nt)(independe,...,1),(~

11

)...(

1

11

1

11

n

ii

n

ii

XXtXtXXXttY

Y

n

ii

X

ii

GammaXY

ttt

tMtMeeEeEeEtM

XY

nittM

niGammaX

n

i in

n

nn

i

i

Linear Function of Independent Normals

n

iii

n

iii

n

iii

n

i iin

i iinn

nn

nXXXtaXtaXaXattY

Y

i

n

iii

iiX

iii

aaNormalXaY

tatatatatata

taMtaMeeEeEeEtM

aXaY

nitttM

niNormalX

n

nnnn

i

1

22

11

21

22

1

2221

21

11

1)...(

1

22

2

,~

2exp

2exp

2exp

)()()(

constants fixed }{

,...,12

exp)(

nt)(independe,...,1),(~

1

1111

Distribution of Z2 (Z~N(0,1))

2

1

21

0

/1

21

2

2/12/1

2/1

0

212/

12/1

0

212/

2/1221

0

2/12/12

221

0

221

2/

2/

)2,2/(~t independenmutually ,...,

)2/1(

)(

:Notes

)2,2/1(~

)21()21(221

212)2/1(

21

21

21

212

5.05.02

1Let

0)about (symmetric 212

21

21)(

21)()1,0(~

2

2222

2

2

n

n

iin

y

tu

tutz

tztzztz

Z

zZ

nGammaZZZ

dyey

GammaZ

tt

tdueudueudze

duudzuudu

dzuzzu

dzedzedzeetM

zezfNZ

Distributions of and S2 (Normal data) X

data sampled theof sdifference theoffunction a is So

)1(2)1(2)0)(0(2)1()1(

)1(21

2)1(2

1

2)1(2

1

2)1(2

1

)1(21)(

)1(21

: oftion representa eAlternativ1

:Variance Sample

0 :Note

,...,111 :Mean Sample

dDistributetly Independen and Normal ),(~,...,

2

22

22

1 11

2

1

2

1 11

2

1

2

1 1

22

1 1

2

1 1

22

2

1

2

2

1111

11

1

21

S

Snn

SnnSnnSnnnn

XXXXXXnXXnnn

XXXXXXnXXnnn

XXXXXXXXnn

XXXXnn

XXnn

S

Sn

XXS

XXXnXXX

nin

aXaXnn

XX

NIDNIDXX

n

i

n

jji

n

jj

n

ii

n

i

n

jji

n

jj

n

ii

n

i

n

jjiji

n

i

n

jji

n

i

n

jji

n

i i

n

ii

n

ii

n

ii

n

ii

i

n

iii

n

ii

n

i i

n

Independence of and S2 (Normal Data)X

))(exp())(exp(

))(exp())(exp(

)]()(exp[

)()(exp)(),(

}exp{2

20exp)()2,0(~

}2exp{2

22exp)()2,2(~

),(~,

212211

212211

212211

12221121,

2222

212

2222

221

221

21

ttXEttXEttXttXE

ttXttXE

XXtXXtEeEttM

tttMNXXD

tttttMNXXT

NIDXX

ind

DtTtDT

D

T

Independence of T=X1+X2 and D=X2-X1 for Case of n=2

X

Independence of and S2 (Normal Data) P2X

)()(2

2exp2

22exp

22

222exp

2)22()(exp

2)()(exp

2)()(exp

))(exp())(exp())(exp())(exp(

21

22

221

2

1

22

221

2

1

2122

2121

22

21

2

2121

221

2

21

221

2

21

212211

212211

tMtMttt

ttt

tttttttttttt

tttttttt

ttXEttXEttXttXE

DT

ind

Independence of T=X1+X2 and D=X2-X1 for Case of n=2

Thus T=X1+X2 and D=X2-X1 are independent Normals and & S2 are independentX

Distribution of S2 (P.1)

2/1)()1()()1(

2

2

2

2

22

1

2

2

1

2

1

2

2

1

22

2

2

12

2

12

2

1

22

1

21

22

)21()()()(

:tindependen are and Now,

)1(01

21

21

11

)2,2/(~

~)1,0(~),(~

2

122

2

2

2

2

22n

XXnSnXnSn

n

ii

n

ii

n

ii

n

iii

n

ii

n

ii

n

i

i

n

n

i

i

ii

ii

tMtMtMtM

SX

XnSnXnXX

XXXXnXX

XXXXXX

XXXXX

nGammaX

ZNXZNIDX

n

ii

Distribution of S2 P.2

212

2

2/)1()2/1()2/(2/1

2/

)(

1

)1(

2/1)(

212

2

2

1

22

1

2

2

2

2/1)()1()()1(

2

2,2

1~)1(

)21()21()21()21()(

)21()(~

)1,0(~1,1~

: :consider Now,

)21()()()(

:tindependen are and Now,

2

2

2

12

2

2

2

2

2

122

2

2

2

2

22

n

nnn

Xn

X

Sn

Xn

X

XX

n

i

n

iXX

n

XXnSnXnSn

nGammaSn

tttt

M

M

tM

ttMXn

NXnn

XXZ

nnnNX

Xn

tMtMtMtM

SX

n

ii

n

ii

Summary of Results• X1,…Xn ≡ random sample from N(2)population• In practice, we observe the sample mean and sample variance (not

the population values: , 2)• We use the sample values (and their distributions) to make

inferences about the population values

ons)distributi-F and for t,on presentati.ppton ngconditioni of method using derivation (See

~)1()1()1(

//

tindependen are ,

~)1(1

,~

121

2

2

2

212

1

2

2

21

2

22

1

n

n

n

n

ii

n

ii

n

ii

tn

Z

nSn

nX

nSXt

SX

XXSn

n

XXS

nNX

n

XX

Order Statistics• X1,X2,...,Xn Independent Continuous RV’s

• F(x)=P(X≤x) Cumulative Distribution Function• f(x)=dF(x)/dx Probability Density Function

• Order Statistics: X(1) ≤ X(2) ≤ ...≤ X(n) (Continuous can ignore equalities)• X(1) = min(X1,...,Xn)

• X(n) = max(X1,...,Xn)

Order Statistics

)()](1[)](1[)](1[

])](1[1[)()(

:Minimum of pdf

)](1[1)(1),...,(1

: Minimum of CDF

)()]([)()]([)]([)()(

:Maximum of pdf

)]([)(),...,(

: Maximum of CDF

11

)1(1

11)1(

)1(

11)(

11)(

)(

xfxFndx

xFdxFn

dxxFd

dxxXdP

xg

xFxXPxXPxXxXPxXP

X

xfxFndxxdFxFn

dxxFd

dxxXdP

xg

xFxXPxXPxXxXPxXP

X

nn

n

nnn

nnn

nn

nnnn

n

Example • X1,...,X5 ~ iid U(0,1)

(iid=independent and identically distributed)

o.w.010)1(5)1)(1(5

)( :Minimum

o.w.0105)1(5

)( :Maximum

o.w.0101

)(11

1000

)(

44

1

44

xxxxg

xxxxg

xxf

xxx

xxF

n

Order Stats - U(0,1) - n=5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

x

pdf

f(x)

gn(x)

g1(x)

Distributions of Order Statistics• Consider case with n=4• X(1) ≤x can be one of the following cases:

• Exactly one less than x• Exactly two are less than x• Exactly three are less than x• All four are less than x

• X(3) ≤x can be one of the following cases:• Exactly three are less than x• All four are less than x

• Modeled as Binomial, n trials, p=F(x)

Case with n=4

))(1()()(12)()(12)()(12)(

)(3)(4

)()(4)(4

)](1[)]([44

)](1[)]([34

)](1[1

)](1[)]([44

)](1[)]([34

)](1[)]([24

)](1[)]([14

2323

43

443

043)3(

4

043

2231)1(

xFxFxfxfxFxfxFxg

xFxF

xFxFxF

xFxFxFxFxXP

xF

xFxFxFxF

xFxFxFxFxXP

General Case (Sample of size n)

elsewhere0...)()...(!

),...,(

:statisticsorder all ofon distributiJoint

)()()](1[)]()([)]([)!()!1()!1(

!

),(:1l)multinomia (uses statsorder and ofon distributiJoint

1)()](1[)]([)!()!1(

!)(

111,...,12

11

1

nnnn

jijn

jij

iji

i

jiij

thth

jnjj

xxxfxfnxxg

xfxfxFxFxFxFjniji

n

xxgnjiji

njxfxFxFjnj

nxg

Example – n=5 – Uniform(0,1)

10)(20),(:5,1

5)1(]1[][!0!4

!5)(:5

)1(20)1(]1[][!1!3!5)(:4

)1(30)1(]1[][!2!2

!5)(:3

)1(20)1(]1[][!3!1!5)(:2

)1(5)1(]1[][!4!0

!5)(:1

10)(1)(

513

155115

455155

345144

2235133

325122

415111

xxxxxxgji

xxxxgj

xxxxxgj

xxxxxgj

xxxxxgj

xxxxgj

xxxFxf

Distributions of all Order Stats - n=5 - U(0,1)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

x

pdf

f(x)

g1(x)

g2(x)

g3(x)

g4(x)

g5(x)