확률6장-2-v2007 [호환 모드]

확률 변수 및 확률과정의 기초확률 변수 및 확률과정의 기초

6.5 Stationary Random Processes

A discrete-time or continuous-time random process X(t)is stationary ify

),,(),,( 1)(,),(1)(,),( 11 ktXtXktXtX xxFxxFkk

…… …… ττ ++=

for all time shifts τ, all k, and all choices of sample times t1 tktimes t1,…,tk.


Jointly stationary

1 1

( ) ( )( ), , ( ) ( ), , ( )k j

X t Y tX t X t Y t Y t′ ′… …

For two processes and ,the joint cdf's of and

k j

do not depend on the placement of the time originfor all and and all choices of sampling times

1, , k

jt t…

p g and 1, , jt t′ ′…


The first-order cdf of a stationary random process must be independent of time.p→ FX(t)(x) = FX(t+τ)(x) = FX (x) all t, τ.→ mX (t) = E[X(t)] = m for all t.

VAR[X(t)] = E[(X(t)－m)2] = σ2 for all t.

The second-order cdf of a stationary random process can depend only on the time difference between the p ysamples.

2121)()0(21)()( ,),(),(1221

ttxxFxxF ttXXtXtX allfor −= 2121)(),0(21)(),( ,),(),(1221 ttXXtXtX


Function of (t t ).,)(

),(

),(

)]()([),(

2112

)(),0(

)(),(

2121

12

21

ttttR

dyxxyf

dyxxyf

tXtXEttR

X

ttXX

tXtX

X

alfo −=

=

=

=

∫∫∫∫∞

∞−

∞

∞− −

∞

∞−

∞

∞−Function of (t2-t1)

)()()()}]()()}{()([{),( 221121 tmtXtmtXEttC XXX −−=

)()(

)()(),(2

12

2121

CmttR

tmtmttR

X

XXX

llf−−=

−=

.,)( 2112 ttttCX allfor −=


Ex. 6.27Is the sum process a discrete-time stationary process?p y p

sol) Sn = X1 + X2 +…+ Xn , where Xi are an iid sequence and n is time index.

cf) Sn : independent incrementstationary incrementstationary increment

mS(n) = nm, VAR[Sn] = nσ2

Note : Stationary process → Constant mean and variance.∴ Cannot be a stationary process.


Ex. 6.28Random process (telegraph signal) X(t) that assumes the valuesRandom process (telegraph signal) X(t) that assumes the values

±1.

X(0) = ±1 with probability of ½X(0) = ±1 with probability of ½.

X(t) changes polarity with each occurrence of an event in a

P i f tPoisson process of rate α.

Show that X(t) is a stationary random process.

Show that X(t) settles into a stationary behavior as t → ∞ even

if P[X(0) = ±1] ≠ ½.


sol) Need to show

])()([ == atXatXP

1])(,,)([

])(,,)([

11

11

±=<<=+=+=

==

kk

kk

attkatXatXP

atXatXP

anyandanyanyfor…

…ττ

h i d d i f h i

11 ±=<< jk attk any andany ,any for

The independent increments property of the Poisson process.

[ ])(])(,,)([ 1111 ==== kk atXPatXatXP …])()([])()([ 111122 −− ====× kkkk atXatXPatXatXP


cf) the sum process

][][ ===== ySPySySySP

][][][

][][

][],,,[

112

121

112

121

−−=−−=−×

=====

− kknnnn

nknnn

SPSPSP

yySSPyySSP

ySPySySySP

kk

k…

∵ The values of the random telegraph at the times t1,…, tk

][][][ 1121 1121 −−− −=−===− kknnnnn yySPyySPySP

kk

g p 1 k

is determined by the number of occurrences of the Poisson process in the time intervals (tj, tj+1).


Similarly

])(,,)([ 11 =+=+ kk atXatXP ττ …

])()([

])()([])([ ])(,,)([

112211

11

=+=+×

=+=+=+=kk

atXatXP

atXatXPatXP

ττ

τττ

Conditional probability (ex 6.22)

])()([ 11 −− =+=+× kkkk atXatXP ττ

p y ( )

{ }12 ( )1

1 12[ ( ) ( ) ]

j jt tj je a a

P X t X t

α +− −+

⎧ + =⎪⎪⎨

if

{ }1

1 12 ( )

1

2[ ( ) ( ) ]1 12

j j

j j j jt t

j j

P X t a X t ae a aα +

+ +− −

+

⎪= = = ⎨⎪ − ≠⎪⎩

if


cf) [ ] ==±=±=∞∞

∑∑22 )()(

])([1)0(1)(

αα jtt

j tt

tNPXtXP integer even

===

−−

=∑∑

2

00

11)!2(

)()!2(

)( αα αα

tttt

j

tt

j jtee

jt

+++

+=+=

−

−−−−

22

2

1111

)1(21)(

21 αααα

αα

tttt eeee

where

cf) Poisson process

−+−=++= 22

!21,

!21 αααα αα eewhere

cf) Poisson process

tk

ektktNP λλ −==!)(])([

k!])([


1 1[ ( ) ( ) ]

1j j j jP X t a X t aτ τ+ ++ = + =

⎧ { }{ }

1

1

2 ( )1

2 ( )

1 121 1

j j

j j

t tj j

t t

e a a

e a a

α τ τ

α τ τ

+

+

− + − −+

− + − −

⎧ + =⎪⎪= ⎨⎪ ≠

if

if{ } 112

j jj je a a +

⎪ − ≠⎪⎩

if

The joint probabilities differ only in the first term.→ P[X(t ) ] and P[X(t + ) ]→ P[X(t1) = a1] and P[X(t1+τ) = a1]


cf)]1)0([]1)0(1)([]1)([ ===== XPXtXPtXP

{ } { }1111]1)0([]1)0(1)([

22

−=−==+ XPXtXP

tt{ } { }1

121

211

21

21 22 −⋅++⋅= −− ee tt αα

1]1)([

2

=−=

=

tXP

1]1)0([1])([])([

2]1)([

1111 =±===+==∴

==

XPatXPatXP

tXP

withτ2

])([2

])([])([ 1111


])([])([21]1)0([ 1111 =+≠=⇒≠±= atXPatXPXPIf τ

,1]1)0([ ==XPIf

[ ] 1]1)0()([)( XXPXP

The process forgets the

[ ]

{ } if 2 111

1]1)0()([)(

ae

XatXPatXP

t

⎪⎪⎧ =+

⋅====

− αThe process forgets the initial conditionand settles down into steady state

{ }

{ } if 2 11212

ae t⎪⎪⎩

⎪⎪⎨

−=−=

− αsteady state→ stationary behavior.

{ }

large becomes as 111 21])([

2

tatXP →==

⎪⎩

2


Wide-Sense Stationary Random Processes

Cannot determine whether a random process is stationary

Can determine whethermX (t) = m for all t.CX (t1, t2) = CX (t1－t2) for all t1, t2

Function of t1- t2 only

Jointly Wide-Sense Stationary

X(t) is wide-sense stationary (WSS).

Jointly Wide Sense StationaryX(t) and Y(t) are both wide-sense stationary.Cross-Covariance depends only on t1－t2p y 1 2


NoteX(t) is Wide-Sense Stationary( ) y→ auto covariance CX (t1, t2) = CX (τ ) and

auto correlation RX (t1, t2) = RX (τ ) where τ = t1－t2

NoteAll stationary random processes are wide-sense stationary.All stationary random processes are wide sense stationary.Some wide-sense stationary processes are not stationary.


Ex. 6.29Xn : Consist of two interleaved sequences of independent r.v.’s.n q p

1, [ 1]2nn P X = ± =

⎡ ⎤

For even

1 9 1, , [ 3]3 10 10n nn P X P X⎡ ⎤= = = − =⎢ ⎥⎣ ⎦

For odd

Xn is not stationary since its pmf varies with n.

X nm = 0)(

for

for

i

jiX jiXE

jiXEXEjiC

⎪⎩

⎪⎨⎧

==

≠==

1][

0][][),(

2

.Stationary Sense-Wide : nX→


Properties of Autocorrelation Function of WSS Process

Average power of the process.RX (0) = E[X2(t)] for all t.X ( ) [ ( )]

Even function of τRX (τ ) = E[X(t +τ) X(t)] = E[X(t) X(t +τ)] = RX (-τ )

Measure of the rate of change of a random process.The change in the process from time t to t+τ :

[ ]2

22

)}()0({2]))()([(

]))()([()()(

ττ

ετετ

XX RRtXtXE

tXtXPtXtXP

−=

−+≤

>−+=>−+

22 εε=≤


cf) Markov inequalityXEaXP ][][ ≤≥

aaXP ][ ≤≥

Observation:If RX (0)－RX (τ ) is small, the probability of a large change in X(t)in seconds is smallin τ seconds is small.

cf) R (0)－R (τ ) is smallcf) RX (0) RX (τ ) is small → RX (τ ) drops off slowly.


RX (τ ) is maximum at τ = 0Proof) )

1E[XY]2 ≤ E[X2]E[Y2] for any two r.v.’s X and Y.- Can be proved using the approach used to prove |ρ|≤1. - HW

2 RX(τ)2 = E[X(t + τ)X(t)]2 ≤ E[X2 (t + τ)]E[X2(t)] = RX(0)2

Thus

)0()( XX RR ≤τ

cf) RX(0) is positive ∵ RX (0) = E[X2(t)]

)()( XX


If RX (0) = RX (d), then RX (τ ) is periodic with period d and X(t) is mean square periodic.( ) q p

pf) )]())()([( 2tXtXdtXE +−++ ττE[(X(t + d)－X(t))2] = 0

p )

)0()}()0({2)}()({)]([]))()([(

)]())()([(

2

22

RdRRRdRtXEtXdtXE

−≤−+→

+−++≤

ττ

ττ

)0()}()0({2)}()({ XXXXX RdRRRdR ≤+→ ττ

periodwithperiodicisallforzero is R.H.S.

dRdRRdRR XX

)()()()()0(

ττττ →+=∴→=∴

Mean square periodic:

. periodwithperiodicis allfor dRdRR XXX )()()( ττττ →+=∴

0)}()0({2]))()([( 2 dRRXdXE 0)}()0({2]))()([( 2 =−=−+ dRRtXdtXE XX


Let X(t) = m + N(t), where N(t) is a zero-mean process for which RN (τ ) → 0 as τ → ∞, N ( ) ,then

( ) [( ( ))( ( ))]R E m N t m N tτ τ= + + +2

2 2

( ) [( ( ))( ( ))]

2 [ ( )] ( )

( )

X

N

R E m N t m N t

m mE N t R

m R m

τ τ

τ

τ τ

= + + +

= + +

= + → → ∞as

Note

( ) Nm R mτ τ= + → → ∞as

NoteRX (τ ) approaches the square of the mean of X(t) as τ → ∞.


Summary: Three type of components

∞→→ ττ as0)(R①

+=

∞→→

ττ

ττ as

22

1

)()(

0)(

dRR

R

XX

X

②

①

∞→→ ττ as 23 )( mRX③


WSS Gaussian Random Processes

If a Gaussian random process is wide-sense stationary, then it is also stationary.

Proof) The joint pdf of a Gaussian random process is completelyThe joint pdf of a Gaussian random process is completely determined by the mean mX (t) and autocovariance CX (t1, t2).

X(t) is wide sense stationary → its mean is constant its autocovariance is only the function of the difference of the sampling times t－t → the joint pdf of X(t) depends only onsampling times ti tj → the joint pdf of X(t) depends only on this set of differences → invariant with respect to time shiftsThus the process is also stationary


Cyclostationary Random Processes

)(

),,,( 21)(,),(),( 21 ktXtXtX

xxxF

xxxFk

……

For all k, m and all choices of sampling times t1,…,tk

),,,( 21)(,),(),( 21 kmTtXmTtXmTtX xxxFk

…… +++=

Wide-Sense Cyclostationary.f h d i f i i i i h:If the mean and autocovariance functions are invariant with

respect to shifts in the time origin by integer multiples of T

)()( tTt),(),(

)()(

2121 ttCmTtmTtCtmmTtm

XX

XX

=++=+


NoteIf X(t) is cyclostationary, then X(t) is also wide-sense

l icyclostationary.

X(t) is a cyclostationary process with period TX(t) is a cyclostationary process with period T.→ X(t) is stationarized by observing a randomly phase-shifted version of X(t)shifted version of X(t)XS (t) = X(t + Θ), Θ uniform in [0, T],

where Θ is independent of X(t).where Θ is independent of X(t).→ If X(t) is a cyclostationary, XS (t) is a stationary random process.p


If X(t) is a wide-sense cyclostationary random process, then XS (t) is a wide-sense stationary random processS ( ) y p

∫=T

Xs dttmT

tXE0

)(1)]([

∫

∫

+=T

XX dtttRT

R

T

s 0

0

),(1)( ττ ∫Ts 0


6.6 Continuity, Derivatives and Integrals of Random Processes

The system having dynamics: described by linear differential eqs.Each sample function of a random process: deterministic signalInput to the system: Sample function of continuous-time random

process O t t f th t A l f ti f th dOutput of the system: A sample function of another random

process Probabilistic methods for addressing the continuityProbabilistic methods for addressing the continuity, differentiability and integrability of random processes

cf) A random process: the ensemble of sample functions ) p p


Mean Square Continuity

X(t, ζ) : A particular deterministic sample function for each point ζ in S of random processp ζ p

The continuity of the sample function at a point t0 forThe continuity of the sample function at a point t0 for each point ζ :If given any ε > 0 there exists a δ > 0 such that |t－t0| < δIf given any ε > 0 there exists a δ > 0 such that |t t0| < δimplies that |X(t, ζ)－X(t0, ζ)| < ε

)()(lim ζζ tXtX ),(),(lim 00

ζζ tXtXtt

=→


All sample functions of the random process are continuous at t0, then the random process is continuous0, pThe continuity of random process in a probabilistic sense is considered.Mean square continuity: The random process X(t) is continuous at the point t0 in

)()(l.i.m. 00

tXtXtt

=→

The random process X(t) is continuous at the point t0 in the mean square sense if

02

0 0]))()([( tttXtXE →→− as

Note: Mean square continuity does not imply that all the sample functions are continuous

00 0]))()([( tttXtXE →→ as

sample functions are continuous


Considering the mean square difference:),(),(),(),(]))()([( 0000

20 ttRttRttRttRtXtXE XXXX +−−=−

Therefore, if RX (t1 t2) is continuous in both t1 and t2 at

),(),(),(),(]))()([( 00000 ttRttRttRttRtXtXE XXXX +

Therefore, if RX (t1, t2) is continuous in both t1 and t2 at the point (t0, t0), then X(t) is mean square continuous at the point t0.p 0

If X(t) is mean square continuous at t0, then the mean (t) s ea squa e co t uous at t0, t e t e eafunction mX (t) must be continuous at t0.

)()(lim 0tmtm XX = )()( 00

XXtt→


Proof2

02

00 )]()([]))()([()]()([VAR0 tXtXEtXtXEtXtX −−−=−≤

If X(t) is mean square continuous L H S → 0 as t → t then

20

20

20 )]()([)]()([]))()([( tmtmtXtXEtXtXE XX −=−≥−∴

If X(t) is mean square continuous, L.H.S. → 0 as t → t0, then R.H.S. → 0, i.e., mX (t) → mX (t0)

Note: If X(t) is mean square continuous at t0, then we can interchange the order of the limit and the gexpectation

⎥⎦⎤

⎢⎣⎡= )(l.i.m.)]([lim tXEtXE ⎥⎦⎢⎣ →→

)()]([00 tttt


For the WSS random process X(t),))()0((2]))()([( 2 ττ RRtXtXE −=−+

: If RX (τ ) is continuous at τ = 0, then the WSS random

))()0((2]))()([( 00 ττ XX RRtXtXE =+

X ( ) ,process X(t) is mean square continuous at every point t0.


Mean Square Derivatives

The derivative of a deterministic functionζζε ),(),(li tXtX −+

: this limit may exist for some sample functions and it may fail to ε

ζζε

),(),(lim0→

exist for other sample functions

Mean Square DerivativetdXtXtX )()()( ζζ

dttdXtXtXtX )(),(),(l.i.m.)(

0≡

−+≡′

→ εζζε

ε

Provided that the mean square limit exists, that is,

0)(),(),(lim2

=⎥⎤

⎢⎡

⎟⎞

⎜⎛ ′−

−+ tXtXtXE ζζε 0)(lim0 ⎥

⎥⎦⎢

⎢⎣

⎟⎠

⎜⎝→

tXEεε


Note: The existence of the mean square derivative does not imply the existence of the derivative for all sample p y pfunctions.

The mean square derivative of X(t) at the point t exists if

)(2

ttR∂

exists at the point (t1, t2) = (t, t)

),( 2121

ttRtt X∂∂

e sts at t e po t (t1, t2) (t, t)

Proof) Use the Cauchy criterion) y


If the random process X(t) is WSS

)()(22

ttRttR ∂∂ )(),(

2

2121

2121

XX

d

ttRtt

ttRtt

∂⎞⎛∂

−∂∂

=∂∂

)()( 2211

τττ XX RttR

dd

t ∂∂

−=⎟⎠⎞

⎜⎝⎛ −−

∂∂

=

The mean square derivative of a WSS random process X(t) exists if RX(τ ) has derivatives up to order two at τ=0.

exists,if,processrandomGaussianaFor )()( tXtX ′

processrandomGaussianabemust then )(tX ′


Mean of X’(t))()(li)()(l i)]([ tXtXEtXtXEXE ⎤⎡ −+⎤⎡ −+′ εε

)()()(li

)()(lim)()(l.i.m.)]([00

tdtmtm

tXtXEtXtXEtXE

XX −+

⎥⎦⎤

⎢⎣⎡ +

=⎥⎦⎤

⎢⎣⎡ +

=′→→

εε

εε

εεε

)()()(lim0

tmdt X

XX ==→ εε

The cross-correlation between X(t) and X’(t))()(l i m)()( 22 tXtXtXEttR ⎥⎤

⎢⎡ −+

=ε

)(),(),(lim

l.i.m.)(),(

2121

0121,

ttRttRttR

tXEttR

XX

XX

∂=

−+=

⎥⎦⎢⎣=

→′

εεε

),(lim 212

0ttR

t X∂==

→ εε


The autocorrelation of X’(t)

)()( tXtX ⎤⎡ ⎫⎧ + ε

)()(

)()()(l.i.m.),( 211

021

ttRttR

tXtXtXEttRX

+

⎥⎦

⎤⎢⎣

⎡ ′⎭⎬⎫

⎩⎨⎧ −+

=→

′

εε

εε

)(),(lim 2,1,21,

0

ttRttR XXXX

∂

−+= ′′

→ εε

ε

),(

2

21,1

ttRt XX

∂

∂∂

= ′

),( 2121

ttRtt X∂∂

∂=


For the WSS random process X(t)

∂∂ )()()( 212

, ττ

τ XXXX RttRt

R

⎫⎧

∂∂

−=−∂∂

=′

)()()( 2

2

2121

ττ

τ XXX RttRtt

R∂∂

−=⎭⎬⎫

⎩⎨⎧

−∂∂

∂∂

=′


Mean Square Integrals

Imply the integral of a random process in the sense of mean square convergenceq g

The integral of the random process X(t)The integral of the random process X(t)The mean square limit of the sequence In as the width of the subintervals approaches zero:subintervals approaches zero:

∑=

Δ=n

kkkn tXI

1)(

∑∫ Δ=′′=→Δ k

kk

t

ttXtdtXtY

k

)(l.i.m.)()(00


Conditions that ensure the existence of the mean square integral

0, 0)()(2

→ΔΔ→⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎭⎬⎫

⎩⎨⎧

Δ−Δ∑ ∑ kjj k

kkjj tXtXE as

:The Cauchy criterion

⎥⎦⎢⎣ ⎭⎩ j k

Expanding the square inside the expected value

⎤⎡∑∑∑∑ ΔΔ=⎥

⎦

⎤⎢⎣

⎡ΔΔ

j kkjkjXkj

j kkj ttRtXtXE ),()()(


The limit of the right hand side approaches a double integral

∫ ∫∑∑ =ΔΔt t

kk dudvvuRttR )()(lim ∫ ∫∑∑ =ΔΔ→ΔΔ t t X

j kkjkjX dudvvuRttR

kj 0 0

),(),(lim0,

The mean square integral of X(t) exists if the double integral of the autocorrelation function exists

If X(t) is a mean square continuous random process, then its integral exists.then its integral exists.


The mean and autocorrelation function of Y(t)

[ ]∫∫ ′′⎤⎡ ′′tt

tdtXEtdtXEt )()()( [ ]

∫

∫∫′′=

′′=⎥⎦⎤

⎢⎣⎡ ′′=

t

X

ttY

tdtm

tdtXEtdtXEtm00

)(

)()()(

∫t X tdtm0

)(

∫∫ ⎥⎤

⎢⎡= 21 )()()(

ttdvvXduuXEttR

∫ ∫

∫∫=

⎥⎦⎢⎣=

1 2

00

),(

)()(),( 21

t

t

t

t X

ttY

dudvvuR

dvvXduuXEttR

∫ ∫0 0t t X


6.7 Time Averages of Random Processes and Ergodic Theorems

To estimate the mean mX (t) of a random process X(t, ζ),

∑N

X )(1)(ˆ ξ

where N is the number of repetitions of the experiment

∑=

=i

iX tXN

tm1

),()(ˆ ξ

where N is the number of repetitions of the experiment

I ti ti th t l ti f ti fIn estimating the mean or autocorrelation functions from the time average of a single realization

∫−=

T

TTdttX

TtX ),(

21)( ξ


Ergodic theorems : when time averages converge to the ensemble average (expected value)g ( p )cf) Strong law of large numbers :

: if Xn is an iid discrete-time random process with finite mean E[Xn] = m, then

11lim =⎥⎤

⎢⎡

=∑n

i mXP

i ld i l b

1lim1

⎥⎦

⎢⎣

∑=

∞→ iin

mXn

P

∫T

dttXtX )(1)( ξ yields a single number ⇒ consider process for which mX (t) = m

∫−=

TTdttX

TtX ),(

2)( ξ


An ergodic theorem for the time average of wide-sense stationary processesy p

X(t) : WSS process

⎤⎡ T1⎥⎦⎤

⎢⎣⎡=

∫

∫−

dttXE

dttXT

EtXE

T

T

TT

)]([1

)(21])([

== ∫−mdttXE

T T)]([

2

2

⎥⎦

⎤⎢⎣

⎡

⎭⎬⎫

⎩⎨⎧ ′−′

⎭⎬⎫

⎩⎨⎧ −=

−=

∫∫ tdmtXT

dtmtXT

E

mtXEtX

TT

TT

))((21))((

21

]))([(])(VAR[ 2

⎥⎦

⎢⎣ ⎭

⎬⎩⎨

⎭⎬

⎩⎨ ∫∫ −− TT TT

))((2

))((2


2

1 ( , )4

T T

XT TC t t dtdt

T − −′ ′= ∫ ∫

2

2

1 ( )4

1

T T

XT T

T

C t t dtdtT − −

′ ′= −∫ ∫

∫2

2 2

2

1 (2 ) ( )41 (1 ) ( )

T

XT

T

X

T u C u duT

uC u du

−= −

= −

∫

∫ 2(1 ) ( )

2 2 XTC u du

T T−∫

)( ⇒T

mtX sense, square mean the in approach will

0)(11lim

0]))([(

2

2

=⎟⎟⎞

⎜⎜⎛

→−

∫T

T

duuCu

mtXE ifonly and if i.e.,

0)(2

12

lim2

=⎟⎟⎠

⎜⎜⎝

−∫−∞→ T XTduuC

TT


Discrete-time random process

∑=T

XX 1

∑

∑−=

=

+=

T

kk

TnnTn

XXXX

XT

X

112

If Xn is WSS

∑−=

++ +=

TnnknTnkn XX

TXX

12

n

⎞⎛

=

T

Tn

k

mXE21

][

( VAR[<X > ] → 0 : mean square sense )

∑−=

⎟⎟⎠

⎞⎜⎜⎝

⎛+

−+

=T

TkXTn kC

Tk

TX

2

2)(

121

121]VAR[

( VAR[<Xn>T] → 0 : mean square sense )


Home work

Ch. 6 Problems3,5,7,10,15,18,21,24,29,34,3,5,7,10,15,18,21,24,29,34,37,40,44,48,52,55,59,63,67,69,71 74 79 83 87 8971,74,79,83,87,89

Documents

확률6장-2-v2007 [호환 모드]