Lecture 15: Expectation for Multivariate Distributions

Probability Theory and Applications

Fall 2008

Those who ignore Statistics are condemned to reinvent it.Brad Efron

Outline

• Correlation

• Expectations of Functions of R.V.

• Covariance

• Covariance and Independence

• Algebra of Covariance

Correlation Intuition

• Covariance is a measure of how much RV vary together.

Wife’s Age and Husband’s Age

Correlation .97

Example from http://cnx.org/content/m10950/latest/

Sometimes not so perfect

Arm Strength Versus Grip Strength

Pearson’s Correlation R=.63

Negative Correlation

Child Labor versus GDP

Extreme Correlation 1

Linear relation with positive slope

Extreme: Correlation -1

Linear relation with negative slope

Zero Correlation

Independent Random

Guess Covariance???Positive, Negative, 0

• Crime Rate, Housing Price

• SAT Scores, GPA Freshman Year

• Weight and SAT Score

• Average Daily Temperature, Housing Price

• GDP, Infant Mortality

• Life Expectancy, Infant Mortality

Expectations of Functions of R.V.

( ( , )) ( , ) ( , )

( ) ( ) ( , )

( , ) ( , )

( ) ( ) ( ) ( )x y

E g x y g x y f x y dydx

E X Y x y f x y dxdy

x f x y dy dx y f x y dx dy

xf x dx yf y dy E X E Y

NOTE substitute appropriate summation for discrete

Variance and Covariance

Univariate becomes variance

Multivariate becomes covariance

Note:

2 2var( ) ( ) ( ) ( )X X Xx

X E X x f x

,

cov( , ) ( )( )

( )( ) ( , )

X Y

X Yx y

X Y E X y

x y f x y

NOTE substitute appropriate integral for continuous

var( ) cov( , )X X X

Calculating Covariance

Can simplify

,

, ,

cov( , ) ( )( )

( , )

( , ) ( , )

( )

( ) ( ) ( )

X Y

X Y X Yx y

X Y X Yx y x y

X Y X Y X Y

X Y E X Y

xy y x f x y

xy f x y y x f x y

E XY

E XY E X E Y

Correlation of X and Y

Definition

The correlation always falls in [ -1, 1]

It a measure of the linear relation between X and Y

cov( , )( , )

X Y

X YX Y

Extreme Cases

If X=Y then ρ=1.

If X=-Y then ρ=-1.

If X and Y independent, then ρ=0.

If X=-2Y then ρ=?.

Example

Joint is

Find correlation of X and Y

2 0, 0, 1( , )

0 . .

x y x yf x y

o w

1

0

1

0

12 2

0

( ) 2 2(1 ) 0 1

( ) 2 (1 ) 1/ 3

( ) 2 (1 ) 1/ 6

var( ) 1/ 6 1/ 9 1/18

y

Yf y dx y y

E Y y y dy

E Y y y dy

Y

1

0

1

0

12 2

0

( ) 2 2(1 ) 0 1

( ) 2 (1 ) 1/ 3

( ) 2 (1 ) 1/ 6

var( ) 1/ 6 1/ 9 1/18

x

Xf x dy x x

E X x x dx

E X x x dx

X

Example

Joint is

Find correlation of X and Y

2 0, 0, 1( , )

0 . .

x y x yf x y

o w

11 12

0 0 0

( , ) 2 (1 ) 1/12

cov( , ) 1/12 1/ 9 1/ 36

cov( , ) 1/ 36( , ) 1/ 2

1/18 1/18

y

X Y

E X Y xydxdy y y dy

X Y

X YX Y

Properties of Covariance

a)

b) cov(aX+bY)=ab[cov(X,Y)]

var( ) cov( , )X X X

( ) ( )

( ) ( ) ( ) ( )

cov( ) ( ) ( ) ( )

[ ( , ) ( ) ( )] cov( , )

E aXbY abE XY

E aX aE X E bY bE Y

aX bY E aXbY E aX E bY

ab E X Y E X E Y ab X Y

Properties of Covariance

c)

d)

2var( ) var( )aX a X

cov( , )( ) ( , )

| | | |

( , ) ( * ) ( , )| || |

X

ab X YaX bY X Y

a b Y

abX Y sign a b X Y

a b

( ) ( * ) ( , )aX bY sign a b X Y

Properties of Covariance

e) If X and Y are independent

Proof:( , ) ( ) ( )

( ) ( ) ( ) ( )

x y

x Y

E X Y xyf x f y dxdy

xf x dx yf y dy E X E Y

cov( , ) 0 ( , ) 0X Y correl X Y

cov( , ) ( ) ( ) ( ) 0X Y E XY E X E Y

Find Covariance

X\Y 0 1

-1 0 .3

0 .4 0

1 0 .3

Are X and Y independent

X\Y 0 1

-1 0 .3

0 .4 0

1 0 .3

Note

Cov(X,Y)=0 does not imply independence of X and Y

Independence of X and Y implies cov(X,Y)=0

In this case Y=X2 so the variables are definitely not independent but their covariance is 0 because they have no linear relation.

Algebra of variance/covariance/correlation

Given:

Calculate mean of Z=2X-3Y+5

variance of Z=2X-3Y+5

2 2

( ) ( )

var( ) var( )

cov( , )

Y Y

X Y

XY

E X E Y

X Y

X Y

Long steps

2 2 2

2

( ) 2 3 5

( ) 2( ) 3( )

[ ( )] 4( ) 3( ) 12( )( )

[ ( )] 4 var( ) 8var( ) 12cov( , )

X Y

X Y

X Y X Y

E X

Z E Z X Y

Z E Z X Y X Y

E Z E Z X Y X Y

Working Rules for linear combinations

Write formula

Discard Constants

Square it

Replace squared R.V

with var and crossterms

with cov

2 2

2 3 5

2 3

4 9 12

4var( ) 9 var( ) 12cov( , )

Z X Y

X Y

X Y XY

X Y X Y

Example

Given var(X)=4 var(Y)=10

ρ(X,Y)=1/2

Find variance of X-5Y+6?

Given same facts as previous problem

Find covariance x-5y+6 and -4X+3Y+2

Working rule works also for more than two variables

Find variance of W=2x-3Y+5Z+1