Correlation & Regressiongtk.uni-miskolc.hu/files/975/L5_corr_reg.pdf · Correlation Regression describes the strength of a relationship, the degree to which one variable is linearly

Miskolci Egyetem Gazdaságtudományi KarMiskolci Egyetem Gazdaságtudományi KarMiskolci Egyetem Gazdaságtudományi KarMiskolci Egyetem Gazdaságtudományi Kar

Üzleti Információgazdálkodási és Módszertani Intézet

Correlation & Regression



Types of dependence

• association – between nominal data

• mixed – between a nominal and a ratio data

• correlation – among ratio data



Correlation Regression

describes the strength of a

relationship, the degree to

which one variable is

linearly related to another

shows us how to

determine the nature of a

relationship between two

or more variables

• X (or X1, X2, … , Xp):

known variable(s) / independent variable(s) / predictor(s)

• Y: unknown variable / dependent variable

• causal relationship: X „causes” Y to change



Correlation Measures

1. Covariance

2. Coefficient of correlation2. Coefficient of correlation

3. Coefficient of determination

4. Coefficient of rank correlation



1. Covariance

• A measure of the joint variation of the two variables;

• An average value of the product of the deviations ofobservations on 2 random variables from their samplemeans.

– ranges from - ∞ to +∞;

– C = 0, when X and Y are uncorrelated;

– its sign shows the direction of correlation;– it doesn’t measure the degree of relationship!!!

( ) ( ) ( )1

yx,C−

−⋅−= ∑

n

yyxx



• Pearson correlation

• A measure of how closely related two data series are.

2. Coefficient of correlation

2y

2x

yx

yx dd

dΣd =

ss

Cr =

• A measure of how closely related two data series are.

• Its sign shows the direction of correlation;• It measures the strength of correlation;• 0 < r < 1 → statistical dependence

r = 0 → X and Y are uncorrelated; ∅r = -1 → negative ☻

r = 1 → positive ☺

• You can use only in case of linear relationship!



3. Coefficient of determination

• r2

• The square of the sample correlation coefficient between

the outcomes and their predicted values.

• Measures the degree of correlation in percentage (%)

• It provides a measure of how well future outcomes are

likely to be predicted by the model.

• Vary from 0 to 1.

y

e

y

y2

S

S - 1 =

S

S r

ˆ=



Example

• A firm administers a test tosales trainees before they gointo the field. Themanagement of the firm isinterested in determining theinterested in determining therelationship between the testscores and the sales made bythe trainees at the end of oneyear in the field. Thefollowing data were collectedfor 45 sales personnel whohave been in the field oneyear.

• Calculate differentcorrelation measures!



Sales-

person

Test

score

Number of

units sold

K. A. 25 188 +9 +22 +198

L. Z. 16 157 0 -9 0

X → Y

independent dependent variable

xi dxx =− yi dyy =− ( ) ( ) yxii ddyyxx =−⋅−

B. E. 30 165 +14 -1 -14

G. P. 5 124 -11 -42 +462

… … … … … …

… … … … … …

S. G. 10 158 -6 -8 +48

J. T. 24 224 +8 +58 +464

V. P. 17 169 +1 +3 +3

T. L. 6 114 -10 -52 +520

Total 716 7 464 0 0 ∑dxdy=8 894.5



Number of observed pairs: n = 45

8.26 s 16 x x ========

30.99 s 166 y y ========

Positive correlation

202.15 1-45

894.5 8

1n

dd C

yx ==−

= ∑



% 62.36 r

0.7897 30.99 8.26

202.15

ss

C

2

yx

=⇒

=⋅

=⋅

=r

There is a strong & positive relationbetween test scores and number of unitssold.

The variation of test scores explains 62.36percent of the variation of number of unitssold.



4. Coefficient of rank correlation

• Spearman correlation

)1 (nn

d6 - 1 ρ

2

2

i

−⋅

⋅= ∑ 1 0 ≤≤ ρ

• Spearman correlation

• Measure of the relationship between two ordinal data;• n = number of paired observations,

d = difference between the ranks for each pair of

observations.

• perfect correlation → ρ= 1 ;perfect inverse correlation → ρ = -1 ;in case of independence → ρ = 0 ;



Student A B C D E F G H I J Total

Example

Ten students were ranked by their

mathematical and musical ability:

Student

Ability

Mathematics 1 2 3 4 5 6 7 8 9 10 -

Music 3 4 1 2 5 7 10 6 8 9 -

di = xi - yi -2 -2 2 2 0 -1 -3 2 1 1 0

di2 4 4 4 4 0 1 9 4 1 1 32

0.806 1) - (1010

326 - 1

)1 (nn

d6 - 1 ρ

22

2

i =⋅⋅

=−⋅

⋅= ∑

strong relationship



Simple Linear Regression Model

• We model the relationship between two variables, X and Y

as a straight line.

• The model contains two parameters:

� an intercept parameter,� an intercept parameter,

� a slope parameter.

y = β0 + β1x + ε

where: y – dependent or response variable (the variablewe wish to explain or predict)

x – independent or predictor variable

ε – random error component

β0 – y-intercept of the line, i.e. point at which theline intercept the y-axis

β1 – slope of the line

E (y)

x

β0 = y-intercept

β1 = slope



Assumptions of the Linear Regression Model

• Assumptions for Error term:

– Normally distributed;

– Expected value = 0 (E(ε)=0);– Expected value = 0 (E(ε)=0);

– The variance is the same for all observations

(Homoscadasticity);

– Uncorrelated across observations (there isn’t any

autocorrelation).

• Assumptions for the Independent Variables:

– Not random, etc.



y

Deterministic component• y = deterministic component +

random error

• We always assume that the mean

value of the random error equals 0

→ the mean value of y equals the

deterministic component.ŷi = b0 + bixi

x

Random

error

deterministic component.

• It is possible to find many lines

for which the sum of the errors is

equal to 0, but there is one (and

only one) line for which the SSE

(sum of squares of the errors) is a

minimum:

→ Least squares line / regressionline.



• The method of least squares gives us the bestlinear unbiased estimators of the regressionparameters: β0, β1.

• The least-squares estimators:

b0 estimates β0

b1 estimates β1b1 estimates β1

• The regression line:

y caret („hat”): Ŷ = b0 + b1X

• The normal equations (with 1 x)

Σy = nb0 + b1Σx

Σxy = b0Σx + b1Σx2



Interpretation

• b0: when x=0, y=b0

• b1: for every 1 unit increase in x we expect

y to change by b1 units



Scatter diagrams

0

400

800

1200

1600

0 10 20 30 40

Sales in $ 0

1000

2000

3000

4000

5000

0 2 4 6 8 10 12

Selling

price

linear

direct relationship

positive slope

0

10

20

30

40

50

0 10 20 30 40

Production (number of products per day)

wastage

0 10 20 30 40

Advertising in $

0 2 4 6 8 10 12Age of a house (year)

0

1000

2000

3000

4000

0 5 10 15Age of a car (year)

Selling

price

curvilinear

inverse relationship

negative slope



No relationship

3000

4000

0

1000

2000

0 10 20 30 40Number of storks

Number of births



Elasticity

% change in x demanded % change in y

xbb

x b x)E(y,

10

1 += E(y, x) = b

x

y1

Elasticity at the mean



Estimation in Regression

• Regression estimation is a technique used to replace

missing values in data.

• If we know:• If we know:

1. The estimated parameter value;

2. The hypothesized value of the parameter;

3. Confidence interval around the estimated parameter.

• The number of degrees of freedom equals the number of

observations minus the number of parameters estimated.

• νννν = n-2



Parameter Estimated value Standard error

β0 b0

β1 b1

Estimation in Regression

2i

2i

)x(xn

x

−Σ

Σes

2i )xx −Σ (

es

µ0

Y0 0y

2i

20

)xx

)xx

n −Σ

−+

(

(1es

0y

2i

20

)x(x

)xx +

n

1

−Σ

−+

(1es

y

y

b

b

sty

sty

stb

stb

ˆ

ˆ

1

0

ˆ

ˆ

1

0

′′⋅±

′⋅±

⋅±

⋅±

π

π

π

π

νννν = n-2

In case of average Y values

In case of discrete Y values



Sum of Squares

DfMean Sum of Squares

F

Regression 1

Analysis of Variance in Regression Analysis

2)yy( = S −Σ ySS

=FyRegression 1

Residual n-2

Total n-1

2e

2y

2y SS S += ˆ

2

i

n

1=i

2n

1=i

i

n

1=i

2

i )y(y + )yy( )y(y −−=− ∑∑∑

2

iy )yy( = S −Σ yS

2

ie )y(y = S −Σ )2/( −= nS s e2e

S = (y y)y i

2Σ −1-n

S y

2)-/(nS =F

e

y



Model testing

H0: β1 = 0

H1: β1 ≠ 0 (linear model)

Test statistic:2)-/(nS

S

s

S =F

y

2

y =

211

H0Test statistic:

• F-statistic tests whether all the slope coefficients

in a linear regression are equal to 0.

• Measures how well the regression equation

explains the variation in the dependent variable.

2)-/(nS

s =F

e

2

e

=0 );( 21

21

ννα−F

);(

1

12

21

ννα−

F

F



Parameter testing

H0: β1 = 0

H1: β1 ≠ 0

b−t 0 t

01

H0

Test statistic:

where: b1 is the least square estimate of the

regression slope

s(b1) is the standard error of b1

)( 1

1

bs

bt =

2/1 α−−t 0 2/1 α−t



Seminar



Exercise 1Book: p185 e44

In a bar waiters believe that there is a

relationship between the amount of

consumption of cola and the average dailyconsumption of cola and the average daily

temperature. To test it a sample of 20 days

was drawn and they examined the amount of

consumption and the temperature in these

days:



• Results:

2

2 2

2

12,129; 537;

330,159; 14,597;

7,505,555; 179

149,923; 4495

x

y x

xy x

y d

d d d

∑ = ∑ =

∑ = ∑ =

∑ = ∑ =

∑ = ∑ =

Day The amount of consumption (l)

The maximum daily temperature (°C)

1. 520 25

2. 534 26

3. 610 28

4. 780 32

5. 708 27

6. 639 25

7. 486 23

8. 423 20

• Determine the

relationship between the

temperature and the

consumption in case of

linear and curvilinear

relationship.

2 149,923; 4495y x yd d d∑ = ∑ = 9. 452 22

10. 597 29

11. 640 30

12. 657 31

13. 678 30

14. 620 27

15. 635 28

16. 610 26

17. 585 25

18. 627 27

19. 608 26

20. 720 30



Exercise 2 (p188 e48)

• The export and import

of Hungary with

European countries are

the following:

Country Export (X) Import (Y)

Austria 406 418

Belgium 87 93

Czech Republic 60 95

France 134 172

Holland 100 102the following:

• Characterize the trade

with European

countries.

2

2

1,195,957; 2,948

3,071; 2,084,046;

1,628,345

x yd d x

y x

y

∑ = ∑ =

∑ = ∑ =

∑ =

Holland 100 102

Poland 95 67

Great-Britain 119 136

Germany 219 291

Italy 181 363

Russia 41 68

Switzerland 27 49

Sweden 49 75

Slovakia 54 21

Slovenia 47 53

Ukraine 1329 1068



Exercise 3p188 e48

• The table shows the inflationrate (x) and theunemployment rate (y) ofGermany between 1972 and

Year Inflation

rate (%)

Unemployment

rate (%)

1972 5.5 1.1Germany between 1972 and1997.

• Results:

• Determine the relationshipbetween unemployment andinflation rate.

2 2

92.4; 171.8

94.54; 195.44

512.9

x y

x y

d d

xy

∑ = ∑ =

∑ = ∑ =

∑ =

1972 5.5 1.1

1973 6.9 1.2

1974 7.0 2.6

. . .

. . .

. . .

1996 1.5 11.5

1997 1.8 9.8



Thanks for your attention!Thanks for your attention!

Documents

Correlation & Regressiongtk.uni-miskolc.hu/files/975/L5_corr_reg.pdf · Correlation Regression describes the strength of a relationship, the degree to which one variable is linearly