ANOVAfor comparing means between more than 2

groups

ANOVA

It’s like this: If I have three groups to compare: I could do three pair-wise t-tests, but

this would increase my type I error So, instead I want to look at the pairwise

differences “all at once.” To do this, I can recognize that variance

is a statistic that let’s me look at more than one difference at a time…

The “F-test”

groupswithinyVariabilit

groupsbetweenyVariabilitF

Is the difference in the means of the groups more than background noise (=variability within groups)?

Recall, we have already used an “F-test” to check for equality of variances If F>>1 (indicating unequal variances), use unpooled variance in a t-test.

Summarizes the mean differences between all groups at once.

Analogous to pooled variance from a ttest.

The F-distribution

The F-distribution is a continuous probability distribution that depends on two parameters n and m (numerator and denominator degrees of freedom, respectively):

The F-distribution A ratio of variances follows an F-

distribution:

22

220

:

:

withinbetweena

withinbetween

H

H

The F-test tests the hypothesis that two variances are equal.

F will be close to 1 if sample variances are equal.

mnwithin

between F ,2

2

~

Today…

Introduce logic of ANOVA Review calculations Work through one example

While only comparing two means, ANOVA will give the same results as the normal t-test.

However, it allows comparing multiple means and thus multiple groups (factor levels) as well as multiple factors simultaneously.

Illustrate Logic of ANOVA

Group1 Group2 Group3 Group4

27 36 17 34

31 35 21 36

25 29 22 30

27 33 22 32

24 38 21 32

27 36 15 32

M = 26.8 M = 34.5 M = 19.7 M = 32.7

We want to evaluate the effects of 4 different drugs on participants level of depression as measured by

the Beck Depression Inventory.

An ANOVA allows us to quantify how far apart the sample means must be before we are no longer

willing to say they are all “approximately” equal.

Introduction to ANOVA ANOVA – the ANalysis Of Variance

(1) Inferential hypothesis-testing procedure

(2) Tremendous advantage over t-tests: used to compare MULTIPLE (two or more)

treatments (3) Provides researchers with much

greater flexibility in design and analysis of experiments.

Introduction to ANOVA (cont.)

ANOVA – the ANalysis Of Variance (4) Multiple Forms – We’ll look at the

simplest: Single-factor, independent measures ANOVA (a) experimental unit :object on which

measurements take place. (a) factor: new name for the independent

variable (b) independent measures: separate sample

for each treatment (c) level: the intensity settings of a factor (d) treatment : specific combination of factor

levels.Eg: Tyre quality study, Weekly production volume.

Research Design for ANOVA

FYI: Factors and Levels Can be multiple factors (IV’s) and levels

(variations) Expressed as factors x levels

How many factors? How many levels?

Therapist Experience

experienced (+) inexperienced (-)

Treatment treatment A tx A + exp tx A + inexp

treatment B tx B + exp tx B + inexp

Example of ANOVA Four different test times (8am, 12pm, 4pm,

and 8pm)

Does time of test affect scores? ANOVA uses variance to assess differences

among the sample means

Tx1 Tx2 Tx3 Tx4

25 30 27 22

28 29 20 27

22 30 21 24

M = 25 M = 29.7 M = 22.6 M = 24.3

Variability Components for ANOVA

The Logic of ANOVA (1) First, determine total variability for

data set

(2) Next, break this variability into two components: (a) Between-Treatments variance – two

sources: Treatment Effect: Differences are caused by

treatments. Chance: Differences simply due to chance.

(b) Within-Treatments variance – one source: Chance: Differences simply due to chance.

Tx1 Tx2 Tx3 Tx4

25 30 27 22

28 29 20 27

22 30 21 24

M = 25 M = 29.7 M = 22.6 M = 24.3

Forming an F-Ratio (3) Finally, determine the variance due to the

treatments alone by forming an F-Ratio.F = Variance Between-Treatments Variance Within-Treatments

Or in terms of sources…

F = Treatment Effect + Differences due to Chance

Differences due to Chance

If no treatment effect exists, F = 1.00 If there IS some treatment effect, F > 1.00

( but not automatically statistically significant)

The Structure of ANOVA Calculations

New Terms and Symbols

k = number of treatment conditions (levels and factors). For independent-measures study, k = # of separate samples.

n = number of scores in a treatment condition N = total number of scores in whole study (N =

nk) T = sum of scores for each treatment condition G = sum of all scores in the study (Grand Total)

Hypothesis Testing with ANOVA (4 steps)STEP 1

STEP 1: State the Hypothesis H0: m1 = m2 = m3 = mk (k = number of factor

levels) H1 : At least one m is different from the

othersTx1 Tx2 Tx3 Tx4

25 30 27 22

28 29 20 27

22 30 21 24

M = 25 M = 29.7 M = 22.6 M = 24.3

Hypothesis Testing with ANOVASTEP 2

STEP 2: Locate the Critical region• a = .05• Calculate dfbetween = k – 1• Calculate dfwithin = N-k

• Calculate dftotal = N-1 • Critical F will be provided for

you • dfbetween + dfwithin = dftotal (always!)• Begin to fill in the Source Table

(ANOVA Table)

k = number of factor levels

n = number of scores in a treatment condition

N = total number of scores in whole study (N = nk)

T = sum of scores for each treatment condition

G = sum of all scores in the study (Grand Total)

Tx1 Tx2 Tx3 Tx4

25 30 27 22

28 29 20 27

22 30 21 24

M = 25 M = 29.7 M = 22.6 M = 24.3

Hypothesis Testing with ANOVASTEP 2 continued…

Basic ANOVA TableSource SS df MS F Between SSbetween k-1 MSbetween F = Fobtained

Within SSwithin N-k MSwithin

Total SStotal N-1

Hypothesis Testing with ANOVASTEP 3

STEP 3: Collect Data and Compute Sample Statistics SSbetween = (T2/n) – (G2/N)

SSwithin = SS inside each treatment =(SS1+SS2+SS3+...

+SSk)

SStotal = X2 – (G2/N) or SSbetween + SSwithin

MSbetween = SSbetween/dfbetween

MSwithin = SSwithin/dfwithin

F = MSbetween/MSwithin Fill in source table (ANOVA Table) *note: SSbetween + SSwithin = SStotal (always!) n = # of scores in a tx conditionN = total # of scores in whole study T = sum of scores for each tx conditionG = sum of all scores in the study (Grand Total)

Hypothesis Testing with ANOVASTEP 4

STEP 4: Make a Decision Given the Critical F-value (Fcritical ) - which

will be provided - decide whether or not to reject the null.

Fobtained < Fcritical --> Fail to reject Ho.

Fobtained > Fcritical --> Reject Ho. Use Appendix B page A-29 to find Fcritical

Bold-Faced = Fcritical for a = 0.01

Light-Faced Fcritical for a = 0.05

df-numerator = df-betweendf-denominator = df-within

Error Term

Error due to chance Does the treatment effect (difference

among means) produce greater variability between groups than that expected by chance?

The denominator in the F ratio

Example 1

A researcher is interested in whether class time affects exam scores.

There are four different class times being examined: 8am, 12pm, 4pm, and 8pm.

Run an ANOVA, = .05, to see if a significant difference exists between these treatments.

Example 1 DATATx. 1 Tx. 2 Tx. 3 Tx. 425 30 27 2228 29 20 2722 30 21 24mean1=25 mean2=29.67 mean3=22.67

mean4=24.33

T1=75 T2=89 T3=68 T4=73

SS1=18 SS2=0.67 SS3=28.67 SS4=12.67

n1=3 n2=3 n3=3 n4=3

X2 = 7893G = 305N = 12 k

= 4

Example 1 Calculations

SSbetween = S(T2/n) – (G2/N)

SSbetween = ((752/3)+(892/3)+(682/3)+(732/3))- (93,025/12)

SSbetween=((5625/3)+(7921/3)+(4624/3)+(5329/3))-7752.083

SSbetween = (1875+2640.33+1541.33+1776.33)-7752.083

SSbetween = 7832.99 – 7752.083

SSbetween = 80.91

SSwithin = SS1+SS2+SS3+SS4

SSwithin = 18+.67+28.67+12.67

SSwithin = 60.01

SStotal = X2 – (G2/N) OR SSbetween + SSwithin

SStotal = 7893-7752.083 OR 80.91+ 60.01

SStotal = 140.92

Short Cut Method Calculate grand Total

G = Σxij

Calculate Correction Factor CF = G2/N = (305)2 /12

Find sum of squares and subtract CF to find SST

SST = (Σx12 + Σx2

2 + Σx32 …+ Σxk

2 ) – CF

SSTr = (Σxj) 2 – CF nj

SSE = SST - SSTr

Example 1 ANOVA and Decision

Source SS df MS Fobtained

Between 80.91 3 26.973.596

Within 60.01 8 7.50TOTAL 140.92 11Fcritical = 4.07

Fobtained < Fcritical fail to reject H0

3.596 < 4.07 Fail to reject H0

Use Appendix B Page A-29 for Fcritical

df numerator = 3 (df for between) df denominator = 8 (df for within)

Example 2

A researcher is interested in whether a new drug affects activity level of lab animals. There are three different doses being examined: low, medium, large. Run an ANOVA, = .05, to see if a significant difference exists between these doses.

Null Hypothesis:

Alternative:

Example 2 DATADose 1 (lo) Dose 2 (med) Dose 3 (hi)0 1 51 3 83 4 60 1 41 1 7

mean1= mean2= mean3=

T1= T2= T3=

SS1= SS2= SS3=

n1=5 n2=5 n3=5

X2 = G = N = k =

Example 2 CalculationsSSbetween = S(T2/n) – (G2/N)

SSbetween =

SSbetween=

SSbetween =

SSbetween =

SSbetween =

SSwithin = SS1+SS2+SS3+SS4

SSwithin =

SSwithin =

SStotal = X2 – (G2/N) OR SSbetween + SSwithin

SStotal =

SStotal =

Example 2 ANOVA and DecisionSource SS df MS Fobtained

Between

Within

TOTAL

Fcritical =

If Fobtained < Fcritical fail to reject H0

Use Appendix B Page A-29-A31 for Fcritical

df numerator = (df for between)

df denominator = (df for within)

Problem 1

To test the significance of variation in the retail prices of a commodity in three principal cities, Mumbai, Kolkata and Delhi, four shops were chosen at random in each city and the prices who lack confidence in their mathematical ability observed in rupees were as follows:Mumbai : 16 8 12 14Kolkata : 14 10 10 6Delhi : 4 10 8 8Do the data indicate that the price in the three cities are significantly different?

Beyond one-way ANOVA

Often, you may want to test more than 1 treatment. ANOVA can accommodate more than 1 treatment or factor, so long as they are independent. Again, the variation partitions beautifully!

  SST = SSB1 + SSB2 + SSW  

Two Way ANOVAANOVA in which two criteria (or variables) are used to analyse the difference between more than two population means

Block : Source of Variation

Blocking Variable : A variable that a researcher wants to control but is not the treatment variable of interest. (agricultural origin: block of land)

Total Variation SST

Variation within Samples(SSE)

Unwanted Variation

between Blocks(SSR)

New Variation due to random

Error(SSE)

Variation between Samples (SSTr or SSC)

Two Way ANOVA Table

Source of Variation

Degrees of Freedom

Sum of Squares

Mean Sum of Squares (MSS)

F Ratio

Between Treatments (Columns)

c – 1 SStreatments Sstreatments / (c-1)

MSStreatments

/MSS error

Between Blocks (rows)

r - 1 SS blocks SS blocks / (r – 1)

MSSblocks

/MSS error

Residual Error

N – c - r + 1 SS error SS error/(N-c-r+1)

Total N – 1 SStotal

Problem 2

The following table gives the number of refrigerators sold by 4 salesmen in 3 months May, June, and July. Is there a significant difference in the sales made by the four salesmen? Is there a significant difference in the sales made during different months?

Month Salesmen

A B C D

May 50 40 48 39

June 46 48 50 45

July 39 44 40 39

ANOVA summary

A statistically significant ANOVA (F-test) only tells you that at least two of the groups differ, but not which ones differ.

Determining which groups differ (when it’s unclear) requires more sophisticated analyses to correct for the problem of multiple comparisons…

Question: Why not just do 3 pairwise ttests?

Answer: because, at an error rate of 5% each test, this means you have an overall chance of up to 1-(.95)3= 14% of making a type-I error (if all 3 comparisons were independent)

 If you wanted to compare 6 groups, you’d have to do

6C2 = 15 pairwise ttests; which would give you a high chance of finding something significant just by chance (if all tests were independent with a type-I error rate of 5% each); probability of at least one type-I error = 1-(.95)15=54%.

Recall: Multiple comparisons

Correction for multiple comparisons

How to correct for multiple comparisons post-hoc…

• Bonferroni correction (adjusts p by most conservative amount; assuming all tests independent, divide p by the number of tests)

• Tukey (adjusts p)• Scheffe (adjusts p)• Holm/Hochberg (gives p-cutoff beyond

which not significant)