Regression to the Mean, Kevin Cummins, Addictions Seminar, UCSD

1

Regression to the Mean

Addictions Seminar

9/17/08

Kevin Cummins

The MEAN

2

Outline

• Objective

• Definition

• Description

• Implications

• Recent addictions literature

• What to do about it

3

Objective

• Be able to identify regression to the

mean

• Know how to respond to its presence

• Recognize that the concept is used

loosely in addictions

4

Definition

• Regression to the Mean (RTM): The statistical phenomenon stating that the greater the deviation of a random variable from its mean, the greater the probability that a subsequent observation will deviate less far.

• In other words, an extreme event is likely to be followed by a less extreme event.

5

When It Happens

• RTM occurs whenever a

unrepresentative sample is selected

from a population and then repeated

measures are taken.

6

Example

7

Simulation

• http://www.ruf.rice.edu/~lane/stat_sim/reg_to_mean/index.html

True Value

Record

ed V

alu

e

8

Why it Happens

0

.1

.2

.3

.4

y

-4 -2 0 2 4

x

y

y

x1

9

Why it Happens

All Variance is Random Measurement Error

0

.1

.2

.3

.4

y

-4 -2 0 2 4

x

y

y0

.1

.2

.3

.4

y

-4 -2 0 2 4

x

y

y

x2x1

10

Why it Happens

No Measurement Error

0

.1

.2

.3

.4

y

-4 -2 0 2 4

x

y

y

x2x1

11

Why it Happens

Person-Person Variation and a

Little Measurement Error

0

.1

.2

.3

.4

y

-4 -2 0 2 4

x

y

y

x2

x1 x2x1

12

Why it Happens

Observed distribution with error

has thicker tails

0

.1

.2

.3

.4

-4 -2 0 2 4

x

yy

y

W/ErrorNo Error

13

Components View:

V ( yij)

Person

2 " Error "

2

V ( yij)

Person

2 measurement

2 unexp lained

2

y

ij person

i (measurement _ error unexp lained _ error)

j

V ( yij)

Person

2 " Error "

2

V ( yij)

Person

2 measurement

2 unexp lained

2

y

ij person

i (measurement _ error unexp lained _ error)

j

Components which vary across measurements

14

0

.1

.2

.3

.4

y

-4 -2 0 2 4

x

y

y

y

y

More subjects with true means below the cutoff are included in the sample than

excluded subjects with true means above the cutoff.

A

Distribution of repeated measurements on subjects

with the same true mean: A) mean above the cutoff,

and B) mean below the cutoff.

B

15

Formal Description

16

Formal Description of RTM

P(x)dx P(x)dx j

j

i

i

for i > j > 0

This is a stochastic property; it applies to

random variates.

As you move away from the mean, the proportion of

the distribution that lies closer to the mean increases

continuously.

17

Outline

• Objective

• Definition

• Description

• Implications



18

Clinical Implications

• Diagnostic Tests

• New Treatments

• Public Health

• Clinical Performance

Adjustments

• Placebo Effect

Morton & Torgenson 2008

“[RTM] can result in

wrongly concluding

that change is due to

treatment when it is

due to chance”

Mistaken spontaneous

reversion

Application to clinical

outliers increases

RTM influenceTreating clusters

Random components

Interpreting change

as a placebo effect

19

Implications: Longitudinal

Research

0

2

4

6

8

10

Measure

1 1.2 1.4 1.6 1.8 2

time

Meas ure

Meas ure

Mean

20


Research

Mean

0

2

4

6

8

wceilin

g

1 1.2 1.4 1.6 1.8 2

time

wceiling

wceiling

Mean

Me

tric

with C

eili

ng

Max

21

Implications for Longitudinal

Research

y

ijt time_ effect

t person

i error

j

If your sample is not representative of the

population there will be a “time effect” due to

regression to the mean.

Side Note: Distributional assumptions can be

violated.

22

Components View:

Two Time Periods & Two Tx

V ( yijt

) Person

2 "error "

2

y

ijkt treatment

k+ development

t person

i error

ijt

y

ijt time_ effect

t person

i error

ijt

Each estimated with comparisons

WARNING: Avoid concluding that an

observed change is due to treatment

or development without comparisons

or corrections.

23

Would you assume that distance of these cows above

sea level is measurement error?

24

Outline

• Objective

• Definition

• Description

• Implications



25

Recent Addictions Literature:

Out Bicycling

Babor 2008

“How often have we heard [treatment researchers] casually invoke the RTM concept as a possible explanation for general improvements in post-treatment drinking behavior?”

26


Stout

“RTM is used in a number of contexts in addiction research”

Contexts of RTM

Context 1 & 2 are actually the presence of random components. Stout distinguishes measurement error and unexplained variation.

Context 3 is “Measurement Bias”/True Change

27


Stout

“...under many circumstances RTM effects may dwarf intervention effects...”

KMC

This will happen whenever unexplained variance is

high relative to intervention effect size.

28

Ripatti

Benefit

Reduces error variance

Assumptions

Errors are assumed ~Niid – RTM may still impact

Stationarity– No trending modeled

Limitations on Interpretations

Confounding not addressed

y

ikt time_effect

k f ( y

t1) error

it

29


Finney

Finney suggests assessment at multiple time points

prior to treatment application

Big Assumption

Stationarity

Limitation

Confounding remains an issue

RTM not eliminated

30


Finney

Observational Studies

• Matching– Does not include the whole sample

• Covariate Adjustment– RTM is not eliminated

31


RTM in Addiction ResearchGmel 2008

Aim: provide statistical methods to disaggregate

change and estimate its components:

1. “True change”

2. “Random fluctuations”

3. “Measurement error”

y treatment + development true_ var iation person error

32


Gmel et al. 2008• Oldham Method

Need a random sample?

• Tu Method

y1 y2 [(y1 y2 ) / 2]

y1 y2 y1Under a corrected null Ho

Restricted use and interpretation & fixed effects not parsed

33


Gmel: Barnett Method

Calculate the expected magnitude of RTM and subtract that from the observed change.

Benefits: Useful when there is no comparison group

Requirements:

1. Need to know population variance and within-subject variance, which must be constant.

2. Need to know population mean.

3. The population and errors must be normally distributed.

Issue: No recognition for sampling error.

34


Gmel: Growth Curve Method

• “RTM is often reported based on the correlation between initial status and observed change”

If the models structure is correct, RTM will reduce because the error variance will shrink. The reduction will be proportional to the number of within-subject observations.

35

Assumptions in the Addictions

Literature

• Stationary

• Normality

• Leaps beyond the limits of observational

studies

36

What to do About RTM

• Minimize measurement error

• Repeated/independent measures– Multiple Measures

– HLM

• Quality comparison groups– Randomization

• Statistical Corrections– Ripatti models

• Make reasonable conclusions

37


RTM in Addiction ResearchFinney 2008

Reducing RTM

• Reducing RTM is not necessary to obtain unbiased treatment effects in RCT

• Take repeated measurements – (ie reduce sampling variation)

– Finney suggests assessment at multiple time points prior to treatment application

38


Research

y treatment + development true_ var iation person error

39


Finney 2008

“the aim of this paper is to raise awareness of RTM”

Comparative Studies

“RCTs do not eliminate RTM”

“...treatment-seeking patients would tend to improve in the absence of treatment as a result of RTM”

“[true changes] fluctuate around a mean level of functioning for an individual over time”

Conclusions: Don’t blindly ascribe change to Tx.

40

Components View:

V ( yij)

Person

2 " Error "

2

V ( yij)

Person

2 measurement

2 unexp lained

2

V ( yij)

Person

2 " Error "

2

V ( yij)

Person

2 measurement

2 unexp lained

2

y

ijt time_ effect

t person

i error

j

Components which vary across measurements

Estimate with

distributionsEstimate with

comparisons

41

42

Objective

• Be able to identify regression to the

mean

• Know how to respond to its presence

• Recognize that the concept is used

loosely in addictions

Data & Analytics

Regression to the Mean, Kevin Cummins, Addictions Seminar, UCSD