InterventionsCWSEI End‐of‐Year event

27 April 2010

Louis Deslauriers & Sara Harris

What is an “intervention”?Typically, an intervention to help low performing students would involve:

1‐ Providing extra review sessions 2‐ Distributing additional practice problems 3‐ providing additional office hours p g4‐ Help rooms where students can work together with TAs5‐ Etc...

Typically not too effective though!!Typically not too effective though!!High performing students avail themselves of these additional resources, but lower performance students DO NOT. Many documented (and undocumented) interventions failed to improve performanceMany documented (and undocumented) interventions failed to improve performance of these lower performing students.

“Alth h t i hi h th th t i t d kill U i it t d t th ff t“Although most programs in which the thrust is study skills use University students, the effect on study skills are minimal”J. Hattie, J. Biggs, N. Purdie, Review of Educational Research 66, 99‐136 (1996)

We successfully intervened with low f i t d t i performing students in

two VERY different courses:

PHYS 250 (Modern Phys) May 2009

Student population: 67 second year engineering phys students

lEOSC 112 (Fluid Earth: Atmosphere & Ocean) Sept 2009

Student population: 180 students any year any facultyStudent population: 180 students, any year, any faculty

Who gets an intervention? PHYS 250

90%

100%Phys250: Modern Physics ‐‐ 67 Students enrolment

60%

70%

80%Scores after Midterm1

40%

50%

60%

10%

20%

30%

0%

10%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67

5 Students emailed, but did not respond

12 Students emailed, and intervened with

Who gets an intervention? EOSC 112

udied

an hou

rs stu

50% Pass/fail lineMed

ia

S di d l di & f il dStudied > class median & failed

Who gets an intervention? EOSC 112

udied

an hou

rs stu

50% Pass/fail lineMed

ia

Came to face‐to‐face interventionInvited for F2F, didn’t come, got emailGot email (not invited to F2F)Could consult study tips on website

What does an intervention look like? PHYS 250

What does an intervention look like? Intervention: Email is sent out to the ones who scored at bottom 25% on Midterm 1 

We sent 17 emails, and intervened with 12 students.We sent 7 emails, and intervened with students.

Took place about 1‐2 weeks after Midterm 1.One on one with one of the instructor lasted about 15 25min eachOne‐on‐one with one of the instructor, lasted about 15‐25min each

Questions asked:

1‐What did you think of M1? (Hard/easy, fair/unfair, blabla)2‐ How did you prepare for it?y p p3‐What do you think could be done to improve on M2? 4‐ Volunteered our help to improve their performance on exams

What does an intervention look like? PHYS 250

What does an intervention look like? Intervention: We asked them to carry out a specific set of activities in studying for future exams. 

1‐ Review the homework problems and solutions2‐ Go to the weekly (optional) problem solving sessions3 Study the learning goals for the course they had been given with the3‐ Study the learning goals for the course they had been given with the 

syllabus4‐ Review the assigned reading for the course 

Each student was asked to agree to do these things when studying for the next exam and told we would be checking on their progress before the next exam.

Did they do better after an intervention? PHYS 250

Final ExamFE ‐M1

M1 (%) M2 (%)Final Exam 

(%)10 students 

(with intervention)64.5 83.4 72.6

8.1

Next 16 studentNext 16 student (w/o inter. or email) 77.4 80.3 72.9

‐4.5

Entire class except 10 intervened stds

81.5 83.1 76‐5.7

Do students do better ft i t ti ?

EOSC 112

after an intervention?

Average, non‐interv students

Medians, all & non=int

Do students do better after an intervention?

EOSC 112

after an intervention?

Intervention students report changing study habits

EOSC 112

changing study habits2 = Changed study habits “a lot”1 = Changed study habits “a little”1   Changed study habits  a little0 = Did not change study habits

How do we know the intervention was responsible?

PHYS 250

was responsible? Previous results clearly established that intervention had large effect, but it does not say why.For instance did they really change their study habits or simply worked much harder forFor instance………did they really change their study habits or simply worked much harder for the rest of the term?

At end of semester, we conducted interviews with the “10 student intervened group”. For comparison, identical interviews were carried primarily with those students scoring in the next higher quartile after the first midterm (“control group”) . 

To avoid biasing the student responses the interviews were carried out by a third person

1‐ How do you typically prepare for exams in science and eng courses here at ubcd d f d h

To avoid biasing the student responses, the interviews were carried out by a third person who was unknown to the students  and followed a very strict and limited script. 

2‐ How did you prepare for midterm 1 in phys250 3‐ How did you prepare for midterm 2 in phys250 ‐‐ did you change anything in the way you 

prepared for M1 Vs M2? If so, what? 4‐What study advice would you give another PHYS250 student in our course for him/her to y y g f /

do well in the course. 5‐What study advice would you give another student in OTHER science and eng courses for 

him/her to do well. 

How do we know the intervention was responsible?

t d tstudied diff tl t HW* LG * b k*

pre d*

rev * thstudent differently  notes HW* LGs* book* read* sess* more other

1 int yes  NEW NEWm1‐NEW NEW NEWm1

2 int yes   m1 m1m1‐NEW NEW

3 int yes  m1 m1 m1 m1 NEW4 int yes NEW NEW NEW4 int yes  NEW NEW NEW5 int yes  m1 NEW m1 NEW6 int no   m1 m17 int yes m1 m1 m1 m1 NEW7 int yes  m1 m1 m1 m1 NEW8 int yes  m1 NEW9 int no  m1 m110 int yes  m1 m1 m1 NEW NEWy

11 cNo Int no  m112 cNo Int no  m1 m113 c

Conclusions andSuggestions for interventions:

‐ These results demonstrate that student performance should not be considered something outside of the instructor’s controlbe considered something outside of the instructor s control

‐ Can make a dramatic difference in performance with minor but i t l t t d i t ti t i t d h bitappropriately targeted intervention to improve study habits.

‐ Course needs to be structured with all components aligned (learning goals, assignments, assessments,…