Implementing Transparency, Motivation and Learning Strategies Interventions in Freshman Barrier 

Science Courses

Dr. Adriana VisbalDr. Eszter Trufan

July 14, 2015  Austin, Texas

STUDENT SUCCES SUMMIT 

Advancing Underserved Student Success Through Faculty Intentionality 

in Problem‐Based Learning

• University of Houston‐Downtown was one of 7 participating institutions in this national project sponsored by AAC&U

• 5 faculty across several disciplines at UHD participated in the project.

Project Design• Each faculty taught two sections of the same course.

• One or several assignments were created or modified with the goal of assessing problem‐solving skills (using AAC&U’s VALUE Rubric).

• For one of the sections taught, transparent assignment design was implemented and emphasized along with other transparency techniques.

• Students took a transparency survey initially and at end of course.

WhatisTransparency?

• Transparent teaching and learning methods help students understand how and why they are learning course content in particular ways.

How can Transparency help your students?

• Transparent teaching/learning methods benefit students who are unfamiliar with college success strategies by explicating learning/teaching processes.

Greater benefits for underrepresented and first-generation students

Winkelmes, AAC&U’s Liberal Education 99, 2 (Spring 2013)

Results

• Preliminary results from tranparency survey results are complete

• Scoring od student work using the Problem-Solving VALUE rubric is underway.

• Project completion and publication at the end of the year.

WhatdoesTransparentAssignmentDesignlooklike?

Faculty/Instructors agreed (in national study)to discuss with students in advance:

•Purpose• What Skills will students practice?• What Knowledge will students gain?

•Task• What students will do • How to do it

•Criteria for success• What excellence looks like (annotated)• Criteria in advance to empower students to self-

evaluate

AdvancingUnderservedStudents’SuccessthroughFacultyIntentionalityinProblem‐centeredLearning

Finley,McNair,Winkelmes©2015

TransparencyTechniquesforAssignmentDesign

1.Flexible formats appeal equitably to various learners’ strengths

2.Build students’ critical thinking skills in a logical sequence

3.Criteria for success (provide to students in advance)

4.Critiqued student work (provide to students in advance)

5.Self, peer and group evaluations (provide to students in advance)

6.Explicate assignments’ purpose, task, criteria in advance

What does Transparent Assignment Design look like?

Faculty/Instructors agreed (in national study)to discuss with students in advance:

•Purpose• What Skills will students practice?• What Knowledge will students gain?

•Task• What students will do • How to do it

•Criteria for success• What excellence looks like (annotated)• Criteria in advance to empower students to self-

evaluateWinkelmes, AAC&U’s Liberal Education 99, 2 (Spring 2013)

Research on Learning Implications for Transparent Design(numbers refer to page numbers in the handout)

Elbow, Jaschik/Davidson, Mazur Low stakes for greater creativity / risk

Aronson, Dweck, Fisk, Light, Schnabel, Spitzer, Steele,Treisman, Yeager/Walton, Vygotsky

Structure and require peer instruction, feedback; positive attribution activities. (5)

Fisk/Light, Tanner Provide annotated examples of successful work w/ criteria applied, before students begin work. (4)

Doyle, Felder, Tanner, Winkelmes

Specify pertinent knowledge/skills, criteria and encourage self-monitoring. (3)

Bass, Bloom, Colomb, Felder, Perry

Build critical thinking skills in intentional sequence.Target feedback to phase, don’t overwhelm.(2)

AAC&U Finley/McNair (HIP, P-B)Winkelmes Yeager, Walton

Explicate purposes, tasks, criteria in advance.Give students a compass, set expectations;Explicate applicability, relevance;Engage students in applying shared criteria to help develop belonging. (6)

Where does Transparent Assignment Design Come From?

Ambrose, BergstahlerGregorc, Kolb

Varied and/or flexible formats appeal equitably to students’ strengths; inclusive (1)

Implementing Transparency in General Biology I Course at UHD

Title of Project:• Making Connections Across Biology:  A problem‐based homework assignment set aimed to connect the major themes of biology and develop critical thinking skills.

• This assignment replaced the online publisher’s HW utilized in other semesters.

Example Assignment ‐Less Transparent

Title of Project:• Making Connections Across Biology:  A problem‐based homework assignment set aimed to connect the major themes of biology and develop critical thinking skills.

• This assignment replaced the online publisher’s HW utilized in other semesters.

Example Assignment ‐More Transparent

Other Transparency Techniques Utilized

• Debrief graded tests and assignments in class. The use of exam wrappers and discussion of most missed iRAT, quiz and HW questions covers this technique.

• Explicitly connect “how people learn” data with course activities when students struggle at difficult transition points.  A 40 min learning and study habit lecture from the first day of class is used as base knowledge and the course gets reminders about Bloom’s Taxonomy and what I hope different assignments or questions will achieve.

• Gauge students’ understanding during class via peer work on questions that require students to apply concepts you’ve taught. The Application Activities of the Team‐Based Learning format of class best exemplifies this technique.

Student Evaluations Reflected Faculty Intentionality in Applying Transparency

Student Evaluations Reflected Faculty Intentionality in Applying Transparency

• Control: 8/10 comments were in direct reference on instructor or course. Some more detailed than others but emphasis was on student's likes or dislikes. 2/10 comments included mentions of how the course and instructor encouraged "thinking outside the box" and how they enjoyed that course material was "directly related to real‐life examples”

• Experimental: 3/8 comments  were in direct reference on instructor or course. Some more detailed than others but emphasis was on student's likes or dislikes.                                                                                      5/8 comments included a reflection of student's learning. Comments included ".It made me learn new ways to study and achieve my goal.""One thing in particular I felt she did well was tying all of the material together.". She made it clear from the beginning that her objective was to get us to the point where we could read a problem and logically tie together an educated answer about biology topics rather than regurgitating information."

CHEM 1307 Redesign

Identify the problem

What do you think is the biggest reason for a student not passing a difficult course?

The Changes Over 10 Years

Fall classes Official enrollment Students taking final exam Difference

2005 60 39 21

59 36 23

61 47 14

180 122 58

60 39 21

2014 55 49 654 53 158 57 145 43 249 45 4261 247 14112 110 2

All Data for Fall (2005‐2014)

YearPercentage not taking final exam

(all sections)Percentage not taking final exam 

(same instructor sections)

2005 32% 35%

2006 28% 43%

2007 35% 36%

2008 33% 35%

2009 28% 35%

2010 22% 26%

2011 27% 30%

2012 21% 22%

2013 20% 11%

2014 5% 2%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Percentage of students not taking the final exam

All sectionsSame instructor sections

Final Exam Scores for Fall Semesters

Year Final exam average2006 31.82007 32.62008 35.12009 34.72010 33.92011 32.82012 31.42013 31.12014 34.52015 33.8

The Changes Over 10 Years

Spring classes Official enrollment Students taking final exam Difference

2006 64 43 21

60 48 12

124 91 33

64 43 21

2015 60 59 1

55 53 2

46 36 10

161 148 13

60 59 1

All Data for Spring (2006‐2015)

YearPercentage not taking final exam

(all sections)Percentage not taking final exam

(same instructor sections)

2006 27% 33%

2007 28% 37%

2008 23% 15%

2009 33% 33%

2010 24% 27%

2011 24% 24%

2012 23% 27%

2013 22% 22%

2014 20% 8%

2015 8% 2%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Percentage of students not taking the final exam

All sectionsSame instructor sections

Exam Scores for Spring Semesters

Year Final exam average2006 29.72007 31.82008 36.52009 33.52010 32.62011 31.02012 32.22013 32.52014 33.62015 35.7

Early TBL implementation

32

Chemistry 1307Instructional Format

Individual Preparation

Daily Quiz (Individual) Daily Quiz (Team)

Case Study and Problem Solving – (Team)

Peer Evaluation (Team)

Lecture (Dr. Christmas)

Dr. Williams

33

TBL INSTRUCTIONAL FORMAT:

This course will utilize a Team-Based-Learning (TBL) format(www.teambasedlearning.org). You will be assigned to a team withapproximately 6 members. Your team will be determined by theinstructors and will be developed to facilitate creation of a learningenvironment involving significant team activity designed to enhanceindividual academic success. The teams will be formed during thefirst week of the semester. You will sit with your team during allclassroom sessions.

34

Phase I – Individual Preparation

You will have required reading and other assignments that youmust complete before each class meeting beginning with the 2nd day ofclass. More details about these assignments will be provided at the BBLwebsite.

35

Phase II – Individual and Team Readiness Assurance Tests (RATs):

Individual RAT (i-RAT) – At the beginning of most classes you will be givena short quiz over the assigned material (see Phase I). This will contribute toyour i-RAT grade for the course. Generally you will be given 10 minutes tocomplete the quiz. This i-RAT quiz will utilize either a Scan-Tron card oryour i>Clicker remote or Smart Phone. Be sure to bring both to all classes!

Team RAT (t-RAT) – After the initial quiz, you will be given the same quiz,or a similar quiz, to take as a team. You will be given approximately 15-20minutes to complete the quiz. The team should agree on the answers. Theresults of this second quiz will contribute to your t-RAT grade for the course.

Mini-Lectures and i>Clicker questions – After the t-RAT, the instructor mayprovide short clarifying lectures to support your understanding of selectconcepts and methods, particularly concepts and methods where significantnumbers of students have failed to answer properly.

36

Phase III – In-Class Team Activities:

You will be guided by the instructors to investigate concepts andmethods that are important in the current topical area being studied.You and your team will utilize the foundational knowledge acquired inthe first two phases to make decisions that will be reported to the classand will be subject to cross-team discussion/critique. We will use avariety of methods to have you report your team’s successes anddecisions at the end of each activity.

This phase will utilize mini-lectures, i>Clicker questions, andtopic facilitation by the instructors, as well as slides and video, to guidestudents through the activities. The student SI instructors will alsoprovide facilitation for the teams.

37

Phase IV – Peer Evaluations:

You will be asked to complete an anonymous peer evaluation of your teammates at the end of the first four weeks. This is a formative evaluation (does not count for points); it is your opportunity to exchange helpful feedback with your team partners. At the end of the course you will repeat this peer evaluation. The assessment of your team contribution by your peers will account for 2% of your course grade!

38

Phase V – Forward Lecture:

The instructor will begin discussion of a new topic and provide reading and exercise assignments that you must complete prior to the next class period. (See Phases I and II)

39

EVALUATION OF STUDENT PERFORMANCE:

4 Exams (Lowest exam score will be dropped!) - 36 %

11 MGC “Homework” Assignments - 18 %(Lowest two will be dropped!)

1 iClicker Participation - 8 %

1 i-RAT - 8 %

1 t-RAT - 6 %

1 Peer Evaluation - 2 %

1 Final Exam (Comprehensive, Chapters 1 – 11) - 22 %

CHEMISTRY 1307 NAME: _______________ i-RAT 5 CRN: _____10064 ______ September 23, 2013

Select the “BEST” answer to all of the following questions.

1. The following reaction is: Fe(s) + Cl2(g) → FeCl3(s) A. Balanced B. Unbalanced 2. Which of the following statements is/are true? 1. Stoichiometric coefficients indicate the mass ratios of the reaction contents 2. The reactant that is completely consumed in the chemical reaction is called the excess reactant.

A. None B. 1 C. 2 D. Both 3. Determine the stoichiometric coefficient in front of Fe2O3 and CO2 in the following equation. Fe2O3(s) + CO(g) → Fe(s) + CO2(g) A. 1 Fe2O3 and 1 CO2 B. 2 Fe2O3 and 1 CO2 C. 1 Fe2O3 and 3 CO2 D. 2 Fe2O3 and 6 CO2 4. If 7.00 grams of nitrogen react according to the following equation, how many moles of NH3 will result? N2(g) + 3 H2 (g) → 2 NH3(g) A. 0.25 mols NH3 B. 2.00 moles NH3 C. 1.00 moles NH3 D. 0.50 moles NH3

5. Determine the limiting reactant if 3 moles H2S of combine with 2 moles of SO2 according to the reaction: 2 H2S + SO2 → 3 S + 2 H2O A. S B. H2O C. SO2 D. H2S 6. Which of the following statements is/are true? 1. Theoretical yield is calculated based on the limiting reactant 2. (Theoretical yield/Actual yield)*100 = Percent yield

A. 1 B. 2 C. None D. Both 7. In the problem: “In the reaction of 3.0 mol of CCl4 with an excess of HF, 2.3 mol of CCl2F2 is obtained.” CCl4(liq) + 2 HF(g) → CCl2F2 (liq) + 2 HCl(g) A. CCl4 is the limiting reactant B. 2.3 mol is the percent yield C. 3.0 is the actual yield D. All are correct 8. What is the molar mass of copper(I) oxide? (MCu =63.5 g/mol and MO = 16 g/mol) A. 63.5 g/mol B. 79.5 g/mol C. 143 g/mol D. 95.5 g/mol 9. How many moles are in 36 grams of B2H6? (Molar mass B2H6 = 27.6 g/mol) A. 1.3 mol B. 0.77 mol C. 36 mol D. 993.6 mol

Students Not Taking the Final Exam

What do you think the students’ concern was and how would you address it?

Survey

1. Are you aware of /happy with your current standing in this class?

2. What are the tools that are available for the class that you found most useful? (lecture, team activities, clicker questions, iRAT, tRAT, homework, tutorials, teams, SI, Learning strategies)

3. Is there anything else that you would like to see included in the course?

4. What was the most useful/ memorable part of the semester so far in this course?

5. What would you advise the students from next semester to do in order to succeed in this course?

6. Have you considered dropping the class during the semester? If yes, why and what held you back?

Spring 2014

Yes: 24No: 23

Interested in learning, not just grade International student Talk to instructor about improving Saw improvement or it seemed possible Late assignments Financial reasons Academic probation Pace was too fast Hard class, but pre‐requisite for major Bad grade on first exam Full‐time employment Praying not to fail Others did poorly too Explore the idea of a major in science Not aware of drop date On scholarship Prove that is doable Medical issues, but caught up

What concerns do you have about asking students to apply material that you haven’t lectured on? How about testing on that 

material?

How would you address the concerns in front of you? List any resources for that you think would 

be necessary.

Changes for Summer

Tutorials for each topic with applications and examplesTake‐home quizzes on material not coveredReal‐life applications assignmentInvert/Semi‐invert classes“No Questions Asked” couponNo dropped assignments

What are your concerns about take‐home assignments?

How would you address the concern in front of you?

In what way is the molecular formula different from the empirical formula? (p. 215)

What is a solution? What are the contents of a solution? (“Preparing Solutions 101” tutorial)

Define molarity. (p. 219‐220) 

How is molarity used to convert between units of volume and moles? (“Conversions with Molarity 101” tutorial)

What are electrolytes? Give at least two examples for each type. (p 236‐238 and “Dissociation Ionization Reactions 101” tutorial)

What is dissociation? (“Dissociation Ionization Reactions 101” tutorial)

What is ionization? (“Dissociation Ionization Reactions 101” tutorial)

What are the characteristics of a precipitation reaction? (p. 239‐241)

How is solubility determined? (p.241‐242 and “Determining Solubility 101” tutorial)

CHEMISTRY 1307 NAME: _______________Quiz 8 CRN:  _____20237_______Due: March 24, 2015 GROUP NUMBER: ______

Solve the following problems neatly and completely. Show all your work.

1. (2 p) What is the molecular formula of a compound with a molar mass of 88.10 g/molthat contains 54.55% carbon, 9.09% hydrogen and 36.36% oxygen?

2. (0.5 p each) Classify the following compounds as strong electrolytes, weak electrolytes or non‐electrolytes.

NaCl:HF:O2:NH3:

3. (0.5 p each) Classify the following ionic compounds as soluble or insoluble in water?

KI:AgF:CaSO4:CaCl2:

4. (1 p) Write the dissociation reaction of FeCl3.

5. (1 p) Write the ionization reaction of H2SO4.

6. (1 p) What is the molarity of the solution that was prepared by dissolving 3 moles of NaCl to a final volume of 5 L?

7. (1 p) How many moles of calcium chloride are in 250 mL of 0.4 M solution?

Adapting the Changes for the Fall

Weekly, not daily quizzesKeep the “No Questions Asked” couponTeam examsGrade discussion after exams

Spring, One Year Later

TutorialsTake‐home quizzes every weekStudent tutorialsTeam exams“No Questions Asked” couponGrade discussions after each examIntroduce Learning Strategies in class and in Intervention Seminars

Chemistry games

56

EVALUATION OF STUDENT PERFORMANCE:

3 Lecture Exams - 36 %

11 MGC “Homework” Assignments - 20 %

iClicker Participation - 10%

13 Quizzes - 10%

1 Final Exam (Comprehensive, Chapters 0 – 10) - 24 %

57

CHEM 1307 CRN 20236Valid until 4/30/2015*

Purpose:_______________________

Name:________________________* Exceptions may apply. Read the disclaimer…

This coupon entitles you to a perfect score on only one of the following: One homework assignment;One quiz;One in-class clicker session.You can use it to replace a low score or as an excuse for an assignment that wasn’t submitted.This coupon can only be used once so be strategic. It has to be filled out to be valid.At the end of the semester, if you still have it, you can trade it in for 2% on your final grade. You will have to fill it out and bring it to class for the Final Exam. If any of your assignments are not completed, you will have to use the coupon toward fixing the mistake.No coupons will be accepted after the Last Class Day.

0%

5%

10%

15%

20%

25%

30%

35%

40%

Percentage of students not taking the final exam

What topics of learning strategies do you think students benefit the most from? How much time if any do you think you would spend on 

these topics in class?

Lesson 1: Unlearn what you’ve learned

Time spent studying vs. Actual score

0

10

20

30

40

50

60

A B C D F

7 or more hours 4‐6 hours 1‐3 hours Less than 1 hour Didn't study

Time spent Actual grade

Not all studying is the same

How does the average student study?

Evidence Based Learning Strategies

• To understand how we learn….

• Let’s look at how we forget

Lesson 2: Do or Do not, there is no Try

What does trying mean?

Responsibility Model

STIMULUS

CHOICE

STIMULUS

CHOICE

VICTIM

ResponseBlamingComplainingExcusingRepeating

behavior

Result

Rarely achieves

goals

VICTIM Voices

INNER CRITIC:The Inner Critic judges us. It blames us , complains about us, demeans us. Thisvoice often sounds like a recording of critical parents and other judgmentaladults we have known. Its positive intention seems to be protecting us fromattach from the outside. Through constant self-criticism, the Inner Critic hopesto perfect our actions, thoughts, and feelings to meet the approval of others andthus avoid their displeasure and possible punishment.

INNER DEFENDER:The Inner Defender judges others. It blames others,complains about them, demeans them. This voice oftensounds much like a scared, confused little child backedinto a dangerous corner. Its positive intention seems tobe fending off criticism and punishment from powerfulothers. As such, the Inner Defender becomes a master ofthe preemptive strike.

CREATOR

STIMULUS

CHOICE

ResponseSeeking

solutionsTaking actionDoing

something new

Result

Often achieves

goals

CREATOR Voice

INNER GUIDE:The Inner Guide offers an objective and wise perspective about ourselves and others. It neither demeans nor attacks. Instead, the Inner Guide observes the events in a Creator’s life asking simply: Am I on course or off course? If I’m off course, how can I best get back on course? Our Inner Guides tell us the absolute truth (as best we can know it at that moment), allowing us to be more fully conscious of the world, other people, and especially ourselves.

Rephrasing Activity

Supplemental Instruction opportunities

0

10

20

30

40

50

60

70

80

SI Office hours SLC More than oneof the above

None

Supplemental Instruction opportunities

0

10

20

30

40

50

60

70

SI Sessions Office Hours SLC Tutoring More than one None

% Stude

nts A

tten

ded

What do you think happens during office hours?

Lesson 3: “I can’t believe it!”

Do you think if you failed the 1st assessment you will fail class?

Spring 2015 Survey

Yes: 38No: 42

Attempt to do better Possibility of passing Scared Motivated to finish Not give up I know I am capable Trying to change study habits 3rd time in class, time to get out Learn form my mistakes Already passing, need to finish Not a quitter Should have dropped Good professor Dropped another class instead International student Like the challenge and the material Need to learn the material Try harder

Positive Changes

Students are more engaged during class and asked more in‐depth questions

Better collaborationsMore interest in chemistry/sciencesTransition to growth mentalityMore interest in the SI programLower withdrawal and failure ratesHigher percentage of completed assignmentsBetter course evaluations

What do you think should be the next step in course redesign?