Dr Robert CockcroftMcMaster Institute for Innovation and Excellence in Teaching and Learning (MIIETL)Physics and Astronomy DepartmentIntegrated Science Program

Kaitlyn GonsalvesDepartment of Psychology, Neuroscience and Behaviour, Level IVMIIETL Student Scholar, 2015-2016

Susan HeDepartment of Biology, Level IVMIIETL Student Scholar, 2015-2016

Devra CharneyArts & Science and English, AlumnaMIIETL Student Scholar, 2014-2015

How Do Students’ Perceptions of SCIENCE 1A03 Compare Over Time?

Outline� SCIENCE 1A03

� Pedagogical study

� 2014 results� Overview of previously

reported results� New: Impressions of

mentors from 2014

� 2015 results� Perceptions over time

� Future work

R

� Piloted Sep-Dec 2014; second offering Sep-Dec 2015� Instructors:

Dr Sarah Symons and Dr Mic Farquharson� Instructional Coordinator:

Geneviève van Wersch

SCIENCE 1A03 Investigating Science: Opportunities and Experience

R

2014 2015Students 158 275

Mentors 25 33

TAs 4 6

� Five hours of class per week� Two hours together as a class� Three hours in sections

� Course components� MRIs (or mini-research investigations)� Skill development activities� Introductory talks from the different science disciplines

� Assessment� Avenue-based weekly quizzes (30%)� MRI deliverables (40%)� Learning Portfolio written reflection (30%)

SCIENCE 1A03 Investigating Science: Opportunities and Experience

R

Pedagogical Study: Impact and Perception of SCIENCE 1A03 � Ethics approval: MREB 2014 162 � Visited students twice in tutorials

� Pre- and post-surveys� Purpose of the study, informed consent, what will happen,

confidentiality & concerns, benefits, questions� Completely voluntary; did not affect grades

� Online / paper copies of survey� Three consent options� Survey follow-up questions

� Focus group � Grant access to course reflections� Follow-up possibility in Level II, III, IV

� New data: mentors and focus group follow-up

S

2014 Results Overview7 = Extremely Useful1 = Extremely Unhelpful

7 = Extremely Enjoyable1 = Extremely Unenjoyable

Most enjoyed components� Introductory lectures (22)� MRIs (12)� Small-group work (10)� Mentors (8)

“I found the ‘introducing...’ lectures to be the most enjoyable component of this course. It helped me choose which program I would like to go into next year and gave me a glimpse of all the possible opportunities in that field.”

“I enjoyed the Mini Research Investigations the most because it engages one in various areas of science and gives an insight to that science that helps develop communication and teamwork skills.”

K

2014 Results OverviewWould you take the course again, now knowing what you know?

Would you recommend this course to other incoming first-year students?

Components with room for improvement:� MRIs (24)

“I could still see the MRI's having the most room for development. A lot of the MRIs are extremely well developed, except certain parts of them are a bit awkward, shall I say. The one where you had to go out and measure the circumference of a tree comes to mind, as that is a more difficult task to accomplish in the short time that was given to us.”

“MRIs, they are just a little rough, but nothing that can't be fixed ”

K

New Results from 2014First-year students’ impressions of peer mentors

� Peer mentors led tutorials� Supported MRIs� Non-MRI weeks:

� Answering questions or concerns raised by the students, � Introducing topics that were deemed relevant based on their

own experience � test and exam preparation, � decreasing stress levels, � Level II program selection

� Elaborated on the material from class. � Preparation for these tutorials occurred as part of the

mentorship course.

R

Mentor Question 1� 45 Responses (29%)� How beneficial were the peer-mentors to your

success this semester?� Ratings 0-4 has zero responses

7 = Very beneficial6 = Beneficial5 = Slightly beneficial

S

Mentor Question 2From your perspective, what were the benefits of having peer-mentors?

� 35 Gave good/helpful advice� 22 Relatable for first year students/can speak from

experience about university life� 18 Offered their insight into upper year programs� 12 Could explain course concepts effectively and

answer questions� 11 Approachable/friendly� 4 They were easier to approach/more useful than

were TAs

K

Mentor Question 3

R

Were there any challenges associated with having peer-mentors?

Mentor Question 4The peer-mentors were working to develop their own skills through the course. Did you notice development in any of your peer-mentors?

� 17 More effective class presentation/leadership skills� 15 More approachable/better communication skills� 13 More confident/comfortable� 5 Better able to explain concepts and answer

questions� 2 More organized/prepared

S

Mentor Question 5Do you have any other comments about the peer mentors that you feel would be important to share?

� 16 Great/awesome/friendly/useful /thank you for mentoring us

� 3 Helped me transition into university � 2 My favourite aspect of this course

K

Dec 2015 Results� Focus groups� Originally planned 6 groups� Chance to win 1 of 2 Bookstore vouchers ($25 each)� 4 interviews conducted in November� Approximately 30 minutes each� Transcribed; finished in December

R

https://rmsbunkerblog.wordpress.com/tag/focus-group-facility-in-syracuse-ny/ http://blog.smartbear.com/requirements/how-to-interview-users-to-find-out-what-they-really-want//

Interview Questions� SCIENCE 1A03’s influence on decisions?

� Transferable MRI experience/skill set?

� Appreciation changed?

� Missing components from “Introductory…” talks?

� Should course be mandatory?

S

Preliminary Coding ResultsDid SCIENCE 1A03 influence you for any of your decisions from Level I to Level II or for your potential future career?

� Level I Æ II: Nuances� More-informed approach� Career: No

S

“[The intro talks] were nice. They helped… I actually thought biochemwas more what I was going to go with then I realized [it was] more bio than biochem that I actually wanted.”

“Being introduced to all the programs and being able to kinda see that, even though you started on program you can still branch off to other ones. I think that was a really important factor in me making my decision.” [Chem phys Ælife sci]

Preliminary Coding ResultsHave you experienced any work in other courses apart from SCIENCE 1A03 where your MRI (mini-research investigations) experience/skill set has helped you?

� How to read a journal� Teamwork

� Leadership� Problem-solving within a group

� Goal-setting� Editing your own writing� Presentations – introductory� Computer - introductory

K

Preliminary Coding ResultsIs there anything from your SCIENCE 1A03 that you didn’t appreciate at the time, but now you have come to appreciate having had the experience? (No, but…)� Opportunities and insights

� Draft and feedback on your work� Introductory talks� Seeing differences between benefits to individuals versus the class� Study tips from mentors� Seeing mentors’ different presentation styles

� Pros� No textbook� “Lighter” course� No exam

� Suggestions� Lab component

R

Preliminary Coding ResultsAre there any aspects of SCIENCE 1A03 that you haven’t found useful [for other courses yet]?

Didn’t find useful:� MRI instructions not clear� Content of a particular MRI (carbon sequestration)� TA was not engaged� Communication between TAs and mentors� How to find/write a journal article was not included

S

Preliminary Coding ResultsNow that you have transitioned into Level II, do you think there was any other material that could have been included in the “Introductory…” Talks but that was missing when you saw them?

� Not really! Very well received� One response: needed

more from the student experience/perspective

K

“… There are so many options and you don’t know any of them… I had no idea that other program was like that… They gave a run-down of what the program is like and then had some videos and examples. I think those are pretty well structured.”

Continuing Work� More detailed analysis

� Comparing incoming students: 1A03 and non-1A03 students� Comparing 1A03 pre- and post-survey� Include perspectives from mentors, TAs, instructors

� Mentors� Enrolled “SCIENCE 3A03 – Peer Mentoring in Science” course � Instructors: Kris Knorr, Dr Lori Goff� Mentors’ reflections on their own experiences

� Survey follow-up questions� Focus group (Jan / Feb 2016)� Fold in course reflection components� Continue follow-up possibility in Levels II, III, IV

� Phase III� Continuation of study for the Sep 2016 course

R