e-AFFECT: What have we learnt?

  • View
    21

  • Download
    3

Embed Size (px)

DESCRIPTION

e-AFFECT: What have we learnt?. Dr Anne Jones Centre for Educational Development. Approach to project. after Cooperrider & Whitney (2005). Participating programmes. Lessons about Assessment & Feedback business processes. Managing assessment and feedback. - PowerPoint PPT Presentation

Transcript

PowerPoint Presentation

e-AFFECT: What have we learnt? Dr Anne JonesCentre for Educational Development

Brief outline Jisc funded for 3 years

Institutional change in A&F using technology where appropriate

1Approach to project

after Cooperrider & Whitney (2005) Appreciative Inquiry = positive approach to change > ownership of the changes proposed and implemented2

Phased over 3 years but actually longer term than that the process has been established and can be continued to be used.3Participating programmesAHSSEPSMHLSBusiness ManagementCivil EngineeringBiomedical SciencesDramaComputer Science

MidwiferyEnglishEnvironmental Planning

Pharmacy (baseline only)Film Studies

PsychologyLawMusic

Social Policy4Lessons about Assessment & Feedback business processes

5Managing assessment and feedbackNot all Schools have assessment policies Sometimes there is one for an area within a SchoolSome are very detailed Others are very prescriptiveHard copy versus electronic submission of courseworkVariation in the release of feedback and marks on courseworkVariation in the provision of feedback on exams

6Lessons about Assessment & Feedback timelines

7Assessment and feedback timelines

Adapted from the ESCAPE project

8Assessment and feedback timelines

ENG9Level 1

Level 3

EVP L1 and 310Level 2

Level 3

DRA L2 MUS L311Student and staff views Student and staff questionnairesStudent questionnaireStudy and learning strategiesExtent to which practice is occurringAbout themselves

Response rate = 20.4%Staff questionnaireAssessment and feedback strategiesExtent to which practice is occurringAbout themselves

Response rate = 47.3%

13Student profileFaculty

Year of study

Gender

Age

AHSS

28.7%Level 1

36.9%Female

74.5%18-22 years

77.1%EPS

47.2%Level 2

33.5%Male25.7%23+ years

22.9%MHLS

24.1%Level 3+29.6%14Student study and learning strategies

15Study and learning strategies Amount and distribution of study effort6 statementsAssignments and learning8 statementsQuantity and timing of feedback6 statementsQuality of feedback6 statementsWhat students do with feedback6 statementsThe examination and learning6 statements

Based on Gibbs and Simpson (2004)

16AnalysisFacultyYear of studyGender AgeAmount and distribution of study effort6 significant5 significant2 significantAssignments and learning5 significant1 significant2 significant4 significantQuantity and timing of feedback6 significant2 significant1 significant4 significantQuality of feedback6 significant6 significant4 significantWhat you do with the feedback4 significant5 significant2 significant5 significantThe examination and learning 6 significant2 significant1 significant4 significant

Significance p 0.05Chi2 more students than expected .

Cramers V for size effect

Add some general comments re these

17Key findingsFacultyYear of studyGender AgeOn this course, it is possible to do quite well without studying much.Moderately strongnsModerately strongnsIn weeks when the assignments are due I put in many more hours.WeaknsModerateWeakThe assignments are not very challenging.ModerateWeakStrongWeakThe feedback does not help me with any subsequent assignments.Moderately strongModeratensnsI understand things better as a result of the exam.WeaknsnsModerate

Cramers V for size effect

Add some comments to this

On this course, it is possible to do quite well without studying much Faculty More students than expected in EPS strongly agreed/agreed; more students than expected in AHSS agreed BUT more students than expected in MHLS strongly disagreed with thisGender more male students strongly agreed/agreed than expected more female students than expected strongly disagreed/disagreed with this statement

In weeks when the assignments are due I put in many more hours.Gender more female students than expected strongly agreed/agreed with this statement more male students than expected strongly disagreed/disagreed with this statement

The assignments are not very challenging.Faculty More students in AHSS and EPS than expected strongly agreed/agreed with this statement more students than expected in MHLS disagreed with the statementGender more female students than expected strongly disagreed/disagreed with the statement more male students than expected agreed with the statement

The feedback does not help me with any subsequent assignments.Faculty more students than expected from EPS strongly agreed/agreed with the statement more students than expected from AHSS and MHLS disagreed with the statementYear of study more students than expected at L1 strongly agreed/agreed with the statement more studnets than expected at L2 strongly disagreed with the statement and more than expected at L3 disagreed with the statement

I understand things better as a result of the exam.Age more older students than expected strongly disagreed/disagreed with the statement more younger students agreed with the statement

18Student and staff perceptions of practice

1927 statements based on ATLAB questionnaire (Whitelock and Cross, 2011)Fully occurring not in a position to sayAssessment criteria are written and usedAll marking criteria are shared with students in advance and feedback refers to themFeedback is intended to acknowledge, consolidate and promote student learningStudents get feedback which corrects errors and supplies further informationStudents have opportunities for cooperative and collaborative assessment

ATLAB Assessment for Teaching and Learning Audit Benchmarks project @ OU20AnalysisStudentsStudent - StaffFacultyYear of studyGenderAgeNumber of significant statements 17164623Significance p 0.05

The four of the statements for which there was no significant difference between students and staff were:Assessment activities and tasks develop students responsibility for their learningThe programme and timetable of assessment are given to students at the outsetStudents have opportunities to engage actively with assessment goals, criteria and standard settingStudents are formally consulted about assessment during the design process21StudentsFacultyYear of studyGenderAgeStudents have opportunities for cooperative and collaborative assessmentModerately strongModerate nsWeak Assessment activities are situated within real-world contexts (i.e. they reflect the problems, tasks or competencies a practitioner would face) Moderatensnsns

Students have opportunities for cooperative and collaborative assessmentFaculty more students than expected in AHSS indicated that this was not occurring more than expected in EPS that is was not/partially occurring more than expected in MHLS that it was mostly/fully occurringYear of study L1 students not occurring L2 students partially occurring L3 students mostly/fully occurring

Assessment activities are situated within real-world contexts (i.e. they reflect the problems, tasks or competencies a practitioner would face) Faculty AHSS mostly occurring EPS not/partially occurring MHLS fully occurring

22Students and staffStaff - studentsStudents have opportunities for a feedback dialogue (peer or teacher-student) around assessment tasks on the courseModerate Assessment includes the use of a portfolio or similar practice Moderately strongA variety of communication channels are used to provide feedback to studentsModerately strongStudents are told what type of feedback they can expect and when and how they will receive it Moderate

The four of the statements for which there was no significant difference between students and staff were:Assessment activities and tasks develop students responsibility for their learningThe programme and timetable of assessment are given to students at the outsetStudents have opportunities to engage actively with assessment goals, criteria and standard settingStudents are formally consulted about assessment during the design process

Other wise the general opinion of students was that particular actions/processes etc were either not occurring or were partially occurring whereas staff opinion was mostly/fully occurring.

Students have opportunities for a feedback dialogue (peer or teacher-student) around assessment tasks on the courseStudents not occurringStaff fully occurring

Assessment includes the use of a portfolio or similar practice Students not occurringStaff partially/mostly occurring

A variety of communication channels are used to provide feedback to studentsStudents not/partially occurringStaff mostly/fully occurring

Students are told what type of feedback they can expect and when and how they will receive it Students not/partially occurringStaff fully occurring

Communications? Do students understand what is meant? Is it assumed that students know/understand?23Lessons from the methodology

24

Appreciative Inquiry approach

after Cooperrider & Whitney (2005)

25

Engendering dialogue time and space

26

Bespoke technology workshops27Lessons from activities28

29

Opportunity to act on feedbackHelp clarify good performanceDevelopment of self-assessment and reflectionCivil Engineering 130

Opportunity to act on feedbackHelp clarify good performanceDevelopment of self-assessment and reflectionEncourage time and effort on challenging tasks

Civil Engineering 231

also includevideo and audio stimuli, to target particular phoneticfeatures.

Fast feedbacks really important in phonetics. You need to establish good habits right from the