Learning about ALearning about Assessment Literacy: A Case Study from the Assessment Literacy Project

Nicky Spawls & Clare O’DonoghueDept of Education

School of Health and Education

15/04/23Slide 1

The Context

• BA Education Studies & BA Early Childhood Studies• 2 Distinct programmes but with some combined modules• Approximately 150- 200 students per year in total• Diverse student cohort:

– Some with A’ Levels– Some with level 3 vocational diplomas– Young and mature students– Multicultural cohort, not all will have completed

secondary school in UK

4 Models of Assessment Literacy

1. The Traditional Model = by Osmosis: Tacit standards absorbed over time informally and serendipitously. The ability of students to absorb these is seen as a mark of their academic ability to guess what the lecturers want.

2. The Explicit Model: Standards explicitly articulated and presented to students (with inevitable limitations of fuzzy description) for transparency

3. The Social Constructivist Model: Actively engaging students in formal processes to communicate tacit as well as explicit understanding of standards

4. The ‘Cultivated’ Community of Practice Model: Tacit standards communicated through informal knowledge exchange networks seeded by specific activities, e.g. reading clubs, informal discussion

(O’Donovan, Price & Rust 2008) Slide 3

Analysis of our Starting Point

From analysis of our own courses we acknowledged some of the following :• Much good practice taking place across the programme• Tutors building in lots of support for assessment • Variety of assessment methods being used

However….• Lack of clear differentiation between formative and summative assessment• Lack of real consideration of why one thing was formative and another

summative• Neglecting to perceive the importance of formative as a building block to

summative • …and in cases, having no obvious relationship between the two• Scope for greater consistency of approach between modules / tutors etc

Slide 4

Students’ experiences of assessment can be…

Scary

Lonely

Confusing

Slide 5

Assessment Literacy: Staff & Students

• Appreciation of assessment’s relationship to learning• Conceptual understanding of assessment• Understanding the nature, meaning and levels of assessment

criteria and standards• Develop student skills in self and peer assessment• Familiarity with different assessment types• Understanding the role of formative assessment• Students understanding the value of feedback and how to

make use of it• Possessing the intellectual ability to select and apply

appropriate techniques and approaches to assessed tasksSlide 6

Need for a Programme Level Approach

• Staff team need a programme view

‘Where there is a greater sense of the holistic programme, students are more likely to achieve the learning outcomes than students on programmes with a more fragmented sense of the programme’.

(Havnes, 2007 in Price et al, 2012))

Slide 7

Our Interpretation

• To put Assessment Literacy at the heart of our team’s work in delivering teaching and learning

• To engender a deep level of engagement in assessment that is participatory and constructivist

• Through involvement with assessment criteria, to engage with feedback and specifically formative feedback,

• Important to note that this is closely integrates with Academic Literacy i.e. how to take good notes, how to write an essay, how to write a report, how to carry out effective research, how to use referencing.

• Also parallels the use of module level descriptors module level descriptors to demonstrate how expectations differ for each level of their studies and to make explicit to students how these differences in expectation can be achieved

Slide 8

Our Goals in the Assessment Literacy Project

• Promote a discourse of assessment between staff and students; ‘a community of practice’

• Why ? student voice, NSS, BoS, PPR• KPIs • Authenticity in assessment • Build on an existing dialogue of academic literacy, built into all

modules, to create systematic scaffolding in A.L. across programmes• To see the relationship between assessment for learning and the

development of successful self-regulated learners • To create fair and transparent assessment practices• To review and enhance assessment practices during this ‘window

of opportunity’ - the periodic reviewSlide 9

Engaging 1st year students with ALEDU 1305 Philosophy of Education

• Prior to term 1 assessment– Students given past years’ assignment titles to brainstorm on

topics after relevant lectures.– Students shown past examples of anonymised essays (1st and

2:1 standard) and given grading scheme to read and mark essays in seminars.

– Discussion of evaluation of essays against assessment criteria starts the dialogic AL process

– Students complete self-evaluation questionnaire on term 1 and term 2 assignment to be submitted with assignments

– 83% pass rate (9% deferred, 6% failed, 2% X grade)

Slide 10

Student Self-Assessment (1)

• The self-assessment has 3 main components:• Part 1: Academic Literacy – word count, sources, bibliography,

Turnitin originality score – paraphrasing and quotation• Part 2 – Engagement with task – most interesting / most difficult

aspects• Part 3 – Assessment Literacy• Degree of confidence (confident / not confident) to which the

marking criteria for the assignment have been met.• Areas to work on identified by previous feedback (term 2

assignment only)• Confidence with extent to which these areas have (not) been

addressed• Specific areas student requests feedback on

Student Self-Assessment (2) - Example

• Assignment feedback from term 1's work indicated that I should work on the following areas of my assignment writing in order to consolidate / improve my grades ..........

• I am confident that I have addressed the following areas of this feedback in this assignment to consolidate / improve the quality of my work ..........

• I am not very confident that I have made significant improvement in the following areas of previous feedback in this assignment.....

• I would particularly welcome feedback on the following aspect of my work in this assignment ..........

Slide 12

Engaging 2nd year students with ALEDU2309 ‘Insights into Play’

• Active engagement with the assessment standards through marking example observations and essays

• Each summative assessment is preceded by a directly linked formative assessment

• Feedback on formative assessment through individual tutorials – dialogic process through which assessment standards are constructed, as well as communicated

Slide 13

EDU2309 ‘Insights into Play’

• Term 1

Formative Assessment 1 - 1% first 45% good honours

Summative Assessment 1 - 14% first 49% good honours

• Term 2

Formative Assessment 2 – 22% first 50% good honours

Summative Assessment 2 – 22% first 58% good honours

Overall module grades - 92% pass rate, 54% good passes

(5% fail, 2% deferred, 1% X grade)• 88% agreed or strongly agreed with the statement: ‘I have received

helpful feedback that has enabled me to make progress.’Slide 14

EDU 2308 Education & The Social World

An experiment prompted by staff workshop on AL.Front-loading feedback:• Term 1 essay only given detailed formative feedback but

no summative grade.– On pressure from students, tutors gave general band

indicators for term 1 essays.• Term 2 essay given summative grade & minimal

feedback• Module pass rate = 91%, 55% good passes

(4% failed, 4% deferred)Slide 15

EDU3309 ‘The Child in Context’

Strategies: •Seminar on Level 6 Module Level Descriptors – unpacking the language and expectations of Level 6 outcomes

•Student marking sample essays with marking criteria followed by discussion of grades with discussion

•2014 - 77% pass rate, 55% ‘good’ grades 1-8•2015 - 92% pass rate, 68% ‘good’ grades

Slide 16

Assessment Literacy Matrix

• The Team have developed an Assessment Literacy Matrix

• In keeping with the tenets of assessment to examine where different strategies are being used and to ensure that they are not being overused

• Realisation that over familiarity may lead to loss of effectiveness and engagement

Slide 17

Summary of Activities to Promote Student A.L.

• Student Activity– Use of exemplars / grading exemplars against assessment criteria– Use of previous cohort generic feedback, or individual examples of

previous task feedback for students to identify strengths and potential pitfalls in completing assignment

– Peer evaluation and feedback– Self assessment– Tasks that require engagement with previous feedback

• Staff Activity– Assignment preparation for students– Formative feedback on drafts or plans– Informative and timely feedback with feed-forward for future work– Planning assessment tasks that build on previous assessment experience

(assessment mapping)Slide 18

Summary of Ways to Promote Staff A.L.

• Staff ownership of assessment design in relation to learning outcomes

• Collaborative staff-writing of assessment criteria• Assessment marking workshops on sample work ->

exemplars of internal standards• Explicit + and – feedback on student work to justify grade

awarded -> transparency • Early moderation / double marking within the marking cycle• Awareness of intra-marker and inter-marker reliability

issues

Slide 19

Going forward…

• Within each module – how is assessment an integrated part of the learning and how is it built towards? [assessment planning; active engagement with assessment standards]

• Within each module – where is the opportunity to engage students in a meaningful feedback process? [feedback timelines]

• Across the programme – how are assessments and feedback opportunities related to each other across modules? [assessment mapping]

• What are the ‘crisis’ moments for students and how can assessment literacy be embedded to ease this?

Slide 20

Questions that were raised:

• Can you get enough information from a 10 minute presentation to determine learning outcomes at Level 5?

• Can you get as much from a 1500 word essay as you can from a 3,000 word essay?

• How much support is too much support?

Slide 21

References

Ball, Bew, Bloxham et al 2013 ‘A Marked Improvement: Transforming Assessment in Higher Education’ HEAhttps://www.heacademy.ac.uk/node/3950

O’Donovan, B., Price, M., Rust, C. (2008) ‘Developing Student Understanding of Assessment Standards: A Nested Hierarchy of Approaches’ Teaching in Higher Education 12/ 2: 205-217

Price, M., Rust, C., Donovan, B., Handley, K., & Bryant, R. (2012) Assessment Literacy : The foundation for improving student learning Oxford: ASKE

15/04/23Slide 22

• Thank you for listening

• Any Questions?

15/04/23Slide 23