Effective evaluation

Rob RoffeSenior Policy and Learning Manager09/12/2012

Objectives

By the end of the workshop we will have:

─ Established the importance of evaluation

─ Considered the process of evaluation

─ Looked at some effective evaluation tools and approaches

─ Discussed issues and challenges associated with evaluation.

Why evaluate?

Evaluating your project has many benefits:

─ Learn as you go: improve project performance

─ Opportunity for beneficiaries to provide feedback

─ Assess progress against your anticipated outcomes

The process of evaluation

Evaluating your project comes down to answering three basic questions:

─ QUANTITY: How much did it do?

─ QUALITY: How well did it do it?

─ EFFECT: What difference did it make?

To answer these you need performance measures

Performance measure examples

Question Performance measures

How much did the project achieve?

Number of older people (beneficiaries) supportedAmount of time spent supporting older peopleNumber of group based sessions with older people

How well did the project do it?

Proportion of older people satisfied with the serviceProportion of older people saying they found the service usefulProportion of older people saying they found the service relevant

What difference did it make?

Proportion of older people more socially involved (e.g. taking part in at least one group activity a week)Proportion of older people feeling less lonelyProportion of older people feeling that they have achieved change in relation to a service they receive

Performance indicators (KPIs)

KPIs are important because:

─ You need them to inform your performance measures

─ They are used to evaluate the success of a particular activity

─ Choosing indicators is reliant on good understanding of what’s important and knowing what kind of information people will be interested in when you share your findings.

Different types of KPIs

─ Quality and quantity KPIs – the difference between qualitative and quantative

─ Outcome KPIs

─ Population KPIs

Establishing a baseline

A baseline consists of some indicators that define the conditions of a community area or group of people before your project is delivered

─ May refer specifically to the beneficiaries of a project or more generally to the group of people being targeted by a project

Evaluation: issues and challenges