Results-Based Accountability (RBA) The Fiscal Policy Studies Institute Coreassets.com Book - DVD Orders amazon.com

Results-BasedAccountability (RBA)

The Fiscal Policy Studies Institutewww.raguide.org www.resultsaccountability.com

Coreassets.com

Book - DVD Ordersamazon.com

resultsleadership.org

Atebolrwyddyn ôl Canlyniadau

(Canlyniadau'n Cyfrif)



The Fiscal Policy Studies Institutewww.raguide.org www.resultsaccountability.com

Coreassets.com

SIMPLE

COMMON SENSE

PLAIN LANGUAGE

MINIMUM PAPER

USEFUL

Results Based Accountabilityis made up of two parts:

Performance Accountabilityabout the well-being of

CUSTOMER POPULATIONSFor Programs – Agencies – and Service Systems

Population Accountabilityabout the well-being of

WHOLE POPULATIONSFor Communities – Cities – Counties – States - Nations

THE LANGUAGE TRAPToo many terms. Too few definitions. Too little discipline

Benchmark

Target

Indicator Goal

Result

Objective

Outcome

Measure

Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic

Lewis Carroll Center for Language DisordersMeasurable urgent systemic indicatorsCore qualitative strategic objectivesYour made up jargon here

Y Trap IeithyddolGormod o dermau, Diffyg diffiniadau, Diffyg disgyblaeth

MeincnodMeincnod

TargedTarged

DangosyddDangosydd NodNod

CanlyniadCanlyniad

AmcanAmcan

DeilliantDeilliant

MesurMesur

AddasyddionAddasyddion MesuradwyMesuradwy CraiddCraidd

PwysigPwysig AnsoddolAnsoddol BlaenoriaethBlaenoriaeth RhaglennolRhaglennol Wedi’u targeduWedi’u targedu PerfformiadPerfformiad

CynyddolCynyddol StrategolStrategol SystemigSystemig

DEFINITIONS

Healthy children, Young people achieve their potential, Safe communities, Clean Environment, Prosperous Economy

Rate of low-birthweight babies, Percent with 5 A-C GCSE's, crime rate, greenhouse gas emissions, employment rate

1. How much did we do? 2. How well did we do it?

3. Is anyone better off?

RESULT or OUTCOME or GOAL

INDICATOR or BENCHMARK

PERFORMANCE MEASURE

A condition of well-being for children, adults, families or communities.

A measure which helps quantify the achievement of a result.

A measure of how well a program, agency or service system is working. Three types:

= Customer Results

Popu

latio

nPe

rfor

man

ce

Healthy children

Rate of low-birthweight babies Percent with 5 A-C GCSE's

Young people achieve their potential

crime rate

Safe communities

greenhouse gas emissions

Clean Environment

employment rate

Prosperous Economy

INDICATOR

RESULT

PERFORMANCE MEASURE

Popu

latio

nPe

rfor

man

ce

= Customer Results

GOAL

From Ends to Means

ENDS

MEANS

From Talk to ActionPo

pula

tion

Perfo

rman

ce

RESULT

INDICATOR

PERFORMANCEMEASURE

Customer result = EndsService delivery = Means

From Talk to Action

1. Safe Community

2. Crime Rate

3. Average Police Dept response time

4. A Wales of vibrant culture & thriving Welsh language

5. % of people able to speak, read and write Welsh

6. People have living wage jobs and income

7. % of people with living wage jobs and income

8. % of participants in job training who get living wage jobs

RESULT, INDICATOR or PERFORMANCE MEASURE?

RESULT

INDICATOR

PERF. MEASURE

RESULT

INDICATOR

RESULT

INDICATOR

PERF. MEASURE

POPULATIONACCOUNTABILITY

Community Outcomesfor Christchurch, NZ

● A Safe City

● A City of Inclusive and Diverse Communities

● A City of People who Value and Protect the Natural Environment

● A Well-Governed City

● A Prosperous City

● A Healthy City

● A City for Recreation, Fun and Creativity

● A City of Lifelong Learning

● An Attractive and Well-Designed City

City of London Ontario, Canada

Community Results

● A Strong Economy

● A Vibrant Diverse Community

● A Green and Growing City

● A Reliable Infrastructure

● A Safe City

Source: Draft Strategic Plan Sept 2011

CARDIFF, WALESCommunity Outcomes

1. People in Cardiff are healthy

2. Cardiff has a clean attractive and sustainable environment

3. People in Cardiff are safe and feel safe

4. Cardiff has a thriving and prosperous economy

5. People in Cardiff achieve their full potential.

6. Cardiff is a great place to live, work and play.

7. Cardiff has a fair, just and inclusive society.

WALESWell-being of Future Generations Bill

NATIONAL GOALS

● A prosperous Wales● A resilient Wales● A healthier Wales

● A Wales of cohesive communities● A Wales of vibrant culture and thriving Welsh language

● A more equal Wales

New Zealand

Colin Area, Belfast NI Cardiff, Wales

Santa Cruz County

Leaking Roof(Results thinking in everyday life)

Experience

Measure

Story behind the baseline (causes)

Partners

What Works

Action Plan

Inches of WaterBASELINE

? Fixed

Not OK

Turning the Curve

Action Plan # 2

Results / Outcomes / Goals

Indicators

Baseline & Story

Experience

Criteria for

Choosing Indicatorsas Primary vs. Secondary Measures

Communication Power

Proxy Power

Data Power

Does the indicator communicate to a broad range of audiences?

Does the indicator say something of central importance about the result?

Does the indicator bring along the data HERD?

Quality data available on a timely basis.

Choosing IndicatorsWorksheet

Outcome or Result_______________________

Candidate Indicators CommunicationPower

ProxyPower

DataPower

H M L

H

Measure 1

Measure 2

Measure 3

Measure 4

Measure 5

Measure 6

Measure 7

Measure 8

HDataDevelopmentAgenda

Safe Community

H M L H M L

H H

H L

Three Part Indicator List for each Result

Part 1: Primary Indicators

Part 2: Secondary Indicators

Part 3: Data Development Agenda

● 3 to 5 “Headline” Indicators● What this result “means” to the community● Meets the Public Square Test

● Everything else that’s any good (Nothing is wasted.)● Used later in the Story behind the Curve

● New data● Data in need of repair (quality,timeliness etc.)

The Matter of Baselines

Baselines have two parts: history and forecast

H

M

L

History Forecast

Turning the CurvePoint to Point

OK?

1975

1980

1982

1990

2000

2005

0

5

10

15

20

25

30

Tho

usan

ds

Source 1982 to 2005: Actual data from the NHTSA Fatality Analysis Reporting System (FARS)Source 1975 to 1981: Estimate based on NHTSA data on % of fatality drivers with BAC of .10 or greater.

Alcohol-Related Traffic FatalitiesUnited States 1975 to 2005

45 people per day

75 people per day

2010

28 peopleper day

MADD

Rebound

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

20.00%

Ncle 14.5 14.5 16.8 14.5 17 15 11.9 10.6 9.5 9.3

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

Source: Connexions Tyne and Wear, UK

Newcastle, UK

Revised 9 Nov 2007

Nov 08 – Jan 09

8.5

Hull, UK

- Rosell

“If I include you,you will be my partner.

If I exclude you,you will be my judge.”

PERFORMANCEACCOUNTABILITY

“All performance measures

that have ever existed

for any program

in the history of the universe

involve answering two sets of

interlocking questions.”

HowMuchdid we do?

( # )

HowWell

did we do it?

( % )

Quantity Quality

Performance Measures

EffortHow hard did we try?

EffectIs anyone better off?


Effort

Effect

HowMuch

HowWell


How much did we do?

Program Performance Measures

How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

How much did we do?

Education

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Number of 16 olds with 5 A to C

GCSE’s

Number with goodschool attendance

Percent of 16 yr olds with 5 A to C

GCSE’s

Percent with goodschool attendance

How much did we do?

Pediatric Practice



Number ofchildrentreated

Percent ofpatients waiting

less than30 min in thewaiting room

Quantity Quality

Effe

ct

E

ffort

#children

fullyimmunized

(in the practice)

%children

fullyimmunized

(in the practice)

How much did we do?

Drug/Alcohol Treatment Program



Number ofpersonstreated

Percent ofstaff withtraining/

certification

Number of clientsoff of alcohol & drugs - at exit - 12 months after exit

Percent of clientsoff of alcohol & drugs - at exit - 12 months after exit

Quantity Quality

Effe

ct

E

ffort

How much did we do?

Waste Management Services



Quantity Quality

Effe

ct

E

ffort

# tonnes of residential

waste

Unit costper tonnecollected

% to land fill

% diverted from landfill

#/amt to land fill

#/amt diverted from landfill

How much did we do?

Fire Department



Number ofresponses

ResponseTime

Quantity Quality

Effe

ct

E

ffort

# of fireskept to

room of origin

% of fireskept to

room of origin

How much did we do?

General Motors



# of production hrs

# tons of steel

Employees pervehicle

produced

# of cars sold

$ Amount of Profit

$ Car value after 2 years

Quantity Quality

Effe

ct

E

ffort

Source: USA Today 9/28/98

% Market share

Profit per share

% Car value after 2 years

How much did we do?

Not All Performance Measures Are Created Equal



LeastImportant

Quantity Quality

Effe

ct

E

ffort

MostImportant

Least

Most

AlsoVery Important

Quality

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

1. Did we treat you well?

2. Did we help you with your problems?

*

Product Output Impact

RBA Categories Account for All Performance Measures(in the history of the universe)

* World’s simplest completecustomer satisfaction survey

Total Quality Mgmt (TQM)

How much did we do?

The Matter of Control



Quantity Quality

Effe

ct

E

ffort

LeastControl

PARTNERSHIPS

MostControl

Comparing Performance

Your Baseline

Comparison Baseline

Goal (line)

Target or Standard

Instead:Count anything better than baseline as progress.

Avoid publicly declaringtargets by year if possible.

●

Create targets only when they are:

FAIR & USEFUL

x

AspirationNOT

Punishment

How much did we do?

Performance AccountabilityTypes of Measures found in each Quadrant



# Clients/customers served

# Activities (by type of activity)

% Common measurese.g. client staff ratio, workload ratio, staffturnover rate, staff morale, % staff fully trained, % clients seen in their own language,worker safety, unit cost

% Skills / Knowledge (e.g. parenting skills)

#

% Attitude / Opinion (e.g. toward drugs)

#

% Behavior (e.g.school attendance)

#

% Circumstance (e.g. working, in stable housing)

#

% Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Point in Timevs.

2 Point Comparison

# %

How much did we do?

Service: __________________________________



Primary customers# studentspatientspersons trained

Primary activity# hours of

instructiondiagnostic testsjob coursesalarms responded to

If your service works really well,how are your customer's better off?

# students who graduatepatients who fully recoverfires kept to room of origin

%

Unit cost

Workload ratio

% of ___x___ that happen on time

persons who get jobs

School Hospital Job Training Fire Department

How much did we do?

Choosing Headline Measures and the Data Development Agenda



Quantity Quality

Ef

fect

Effo

rt

# Measure 1 ----------------------------

# Measure 2 ----------------------------

# Measure 3 ----------------------------

# Measure 4 ----------------------------

# Measure 5 ----------------------------

# Measure 6 ----------------------------

# Measure 7 ----------------------------

#1 Headline

#2 Headline

#3 Headline

#1 DDA

#2 DDA

#3 DDA% Measure 8 ----------------------------

% Measure 9 -----------------------------

% Measure 10 ---------------------------

% Measure 11 ---------------------------

% Measure 12 ---------------------------

% Measure 13 ---------------------------

% Measure 14 ---------------------------

# Measure 15 ----------------------------

# Measure 16 ----------------------------

# Measure 17 ----------------------------

# Measure 18 ----------------------------

# Measure 19 ----------------------------

# Measure 20 ----------------------------

# Measure 21 ----------------------------

% Measure 15 ----------------------------

% Measure 16 ----------------------------

% Measure 17 ----------------------------

% Measure 18 ----------------------------

% Measure 19 ----------------------------

% Measure 20 ----------------------------

% Measure 21 ----------------------------

Select 3 to 5 Performance MeasuresACROSS THE BOTTOM OF THE ORG CHART

3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 3 - 53 - 5

3 - 5 3 - 5 3 - 5

3 - 5

20 – 60 – 20 Rule

LR

UR

Primary v. Secondary

Direct v. Indirect

Internal v. External

Baseline & Story

Ruth Jordan

Watermain Breaks per yearLondon Ontario

Next GenerationContracting / Commissioning

How Population

&Performance Accountability

FIT TOGETHER

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE

POPULATION ACCOUNTABILITY

Healthy Births Rate of low birth-weight babiesChildren Ready for School Percent fully ready per K-entry assessmentSelf-sufficient Families Percent of parents earning a living wage

CUSTOMEROUTCOMES

# personsreceivingtraining

Unit costper person

trained

# who getliving wage jobs

% who getliving wage jobs

PERFORMANCE ACCOUNTABILITY

POPULATIONOUTCOMES

Job Training Program

Every timeyou present

your program,

Use atwo-part

approach.

Result: to which you contribute to most directly.

Indicators:

Story:Partners:What would it take?:

Your Role: as part of a larger strategy.

Population Accountability

Program:Performance measures:

Story:Partners:Action plan to get better:

Performance Accountability

Your Role

Every timeyou present

your program,

Use atwo-part

approach.

Result: to which you contribute to most directly.

Indicators:

Story:Partners:What would it take?:

Your Role: as part of a larger strategy.

Population Accountability

Program:Performance measures:

Story:Partners:Action plan to get better:

Performance Accountability

Your Role

Shortcut

Different Kinds of Progress

1. Data

a. Population indicators Actual turned curves: movement for the better away from the baseline.

b. Program performance measures: customer progress and better service: How much did we do?

How well did we do it? Is anyone better off?

2. Accomplishments: Positive activities, not included above.

3. Stories behind the statistics that show how individuals are better off.

Board of Directors MeetingAGENDA

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

SUMMARY

3 - kinds of performance measures. How much did we do? How well did we do it? Is anyone better off?

RBA in a Nutshell2 – 3 - 7

2 - kinds of accountability Population accountability Performance accountability

7 - questions from ends to means in less than an hour.

plus language discipline Results & Indicators Performance measures

Baselines & Turning the Curve

THANK YOU !



DIOLCH !



Turn the Curve Exercise

Creating a Working Baselinefrom Group Knowledge

Now

Indicator or Performance Measure

65% Not OK?

BackcastingForecasting

Turn the Curve Exercise: Population Well-being

5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s)

10 min: Baseline - pick a result and a curve to turn - forecast (to 2017) – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report convert notes to one page

Two pointers to action

ONE PAGE Turn the Curve Report: Population

Result: _______________Indicator

(Lay Definition)IndicatorBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

SharpEdges

4. --------- Off the Wall


Turn the Curve Exercise: Program Performance

5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s)

10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast (to 2017) – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report convert notes to one page

Two pointers to action

Program: _______________Performance Measure

(Lay definition)PerformanceMeasureBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

ONE PAGE Turn the Curve Report: Performance

SharpEdges



Turn the Curve Exercise – LessonsTalk to Action in an hour

1. How was this different from other processes? What worked and what didn’t work?

2. Why did we ask for:a. Results before indicators?b. Forecast?c. Story? d. No cost / low cost?e. Two hats?f. Crazy idea?g. Only 3 best ideas?

3. Do you think a lay audience could understand the reports?

4. How many think you could lead this exercise with a small group? (2+ curves at the same time)

Next Steps:

What’s one thing I could personally do with what I learned today?

Resources



raguide.orgresultsaccountability.com/publications

RBA Facebook GroupCoreassets.com

(work in Merthyr Tydfil and 3 other local counciils)

DIOLCH !



Documents

Results-Based Accountability (RBA) The Fiscal Policy Studies Institute Coreassets.com Book - DVD Orders amazon.com