Indicators, Evaluation and Performance Budgeting

Performance Budgeting Course绩效预算课程

AFDC, Shanghai, October 2010

Prepared by Marc Robinson

指标、评价和绩效预算

评价与结果学习中心

Key performance concepts 关于绩效的几个重要概念◦ Outcomes, outputs, activities/processes and inputs 结果、产出、活动 / 过程和投入

Performance indicators 绩效指标 Evaluation 评价

This session 本节内容

Performance Concepts绩效相关的几个概念

Desired impacts of services: 服务的预期影响◦ Upon the individuals, social structures, or physical environment◦ 对个人、社会结构或自然环境的预期影响

Examples: 例子◦ Improved health of patients 病人健康状况的改善◦ Reduced crime 犯罪的减少◦ Conservation of natural environment 对自然环境的保护

Effectiveness an outcome concept 一个结果概念—有效性 High-level vs Intermediate outcomes 高阶结果与中间结果 Impacts sometimes used for high level outcomes 影响有时是指高阶结果

Outcomes

Output Intermediate Outcome High Level Outcome

结果

产出 中间结果 高阶结果

Outputs = products 产出 = 产品◦ But in govt, most are services, not goods 但在政府部门，产出大多是服务，而非商品

Services to external client or subject 提供给外部客户或对象的服务 Examples of outputs: 有关产出的例子

◦ A school student who receives teaching 受教育的学生◦ A patients treated in a public hospital 在公立医院接受治疗的病人◦ Benefit payment administered 管理的保险费◦ Enforcement of legislation 法律的强制执行

Quantity, quality & efficiency measures 数量、质量和效率衡量 Outcomes & outputs = “results” 结果 + 产出 = 成效

Outputs 产出

Resources used to produce outputs 用于产出的资源◦ Human & organizational resources 人力资源和组织资源◦ Physical assets 实物资产◦ Materials and other inputs 原料投入和其他投入◦ Some purchased, some created internally 有些是购买的，有些是内部生成的

Workforce indicators 员工指标◦ Training conducted, staff turnover etc 员工培训、员工流失率

Asset indicators 资产指标◦ School classrooms per school-age child 学校教室 / 学龄儿童

Inputs 投入

Activities produce outputs 活动实现产出◦ Support activities – human resources, IT ... 支援活动—人力资源、 IT 活动◦ Direct service activities – nursing 直接服务活动—护理◦ Processes: sequence of activities 过程：活动的顺序

Activity indicators 活动指标◦ Job vacancies filled 填补的空缺职位◦ Pension applications processed 处理的养老金申请

Activities are not outputs 活动不是产出◦ Not capable alone of delivering outcome 不能独自产生结果的活动

Quantity, quality and efficiency measures 数量、质量及效率的衡量

Activities and processes活动和过程

The Results Chain

INPUTS

ACTIVITIES/ PROCESSES

OUTPUTS

IntermediateOUTCOMES

High-levelOUTCOMES

结果链

投入

高阶结果

中间结果

活动 / 过程

产出

Results Chain Example

Output (service)

Intermediate outcome

Intermediate outcome

Intermediate outcome

OUTCOME

Awareness on health dangers of smoking

People view smoking negatively

Fewer people take up smoking

Fewer people smoking

Fewer people develop smoking-related diseases结果

中间结果

中间结果

中间结果

产出（服务）

因吸烟患病的人越来越少

吸烟的人越来越少

开始吸烟的人越来越少

人们对吸烟持负面观点

意识到吸烟对健康有危害

结果链举例

Outcomes and outputs delivered to external parties 提供给外部当事人的结果和产出 For individual ministries, this includes services to rest of government

对单个政府部门而言，这包括对其余政府部门的服务Not just to the community 不是仅对本团体而言

Civil service ministry example: 例如公务员局◦ Output: recruitment of civil servants 产出：公务员的招募◦ Outcomes: a better quality civil service, improved governance 结果：更好的公共服务，更好的政府管理

Ministry perspective 政府部门视角

What are the intended outcomes of: 预期的结果是什么 Anti-pollution programs? 污染治理项目？ School education? 学校教育？

Outcomes: some questions …

结果：一些问题……

Patient given anesthetic before operation 手术前麻醉的病人◦ Is an output provided? 提供产出了吗？

Roads and bridges 道路及桥梁◦ Are they outputs? If not, what is? 它们是产出吗？如果不是，那什么才是产出？

Educational output: the best measure is: 教育产出：最好的衡量方法是：◦ Teaching hours? Numbers of students taught? 教学时间？受教育学生的数量？

Outputs: some questions …

产出：一些问题……

Performance Measures 绩效测量方法◦ Quantitative measures of performance 定量的绩效衡量方法◦ Performance indicators the same thing 同样的绩效指标

Evaluation 评价◦ Retrospective analysis of performance 绩效的回顾性分析◦ Interpretive, not merely measures 解释，而不仅仅是测量方法◦ Can use, for example, relevant theory 例如，能使用相关定理

Performance budgeting requires both 绩效预算时 , 两者都需要

Performance Information 绩效信息

Performance Indicators绩效指标

Performance measure: 绩效测量方法◦ Survival rate of lung cancer patients 2 year after diagnosis 肺癌确诊 2 年后病人的存活率

Objective 目的◦ Reduce death and disability arising from lung cancer 降低肺癌引起的死亡和残疾率

Target 目标◦ Increase the 2 year survival rate by at least one-third within the

next decade 在未来十年内将确诊 2 年的肺癌病人的存活率提高至少三分之一

Measures vs. Objectives & Targets衡量方法与目的和目标

Clearly linked to objectives 与目的有清晰的联系 Comprehensible and unambiguous 容易理解而且非常清晰 Reliable 可靠性 Timeliness 及时性 Perverse effects minimized 负面影响最小 Cost effective 低成本

Criteria for good indicators有效指标的标准

The more PMs matter, the more they: 绩效管理越重要，他们就◦ May be manipulated 越有可能被操控◦ May lead to perverse results 越有可能导致负面结果

Dumping, creaming, storming etc 倾销、撇脂、震荡等

Will explore throughout workshop 将通过专题研讨会探索◦ Particularly in session on Targets 尤其是在目标这个阶段

Design should aim to minimize this 设计的目的应该是尽量减少负面效果 But can never eliminate

但不可能永远将其消除◦ Consequence of inherent limitations of PMs 是由绩效管理的内在缺陷导致的

Perverse Effects 负面效果

Too much PI can be a problem 绩效信息太多可能会带来麻烦◦ For budget decision-makers 对预算决策者来说

Limited, strategic information 有限的战略信息 Results indicators are primarily relevant for budget decisions 结果指标主要与预算决策紧密相关

◦ Outcomes, outputs … 结果、产出……◦ Activity, input indicators primarily of internal relevance 活动与投入指标具有内在关联性

Measures presented in context 具体情况中的测量方法◦ Description of program performance 对项目绩效的描述◦ Drawing on evaluations etc 对评价等方面的描述◦ One page or so per program 每个项目一页纸左右

Usable Performance Information可用的绩效信息

Evaluation and Performance Budgeting

评价和绩效预算

Role of evaluation under program budgeting: 项目预算下评估的作用 Expenditure prioritization 支出的优先顺序

◦ Which programs which should be cut back or eliminated? 哪些项目应该减支或淘汰？ Pressure on agencies to perform 对执行机构的压力

◦ By reviewing their budget requests in the light of their past performance in effectively using resources 根据机构过去实际使用资源的绩效来审查其预算请求

Role in budget-linked performance targets system 预算相关的绩效目标体系的作用◦ Assess how far the agency is responsible for any failure to meet target 对机构未实现目标而评估其应该负多少责任◦ E.g. versus impact of external factors 例如，有多少是外部因素影响而导致的

Performance Budgeting & Evaluation 绩效预算与评价

Outcome evaluations most relevant 最相关的结果评价◦ Ineffective programs are key candidates for cuts 没有产生预期效果的项目属于应该进行削减的首选◦ Agencies with many ineffective programs should probably not be given more

money 如果一个机构承担的项目很多没有产生预期效果，那么这个机构或许不应该得到更多的经费

Process evaluations relevant to more limited degree 有限相关的过程评价◦ Before cutting a program which is ineffective 在削减没有产生预期效果的项目之前◦ Need to know whether could be effective if redesigned or better managed 需要知道重新设计项目或更好地加以管理是否能使项目有效◦ But detail of process improvements not relevant for PB 但是对绩效预算来说，过程改善的细节是无关的◦ In-depth process evaluations more relevant for internal management purposes

对内部管理来说，深度过程评价更相关

Relevant types of evaluation评价的相关类型

Spending Review (SR) 支出审查◦ Systematic scrutiny of ongoing spending 对正在现有支出进行系统的审查◦ To identify cuts in low-priority & ineffective programs 确认哪些优先程度较低而且没有达到预期效果的项目需要削减◦ And savings from efficiency enhancement 因效率提高而产生的结余◦ Evaluation as key information input to SR 对作为信息关键而投入到支出审查中的评价

Not the only input 不是唯一的投入◦ Performance indicators 绩效指标◦ Analysis of program priorities (not part of evaluation) 项目优先级分析（不属于评价的内容）

Selection of programs to be evaluated 待评价项目的选择◦ SR process often includes selection of programs for evaluation 支出审查过程常常包括评价项目的选择◦ Programs suspected of being ineffective or low-priority 被认为效果小或优先级别较低的项目◦ Annual budget-related evaluation program 与年度预算相关的评价项目

Evaluations and Spending Review 评价和支出审查

Evaluations timing must fit in with budget preparation 评价的时机必须与预算的准备相一致 Means evaluations usually need to be quick 这就意味着评价通常要迅速

◦ E.g. commission at or before start of budget process 例如预算过程开始或之前的委托◦ Results before final budget decisions taken 作出预算决定之前的结果

Rapid evaluation is most relevant 快速评价是最相关的 Evaluation reports designed to tell budget decision-makers what they need

to know 评价报告旨在告诉预算决策者他们所需要知道的◦ How effective the program is 项目的有效性如何◦ Whether it is inherently ineffective 是否其内在因素决定了其无法达到预期效果◦ Or could be effective with better implementation 或者是可通过更好实施才能有效

Evaluation & the Budget 评价与预算

Relation of Evaluation to Budget Process

Spending Review

Budget Preparation Process

Budget-Linked Evaluations

Performance Indicators

Time时间

评价和预算过程间相关性

预算相关的评价

支出审查

预算编制过程

绩效指标

Purely internal evaluations not sufficient for budget 纯粹的内部评价对预算来说是不够的◦ Danger that these will be self-serving rather than objective 内部评价往往缺乏客观性，所以会导致为己牟利的危险◦ Many are detailed process evaluations 许多都是详细的过程评价

External evaluations needed 外部评价是必需的◦ Under MOF leadership 在财政部的领导下◦ Although ministry concerned may share direction 尽管相关部委可能参与指导

MOF will usually not carry out the evaluations itself 财政部通常不会自己进行评价 ◦ Commission evaluations from experts 由专家组成评估委员会◦ Retired senior civil servants can be good evaluation leaders 退休高级公务人员将是称职的评估小组领导人

Can’t necessarily expect ministry to “own” results 不一定指望政府部门“拥有”评价结果◦ But that’s not the primary objective of these evaluations 但是这不是这些评价的首要目标

Who conducts budget evaluations?谁是预算评价的实施者？

System of evaluation linked to budget since 1997 从 1997 年开始建立与预算相关的评价体系 Evaluations geared to budget preparation 面向预算编制的评价

◦ Started with desk (rapid) evaluations 从桌面 (快速 ) 评价开始◦ Later expanded to more detailed effectiveness evaluations 后来延伸到更详细的有效性评价◦ Desk evaluations still dominate (13 out of 17 conducted in 2006) 快速评价仍旧占主导地位（ 2006 年进行的 17项评价中快速评价就有 13项）◦ Evaluations delivered within 4-10 months 历时 4-10 个月的评价◦ Selection of programs for evaluation with budget in mind 根据预算仔细选择待评价的项目

Proven impact on budget funding decisions 对预算分配决策的实际影响 Finance ministry role 财政部的职责

◦ Key role in selecting programs for review 在选择审查项目时具有关键作用◦ Key role in selecting independent evaluators & managing evaluations 在选择独立评价人员和管理评价活动时具有关键作用

Chile 智利

Limited financial & skilled human resources 财力和技术人才的缺乏 Need to be very selective in performance information 需要精心选择绩效信息 Rapid evaluations should play dominant role 快速评价应该占据主要位置 Because low cost 原因在于成本较低 Availability of good performance indicators important 能否获得好的绩效指标是非常重要的

◦ Otherwise rapid evaluations have to rely entirely on evaluation of program logic 否则快速评价必须完全依赖项目逻辑性的评价

Evaluation capacity to be gradually built up 评价能力有待逐步建立

Evaluation and PB in Low Income Countries低收入国家的评价和绩效预算

Results chain 结果链◦ Clear framework for PI 绩效指标的清晰框架

Choosing right performance measures 选择正确的绩效衡量方法◦ For budget decision makers 对于预算决策者来说◦ Relevance, reliability etc 相关性、可靠性等◦ Not too many 不需要太多

Conclusions 总结