計劃評估對服務學習發展的重要性－ 香港紅十字會國際人道服務領袖培育計劃經驗分享 Experience sharing on the importance of service evaluation for Service Learning – International humanitarian service leadership programme (ISLP) illustrated

梁世傑Leung Sai Kit, 青年及義工事務高級主任Senior Staff Officer 9 April 2016

Introduction of ISLP

• International humanitarian service leadership programme (ISLP)

• Starts at 2005 • Included to Youth Leaders Development programme

in 2012 • Expand RCY spectrum to tertiary education sector

Primary/ Secondary School RCY Members

Tertiary student/ Future RCY Leaders

Rationale and Core Value

• Networking with overseas and mainland RC society, and local tertiary education institutions

• Platform for youth to actualize humanitarian service

• Cultivation of humanitarian value & volunteerism

Humanitarian Voluntary

Service

Humanitarian Leadership Qualities

Humanitarian Global Vision

人道國際視野 人道志願服務

人道領袖素質

How to deliver?

• Overseas care and concern action

• 800 participants • 12 teams • 8 countries • Over 70 overseas trips • Over 3,000 vulnerable served • Reached over 3000 university students and public every year

How to deliver?

• ISLP workshop • Local dissemination events • Publications

Programme Evaluation in 2012 – why?

• After 10 years of operation: – Competition from others organizations – Clearing the Future direction and Positioning of

the project, while refocusing of services departments at the same time

– Exploring different mode of operation • Funding application:

– CC Internal funding Regular Service

Evaluation Process – Market analysis

• Market research/ analysis done by staff and stakeholders – SWOT analysis – Studies on other service

learning programmes in HK

Strengths Opportunities

Weaknesses Threats

Comparison on Experiential Learning Programmes in HK Project Characteristic

HKRC ISLP Oxfam HK Oxfam Club

HKIEd LEAD AIESEC

Concept involved Humanitarian Global citizenship Community service Global impact

Choice of location Free choice under HKRC network

Fixed by the organiser

Fixed by the organiser

Choices under AIESEC network

Self-planning of programme by the participants

Highly required Planning is mostly being done by the organiser

Planning is mostly being done by the organiser

Self-planning under a framework of preset workdone

Leadership quality development

Highly emphasised Highly emphasised Highly emphasised Highly emphasised

Financial support Minimal support by HKRC; supported by institution or self-finance

Supported by Oxfam Hong Kong

Supported by HKIEd, except for the flight

Self-financed by the participant

Expansion in other programmes’ involvement

Can participate in other HKRC events

Can participate in other Oxfam Club events

An isolate project Can participate in other AIESEC events

Evaluation Process – Programme Evaluation

• Done by Dr CK Law and his team • Evaluation design based on 3 levels of capacity

building Programme Level

•Relevance •Effectiveness •Efficiency

Community Interface Level

•Collaboration •Community Involvement

Development Level

•Impact •Sustainability

Evaluation Process – Programme Evaluation

Methodology • Documentary analysis – study

all proposals, reports, publications and journals on ISLP

• Focus Group Interview & Structure interview– mentors, participants with different levels of participations, staff members and working partners

Evaluation Process – Programme Evaluation Levels Strengths Weaknesses

Programme operational level

1. Participants were able to acquire new knowledge and skills on programme planning and humanitarian concepts

2. Provide a platform to actualise the creativity of the participants

3. Mentors provide important support and guidance to the teams as well as lessen the workload of the staff

1. Humanitarian concept being introduced is being too general and difficult to be consolidated

2. HKRC has a lot of internal regulations which could be seen as hindrance to programme planning

3. Support from HKRC was insufficient especially on financial resources and skills and knowledge transfer

4. Limited avenue for skills and knowledge exchange and communication among teams

5. Outcome of each team varied due to limited supervision

Evaluation Process – Programme Evaluation Levels Strengths Weaknesses

Community interface level

1. Working partners understand well and agreed with the ideology of ISLP

2. Trainings provided by HKRC are more resourceful when compared with other overseas study tour organised by other NGOs

3. When the institution values the opportunities provided by ISLP, more highly-motivated students could be recruited

1. Insufficient communication between HKRC and working partners which caused the working partners could not keep track the participants’ work

2. HKRC staff does not understand well the internal administrative procedure of the institutions

3. Recruitment would be difficult if the institution does not emphasise ISLP

4. Limited follow-up actions in Hong Kong have been carried out due to the time limitation and insufficient skills and knowledge to consolidate the experiences of the participants

Evaluation Process – Programme Evaluation Levels Strengths Weaknesses

Developmental level

1. Strengthening humanitarian concept

2. Increase the chance of the participant in future engagement in humanitarian and HKRC work

3. Leadership quality and social understandings could facilitate participants with their outside work

4. Financial stability could be maintained internally if no outside funding could be sought

1. Difficult to expand the service scope due to limitation of participants’ knowledge and skills; and reluctance from overseas partners in making special arrangements

2. Demanding workload difficult to recruit new members under the demanding education environment

Evaluation Process – Programme Evaluation

Result: • Regular programme evaluation means for ISLP

developed for future use • Regular funding for HKRC

Evaluation Process – Programme Evaluation

Result – by-product: • A good chance to communication with high level

management for their idea on YOUTH and future development of the project

• An educational process for participants – their mindset to include evaluation in their projects

• Enhancement and development of the evaluation protocols (observations, focus groups, different levels of participation of volunteers)

Difficulties and Solutions

• Impact of the project on individual/ to society is difficult to measure/ quantify

• $/ Human Resources/ time/ workload!!!

• Difficult to translate findings to practical proposal/ programme means

• Managing expectations

• Develop agency’s own service development and evaluation standard, esp on value education

• Agreed framework - Involve management and other stakeholders at initial stage, include their opinion

• Include research budget and human resource when planning time based projects

• Balance and pragmatic

Thank You

Q & A session