Audit kualiti dalam pendidikan tinggi (Quality Audit in Higher Education)

Audit kualiti dalam pendidikan tinggi(Quality Audit in Higher Education)

After completing this topic, you should be able to:Explain the meaning of quality auditDescribe the components of quality audit Learn the importance of quality audit in HEI

Quality AuditQuality Audit

Pendidikan tinggi merupakan pemangkin utama penjanaan sumber tenaga manusia.

Pelan Strategik Pendidikan Tinggi Negara menggariskan 7 tuntutan cabaran pendidikan tinggi:Peranan universiti dan ahli akademikPerkembangan kurikulum mengikut keperluan pasaranPenyelidikan, pembangunan dan pengkomersialan dalam sistem inovasi kebangsaanKaedah pengajaran dan pembelajaranGlobalisasi dan piawaian melalui pemeringkatan dan penarafanPeluang guna tenaga


Senario pengajian tinggi negara

Universiti awam dikategorikan kepada tiga kumpulan iaitu universiti penyelidikan, universiti berfokus (teknikal, pendidikan, pengurusan dan pertahanan), universiti komprehensif.

Universiti penyelidikan:Ciri-ciri – bidang pengajian fokus kepada penyelidikan, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 50:50.UMUSMUPMUKM



Universiti komprehensif:Ciri-ciri – pelbagai bidang pengajian, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 70:30.UiTMUIAMUMSUnimas



Universiti berfokus: Ciri-ciri – bidang pengajian berfokus, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 70:30.UTMUUMUPSIUTHMUTeMUMPUMTUDMUMKUPNM



Audit kualiti dalam pendidikan tinggi(Quality Audit in Higher Education)

A Quality Audit has been defined as:A systematic and independent determination of whether: the planned arrangements are suitable to achieve the objectives; the actual quality activities conform to the planned arrangements; and the arrangements are being implemented effectively.(Australian Universities Quality Agency (AUQA), 2009)

Relevant web site: http://www.auqa.edu.au


Learning Outcomes(Hasil Pembelajaran)

MQA: competencies as to what a student will be able to do at the end of a period of study.

Eight domains:Knowledge (menguasai ilmu dalam bidang Pentadbiran Pendidikan) Practical skills (menguasai kemahiran praktikal/teknikal/psikomotor dalam bidang berkaitan) Social skills and responsibilities (mempunyai akauntabiliti dalam mentadbir dan mengurus organisasi dan sosial ) Values, attitudes and professionalism (mempunyai tahap profesionalisme, nilai, sikap dan tingkah laku beretika)


Learning Outcomes(Hasil Pembelajaran)

Continue:Communication, leadership and team skills (cekap berkomunikasi dan bekerja dalam pasukan) Problem solving and scientific skill (berupaya berfikir secara kritis dalam mencari penyelesaian kepada

masalah) Information management and lifelong learning skill (cekap mengurus maklumat dan berkebolehan belajar

sepanjang hayat) Managerial and entrepreneurial skills (mempunyai kemahiran mengurus dan keusahawanan)


KEMAHIRAN INSANIAH – SOFT SKILLSKEMAHIRAN INSANIAH – SOFT SKILLS

KI – kemahiran generik yang merentasi pelbagai domain pembelajaran merangkumi aspek keperibadian dan kemahiran berkumpulan

UPM: TUJUH ELEMEN KI



Kemahiran BerkomunikasiPemikiran Kritis dan Kemahiran Menyelesaikan MasalahKemahiran BerpasukanPembelajaran Berterusan dan Pengurusan MaklumatKemahiran KeusahawananKemoralan dan Etika ProfesionalKemahiran Kepemimpinan



Kemahiran Insaniah Mesti (KIM) – kemahiran yang mesti ada (must have)Kemahiran insaniah tambahan (KIT) – kemahiran tambahan yang baik jika ada (good to have).Contoh: Untuk Kemahiran BerkomunikasiKIM – Kebolehan menyampaikan idea dengan terang, berkesan dan dengan penuh keyakinan, secara lisan dan bertulisKIM – Kebolehan mengamalkan kemahiran mendengar yang aktif dan memberikan maklumbalasKIT – Kebolehan berkomunikasi dengan peserta komunikasi yang berlainan budaya



Contoh: Kemahiran Pembelajaran berterusanKIM – Kebolehan untuk mencari dan menguruskan maklumat yang relevan daripada pelbagai sumberKIM – Kebolehan untuk menerima idea baharu dan berkeupayaan untuk pembelajaran autonomiKIT – Kebolehan untuk mengembangkan minda ingin tahu dan dahagakan ilmu



Contoh: Kemahiran Kerja BerpasukanKIM – Kebolehan untuk membina hubungan yang baik, berinteraksi dengan orang lain dan bekerja secara efektif bersama mereka untuk mencapai objektif yang samaKIM – Kebolehan untuk memahami dan mengambil peranan bersilih ganti antara ketua kumpulan dan ahli kumpulanKIM – Kebolehan untuk mengenal dan menghormati sikap, kelakuan dan kepercayaan orang lain KIT – Kebolehan untuk memberikan sumbangan kepada perancangan dan menyelaraskan hasil usaha kumpulan


Dimensions of evaluation

AUQA methods of Evaluation: Evaluates institution quality assurance arrangements:ApproachDeployment ResultsImprovement

Does not impose an externally prescribed set of standards upon auditees, but uses it as primary starting point for audit of organization’s own objectives.


AUQA AUQA Objectives

AUQA has several objectives:Arrange and manage a system of periodic audits of the quality of academic activities, the quality assurance arrangements, institutional compliance to criteria set out in the National Protocols for Higher Education Approval Processes. Monitor, review, analyze and provide public reports on the quality of outcomes in Australian universities and higher education institutions.Arrange and manage a system of periodic audits of the quality assurance processes, procedures, and outcomes.Publicly report periodically on matters relating to quality assurance.


Approach

The trail from an organization’s mission, vision and values (the overall objectives). Audit questions include:What is this organization about?What outcome is it trying to achieve?What, if any, reference points are used in establishing the organization’s objectives? How does the organization plan to achieve its objectives?Does it understand its context and capabilities?Are the organization’s objectives set against appropriate benchmark?What risk management processes does it have in place?Is the approach aligned and communicated throughout the organization and more widely?


Deployment

The deployment dimension considers whether and how effectively, the approach is being put into effect. Audit questions include:Is the approach being deployed in the best possible manner? According to whom?What standards and benchmarks is the organization using to assess this?If the approach is not being deployed, why not, and how is this managed?Are staff appropriately trained and resources appropriately deployed, to fulfill the approach?


Results

The result dimension looks at results as a means of determining how ell the deployment is achieving the planned approach. Audit questions include:Is the organization achieving its intended objectives and outcomes?Does the organization understand why and how it achieved those particular results?How are the results reported and used within the organization?


Improvement

The improvement dimension focuses on whether the organization is actively and continuously engaged with understanding its performance in each of the A-D-R dimensions, and is using this understanding to bring about improvements . Audit questions include:Does the organization know how it can improve?How does it know this (e.g. through the use of external benchmarks)?How is it acting upon this knowledge?Does the organization have a sustained history of improvement?


The Practice in MalaysiaThe Practice in MalaysiaMQA – Malaysian Qualifications AgencyMQA – Malaysian Qualifications Agency

MQA established under the Malaysian Qualifications MQA established under the Malaysian Qualifications Agency Act 2007. It was the merging of LAN (Lembaga Agency Act 2007. It was the merging of LAN (Lembaga Akreditasi Negara) and QAD (Quality Assurance Division Akreditasi Negara) and QAD (Quality Assurance Division of the Ministry of Higher Education).of the Ministry of Higher Education).

MQF (Malaysian Qualifications Framework) was MQF (Malaysian Qualifications Framework) was developed to unify and harmonize all Malaysian developed to unify and harmonize all Malaysian qualifications.qualifications.

Code of Practice for Program Accreditation (COPPA) Code of Practice for Program Accreditation (COPPA) and Code of Practice for Institutional Audit (COPIA) are and Code of Practice for Institutional Audit (COPIA) are guidelines, standards and code of practice to ensure guidelines, standards and code of practice to ensure quality in the academic performance and institutional quality in the academic performance and institutional effectiveness.effectiveness.

The codes contain an overview of the Malaysian quality The codes contain an overview of the Malaysian quality assurance system for higher education. assurance system for higher education.


MQA – Malaysian Qualifications AgencyMQA – Malaysian Qualifications Agency definitionsdefinitions

Institutional Audit – an evaluation of an institution to Institutional Audit – an evaluation of an institution to determine whether it is achieving its mission and goals, determine whether it is achieving its mission and goals, to identify strengths and areas of concern, and to to identify strengths and areas of concern, and to enhance quality.enhance quality.

Internal Quality Audit – an evaluation conducted Internal Quality Audit – an evaluation conducted internally by a higher education provider (e.g. the internally by a higher education provider (e.g. the university) to determine whether it is achieving its university) to determine whether it is achieving its mission and goals; to identify strengths and areas of mission and goals; to identify strengths and areas of concern, and to enhance quality. It generates a Self-concern, and to enhance quality. It generates a Self-Review Portfolio.Review Portfolio.

Program Accreditation – an exercise to accredit the Program Accreditation – an exercise to accredit the teaching, learning and all other related activities to a teaching, learning and all other related activities to a program provided by a higher education provider. program provided by a higher education provider.


MQA – Malaysian Qualifications Agency definitions

Quality Assurance – comprises all those planned and Quality Assurance – comprises all those planned and systematic actions (policies, strategies, attitudes, systematic actions (policies, strategies, attitudes, procedures and activities) necessary to provide procedures and activities) necessary to provide adequate demonstration that quality is being maintained adequate demonstration that quality is being maintained and enhanced, and the products and services meet the and enhanced, and the products and services meet the specified quality standards. In higher education, quality specified quality standards. In higher education, quality assurance is the totality of systems, resources and assurance is the totality of systems, resources and information devoted to maintaining and improving the information devoted to maintaining and improving the quality and standards of teaching, scholarship and quality and standards of teaching, scholarship and research as well as students’ learning experience.research as well as students’ learning experience.


MQA – 7 responsibilities

To implement the MQF as a reference point for the To implement the MQF as a reference point for the Malaysian qualifications.Malaysian qualifications.

To develop standards and criteria and all other relevant To develop standards and criteria and all other relevant instruments as a national references for the conferment of instruments as a national references for the conferment of awards with the co-operations of stakeholders.awards with the co-operations of stakeholders.

To quality assure higher education providers and To quality assure higher education providers and programs.programs.

To accredit programs that fulfill a set of criteria and To accredit programs that fulfill a set of criteria and standards.standards.

To facilitate the recognition and articulation of qualificationsTo facilitate the recognition and articulation of qualifications To establish and maintain the Malaysian Qualifications To establish and maintain the Malaysian Qualifications

Register (MQR).Register (MQR). To advise the Minister on matters relating to quality To advise the Minister on matters relating to quality

assurance in higher education. assurance in higher education.


MQA Council

MQA is headed by a Council comprising a Chairman and 16 MQA is headed by a Council comprising a Chairman and 16 members. The functions of the Council are to:members. The functions of the Council are to:

Approve plans and policies for the management of the Approve plans and policies for the management of the Agency.Agency.

Approve any amendment or update of the MQF.Approve any amendment or update of the MQF. Approve policies and guidelines relating to audit processes Approve policies and guidelines relating to audit processes

and accreditation of programs, qualifications and higher and accreditation of programs, qualifications and higher education providers.education providers.

Receive and monitor reports and other information relating Receive and monitor reports and other information relating to accreditation, institutional audit and evaluation.to accreditation, institutional audit and evaluation.

Continuously guide the Agency in its function as a quality Continuously guide the Agency in its function as a quality assurance body.assurance body.


Malaysian Qualifications Framework - MQF

The MQF is an instrument that classifies qualifications The MQF is an instrument that classifies qualifications based on a set of criteria that are approved nationally and based on a set of criteria that are approved nationally and benchmarked against international best practices. benchmarked against international best practices.

The framework clarifies the academic levels, learning The framework clarifies the academic levels, learning outcomes and credit systems based on student academic outcomes and credit systems based on student academic load.load.

The MQF integrates with and links all national The MQF integrates with and links all national qualifications.qualifications.

3 types of quality assurance: 3 types of quality assurance: Provisional Accreditation; Full Accreditation; and Institutional Audit.


COPIA, COPPA, HEP, MQA, MQF

COPIACOPIA – Code of Practice for Institutional – Code of Practice for Institutional AuditAudit

COPPACOPPA – Code of Practice for Programme – Code of Practice for Programme AccreditationAccreditation

HEP HEP – Higher Education Provider / Pemberi – Higher Education Provider / Pemberi Pendidikan TinggiPendidikan Tinggi

MQAMQA – Malaysian Qualification Agency / – Malaysian Qualification Agency / Agensi Kelayakan MalaysiaAgensi Kelayakan Malaysia

MQFMQF – Malaysian Qualifications Framework / – Malaysian Qualifications Framework / Kerangka Kelayakan Malaysia (KKM) Kerangka Kelayakan Malaysia (KKM)


Code of Practice for Institutional Audit (COPIA)

Contains guidelines on criteria & Standards for Higher Education Provider (HEP).

Define requirements for higher education in broad terms. Divided into 9 areas of evaluation. Standards are categorized into 2 attainment levels: Benchmark – expressed as a “must”. Enhanced – expressed as a “should” Purpose: multipurpose audit code , comprehensive. Guided information template. Explains preparations for audits, roles of audit panels, MQA Explains processes, responsibilities, audit criteria, report

writing formats.


Findings and Judgments of COPIA

For Academic Performance Audit (APA): highlight affirmation, commendation and areas of concern. Indicate performance of institution on 9 areas of evaluation. No specific decision of pass or fail. For continuation or cessation of Program Accreditation: Maintain accreditation Cease accreditation For Self-accreditation Status: Grant Self Accreditation Status Not granted Self Accreditation status MQA Institutional Audit Committee will make decisions base

on proposal by the panel of assessors.


Malaysian Qualifications Framework - MQF

Provisional AccreditationProvisional Accreditation – seeking approval from the – seeking approval from the Ministry of Higher Education (MOHE) to conduct new Ministry of Higher Education (MOHE) to conduct new programs.programs.

Full AccreditationFull Accreditation – a conferment to denote that a program – a conferment to denote that a program has met all the criteria and standards set for that purpose.has met all the criteria and standards set for that purpose.

Institutional AuditInstitutional Audit – periodic academic performance audit – periodic academic performance audit on all universities or to establish the continuation or on all universities or to establish the continuation or maintenance of program accreditation status. It could take maintenance of program accreditation status. It could take the form of an exercise for purposes of verifying data, the form of an exercise for purposes of verifying data, public policy input or for rating and ranking of institutions public policy input or for rating and ranking of institutions and programs. The highest form of IA is the self-and programs. The highest form of IA is the self-accreditation audit which can lead to a conferment of a accreditation audit which can lead to a conferment of a self-accreditation status for the institution whereby the self-accreditation status for the institution whereby the institution can accredit its own programs.institution can accredit its own programs.


9 Areas of evaluation COPIA

Vision, mission, educational goals and learning outcomesVision, mission, educational goals and learning outcomes Curriculum design and deliveryCurriculum design and delivery Assessment of studentsAssessment of students Student selection and support servicesStudent selection and support services Academic staffAcademic staff Educational resourcesEducational resources Program monitoring and reviewProgram monitoring and review Leadership, governance and administrationLeadership, governance and administration Total continual quality improvement Total continual quality improvement


2 levels of standards attainment 2 levels of standards attainment

Benchmark standards – are standards that must be met – are standards that must be met and compliance demonstrated during institutional audit. and compliance demonstrated during institutional audit. Benchmark standards are expressed as a “MUST”.Benchmark standards are expressed as a “MUST”.

Enhanced standardsEnhanced standards – are standards that should be met – are standards that should be met as the institution strives to continuously improve itself. as the institution strives to continuously improve itself. Enhanced standards reflect international and national Enhanced standards reflect international and national consensus on good practices in higher education. consensus on good practices in higher education. Enhanced standards are expressed by a “SHOULD”. Enhanced standards are expressed by a “SHOULD”.

The standards define the expected level of attainment for The standards define the expected level of attainment for each criterion and serve as a performance indicator.each criterion and serve as a performance indicator.


9 Areas of evaluation – Benchmark standards

Statements of Vision, Mission, and Educational goals: Set the direction, guide academic planning and Set the direction, guide academic planning and

implementation, and bring members towards achievement implementation, and bring members towards achievement of world class status and tradition of excellence.of world class status and tradition of excellence.

HEP HEP must state vision, mission and educational goals and state vision, mission and educational goals and make known to internal and external stakeholders.make known to internal and external stakeholders.

Mission statements and goals Mission statements and goals must reflect crucial reflect crucial elements of the processes and outcomes of higher elements of the processes and outcomes of higher education.education.

Must formulate goals consistent with vision and mission. formulate goals consistent with vision and mission. Publications Publications must accurately cite the current vision and accurately cite the current vision and

mission.mission.


9 Areas of evaluation – Enhanced standards

Statements of Vision, Mission, and Educational goals: Mission and goals Mission and goals should encompass leadership qualities encompass leadership qualities

in the areas of social responsibility, research attainment, in the areas of social responsibility, research attainment, community involvement, ethical values, profesionalism and community involvement, ethical values, profesionalism and knowledge creation.knowledge creation.

Should demonstrate that planning and evaluation demonstrate that planning and evaluation processes, educational programs, educational support processes, educational programs, educational support services, financial and physical resources, administrative services, financial and physical resources, administrative processes are adequate and appropriate to fulfill stated processes are adequate and appropriate to fulfill stated purpose of mission and goals.purpose of mission and goals.



Curriculum Design and Delivery: Must have a clearly defined process by which the

curriculum is established, reviewed and evaluated. All Educational programs must be considered only after a

needs assessment have been identified. All Educational programs must be considered only after

resources to support the programs have been identified. All aspects of educational programs must be related to

HEP’s vision and mission. Must show that educational content and approach,

teaching learning methods are appropriate, consistent with and support the attainment of learning outcomes.

Teaching learning methods must ensure that students take responsibility for their own learning and prepare for life long learning.



Curriculum design and delivery: The curriculum The curriculum should encourage multi-disciplinary encourage multi-disciplinary

approaches to enhance personal development through approaches to enhance personal development through electives, study pathways and other means which should electives, study pathways and other means which should be monitored and appraisedbe monitored and appraised

There There should be co-curricular activities that will enrich be co-curricular activities that will enrich students’ experiences, foster personal development and students’ experiences, foster personal development and responsibility.responsibility.



Assessment of students: Assessment and Learning Assessment principles, processes, methods and practices

must be aligned with learning outcomes and program content.

The process and methods of assessment must reflect the change to the program outcomes from any review exercise.

The assessment must be consistent with the levels defined in the MQF and the MQF eight domains of learning outcomes.



Assessment of students: Assessment and Learning Assessment methodology Assessment methodology should be reviewed periodically be reviewed periodically

to ensure currency with development in best practices. to ensure currency with development in best practices.



Assessment of students: Assessment methods Frequency and methods of student assessment including

grading criteria and award must be documented and communicated to students.

The assessment methods must comprise of summative and formative purposes compatible with learning outcomes.

A variety of methods and tools must be used appropriately for assessing the given learning outcomes and competencies (communication, problem solving, teamwork and self-directed learning).

There must be mechanisms to ensure the validity, reliability and fairness of the assessment system.



Assessment of students: Assessment methods Methods of assessing Methods of assessing should be comparable to be comparable to

international best practices.international best practices. The assessment systems The assessment systems should beshould be reviewed periodically. reviewed periodically.



Student selection and support services: admission and selection – 13 standards (here are five examples)

The HEP must have an admission policy with clear statements on the criteria and processes of student selection including transferring students.

The criteria and processes of selection must be published and disseminated to students.

If a selection interview is used, the HEP must demonstrate that it is objectively and fairly structured.

Student selection must be free from discrimination and bias. The HEP must provide evidence that it selects students

whose capabilities are consistent with the admission policies.



Student selection and support services: admission and selection

The admission policy should be reviewed periodically. The outcome of selection and student performance should

be monitored to improve the selection process. The size of student intake should be reviewed in

consultation with stakeholders. Students intake should include a consideration of gender

and ethnic balance and social responsibilities such as policies for disadvantaged students.


The Practice: Academic Performance Audit – Audit Prestasi Akademik

Effectiveness of the internal management systems for quality by which institutions manage their core functions with resources and manpower available.

Strength & areas of concern in the academic performance of the universities.

Improvements needed in the areas of concern. Action plans by institutions. Institutions demonstrate that they have systems and

resources in place for institutional effectiveness.


The Practice: Academic Performance Audit – Audit Prestasi Akademik

The audit is in the form of an external quality assurance in which institutions perform self evaluation against standards and criteria.

The quality assurance standards and criteria are intended to help develop, enhance, and strengthen the institutional effectiveness.

The results do not “pass” or “fail”. Reports are not intended to penalize or reward institutions. Information will give trends, differences between institutions,

indication of impact of policy implementation, constraints and expectations.


Some Comparisons between COPIA & COPPA

Area 1: Vision, Mission, Educational Goals & Learning Outcomes

1.1 Statements of vision, mission and educational goals: COPIA – sets the direction of the HEP and guides

educational planning & implementation. COPPA – at the program level, it is more directed to see

how a specific program supports the larger institutional vision.




1.3 Academic Autonomy: COPIA – specifies the autonomy of HEP over academic

matters. COPPA – autonomy in designing the curriculum of a specific

program.




1.4 Learning Outcomes (LO): COPIA – relates LO and competencies to vision and

mission of HEP. COPPA – emphasis on learning outcomes of a particular

program.



Area 2: Curriculum Design and Delivery 2.1 Curriculum Design and Teaching Learning Methods: COPIA – perspective of curriculum design is largely about

institutional policies, structures, processes and practices related to curriculum development across the institution.

COPPA – refers to specific description, content and delivery of a particular program.



Area 2: Curriculum Design and Delivery 2.2 Curriculum Content and Structure: COPIA – relates to HEP’s processes in developing all

program content in general. COPPA – focuses on the processes of developing one

particular program.



Area 3: Assessment of Students 3.3 Management of Student Assessment: COPIA – looks at the HEP’s students assessment system. COPPA – how is the students assessment system

implemented for a specific program.



Area 4: Student Selection and Support Services 4.1 Admission and Selection: COPIA – how the institution comply to the MOHE admission

and selection policies. COPPA – how are the policies cascaded down (diturunkan)

and implemented at the departmental level for a particular program.



Area 4: Student Selection and Support Services

4.5 Alumni:COPIA – incorporating alumni into curriculum

development of HEP.COPPA – involvement of alumni in

developing a specific program



Area 5: Academic StaffCOPIA – looks at HEP policies on academic

staff recruitment, appointment and promotion, and academic staff development.

COPPA – looks at policies and requirements for academic staff involved in teaching a specific program.


Documents

Audit kualiti dalam pendidikan tinggi (Quality Audit in Higher Education)