HIGHER EDUCATION AND TRAINING AWARDS COUNCIL, … · international Microsoft certification by HETAC, which could result in the parallel issue of an ... • Dr Liberty Munson, Psychometrician

Page 1 of 26

Q:\Provider Registration\Providers\Applicants\Applicants\Microsoft\10 05 04 MS Final.doc

HIGHER EDUCATION AND TRAINING AWARDS COUNCIL, IRELAND Comhairle na nDámhachtainí Ardoideachais agus Oiliúna, Éire

Report of Expert Panel convened to consider the application by Microsoft Ireland to HETAC

to Agree Quality Assurance Policies and Procedures for the purposes of providing higher education and/or training programmes

and to consider the associated application by Microsoft Ireland for the validation of five

programmes as Special Purpose Awards at Level 6 on the National Framework of Qualifications

4th May, 2010

Page 2 of 26


Contents PART A Background PART B Summary of Findings PART C Detailed Evaluation of Quality Assurance Policies and Procedures PART D Detailed Evaluation of five programmes submitted for Validation as

Level 6 Special Purpose Awards Appendix One Reports of Site Visits, July and August 2008 Appendix Two Letter to Dr Kevin Marshall, October 2008

Page 3 of 26


HIGHER EDUCATION AND TRAINING AWARDS COUNCIL, IRELAND Comhairle na nDámhachtainí Ardoideachais agus Oiliúna, Éire

PART A Background Initial meetings between Microsoft Ireland and Higher Education and Training Awards Council (HETAC) occurred early in 2008 at which Microsoft sought to establish if they were eligible to seek HETAC registration and the validation of a suite of higher training programmes. Information was provided about the inclusive nature of the National Framework of Qualifications (NFQ), the provisions of the Qualifications (Education and Training) Act 1999 and the relevant HETAC processes for validation and registration. In addition there was significant discussion of the fact that a successful validation of a suite of proprietary awards of a company which provides those certifications on an international basis would also have international implications. With the formal adoption of the European Qualifications Framework (EQF) in April 2008, and Ireland’s formal referencing to the EQF in June 2009, the implications of a validation of an international Microsoft certification by HETAC, which could result in the parallel issue of an Irish Framework award in respect of a Microsoft certification were significant. It was noted that any validation in the Irish jurisdiction leading to a HETAC award would automatically carry a reference to a Level on the EQF. Should Microsoft engage in parallel processes in other jurisdictions the possibility of the same certification being validated at a different Level could not be discounted, which would be unhelpful to learners and employers alike. Microsoft Ireland indicated that it understood the implications of the application and that it was being undertaken as part of an overall strategy to improve learner options for credit transfer across Europe as well as a strategy to engage with Frameworks and Qualifications Agencies worldwide. Microsoft Ireland indicated that it was therefore happy to submit an application in this context and undertook to liaise with its European offices on the matter. A consultant, Mairead Cosgrave of Ashling Communications, was appointed by Microsoft Ireland to project manage the application and the preparation of the associated submission documents. An initial submission, comprised of quality assurance documents and programme documents, was received in August 2008. The evaluation of Microsoft Ireland was conducted using a bespoke process combining the consideration of quality assurance documentation for the organisation and the validation of five programmes during a single exercise. It was informed by the model utilised for applicants for HETAC registration prior to December 2008 and, whilst it contains some of the features of the Registration Policy and Procedure implemented in December 2008, it was not bound by these. A single panel was convened to consider all of the documents and make recommendations on the efficacy of the quality assurance arrangements and the programmes submitted for validation. The panel met with Microsoft Ireland personnel at a site visit on 8th September, 2008.

Page 4 of 26


Following this initial submission and associated evaluation in September 2008 a letter was issued by Dr Bryan Maguire to Microsoft Ireland indicating concerns identified by the expert panel and inviting a resubmission of all documents (appendix one). A resubmission was received in February 2010 and the original panel, with two changes, reconvened to consider the submission and meet with Microsoft Ireland. This report primarily identifies the findings following the second meeting on February 26th 2010, but also captures supporting evidence for positive elements obtained at the meeting of September 8th 2008. Members of the expert panel were:

1. Dr Irene Ainsworth, Head of Degree Awarding Powers and University Title, Quality Assurance Agency, UK

2. Mr Dermot Finan, Chairman, former Registrar, Institute of Technology, Sligo 3. Mr Seamus Gallen, former Director, National Software Directorate 4. Ms Sara Karlsson, Hogskoleverket, National Agency for Higher Education, Sweden

(Ms Karlsson could not attend the second site visit in 2010, but offered feedback and

advice to HETAC on the documentary resubmission) 5. Professor Michael Ryan, Head of Department of Computing, Dublin City University

(Retired at time of second site visit) 6. Dr Áine Uí Éigeartaigh, Programme Specialist, Information and Communications

Technology Department, Tipperary Institute 7. Mr Simon Woodworth, Lecturer, Department of Business Information Systems,

University College, Cork (February 26th, 2010 event only) Mr Joe Cox, Head of Validation (HETAC); Dr Bryan Maguire, Director of Academic Affairs,

(HETAC); and Ms Tara Ryan, Head of Provider Registration and Professional Body Recognition

(HETAC) were in attendance in October 2008. In February 2010 Dr Bryan Maguire (for final

session) and Ms Tara Ryan were in attendance. Tara Ryan acted as secretary to the panel on

both occasions.

During the site visit to Microsoft Ireland in 2008 the panel met with the following persons representing Microsoft: • Mr Michael Brosnan, Microsoft Certified Trainer, Moresoft CPLS Resident trainer • Ms Mairead Cosgrave, Managing Director, Ashling Communications • Mr Kevin Cotter, Microsoft Certified Trainer, Self employed Network Trainer & Consultant • Ms Sinead Doohan, Microsoft Ireland • Mr David Gibson, Microsoft Learning Corp, US • Dr Kevin Marshall, Head of Education, Microsoft Ireland • Dr Liberty Munson, Psychometrician Program Manager, Microsoft Learning, US During the site visit to Microsoft Ireland in 2010 the panel met with the following persons representing Microsoft: • Mr Michael Brosnan, Microsoft Certified Trainer, Moresoft CPLS Resident trainer • Ms Solenn Canny, Channel Operations, Microsoft Learning, Western Europe • Ms Mairead Cosgrave, Managing Director, Ashling Communications • Mr Kevin Cotter, Microsoft Certified Trainer, Self employed Network Trainer and Consultant • Dr Kevin Marshall, Head of Education, Microsoft Ireland • Dr Liberty Munson, Psychometrician Program Manager, Microsoft Learning, US

Page 5 of 26


• Mr Jeroen van Hilten, Area Sales Manager, Western Europe Microsoft Learning • Mr Desmond Waterstone, Microsoft Certified Trainer, Self employed, independent consultant in programming and database development Mr Colin Cassidy, Microsoft Ireland, the designated programme director for all programmes, was unable to meet with the panel members on either occasion, but was represented in February 2010 by Mr Jeroen van Hilten, to whom he reports.

Page 6 of 26


Part B Summary of findings: The expert panel recommends to Council:

• that provided Microsoft Ireland addresses the specific conditions stipulated herein in respect of the quality assurance policies and procedures for the management of its programmes for which it seeks HETAC awards it be eligible to be registered as a HETAC provider and;

• that subject to Microsoft Ireland addressing these quality assurance matters, and additional conditions identified herein in respect of the five submitted programmes, that the following be validated as programmes leading to Level 6, Special Purpose Awards

→ Microsoft Certified Systems Engineer (MCSE)

→ Microsoft Certified Technology Specialist (MCTS)

→ Microsoft Certified Professional Developer (MCPD)

→ Microsoft Certified Systems Administrator (MCSA)

→ Microsoft Certified Trainer (MCT).

The Executive of HETAC is mandated to ensure compliance with indicated conditions before the panel’s recommendations to validate programmes and to register the provider be considered by Council.

Page 7 of 26


PART C Detailed Evaluation of Quality Assurance Policies and Procedures 1 Objective of the Evaluation The objective of this part of the evaluation was to assess whether or not the applicant Microsoft Ireland, as an organisation, can provide high quality higher education and training to the level sought. Evaluation of the quality assurance documentation was conducted broadly using the seven principles of the HETAC registration policy as criteria in order to guide the panel members in making their assessment. Principles 2, 3 and 5 encompass the seven elements of internal quality assurance for a provider as described in Part One of the Standards and Guidelines for

Quality Assurance in the European Higher Education Area (ESG). The applicant was assessed on its understanding of, commitment to and compliance with the identified principles, based on: • documented policies and processes as (re)submitted in a quality manual February 2010; • site visits to Certified Partners for Learning Solutions (CPLS) in July and August 20081 and

in February 20102, 3 and; • oral communications made at the evaluation meetings held on September 8th, 2008 and

February 26th 2010 respectively. Following an initial submission and evaluation in September 2008 a letter4 was issued by Dr Bryan Maguire indicating concerns identified by the expert panel and inviting a resubmission. The resubmission was received in February 2010 and the original panel, with two changes, reconvened to consider the submission and meet with Microsoft Ireland. This report emanates from that second meeting in February 2010. 2 Formal Recommendation of the Panel to Council regarding Quality Assurance In assessing the applicant provider, as an organisation, and answering the question – has the applicant demonstrated the capacity to provide high quality higher education and training to the level sought the panel finds:

1. No 2. No, but a resubmission should be accepted 3. Yes, but with conditions and recommendations X 4. Yes, but with recommendations 5. Yes with no conditions or recommendations

Detail on the conditions and recommendations, together with relevant evidence, is provided in the relevant sections.

1 July 24

th, 2008 - Rosaleen Barrett, Bryan Maguire & Tara Ryan visited BT Training, Hanover St, Dublin 2. The HETAC team

met with Gareth Fennel of BT Training. August 7

th, 2008 - Tara Ryan visited PFH, EastGate Avenue, EastGate, Little Island, Co. Cork. T. Ryan met with Ms Catherine

O’Keeffe, Training Manager, of PFH Technology Group. 2 February 25

th, 2010 - Irene Ainsworth, Dermot Finan, Michael Ryan and Simon Woodworth (and Tara Ryan) visited Global

Knowledge Ireland, Dublin 1. 3 See Appendix One for copies of the reports from July and August 2008.

4 See Appendix Two

Page 8 of 26


3 Main Findings The reasons for the panel’s findings are indicated under the headings hereunder which provided a broad basis for the panel’s evaluation. A Principle 1

There is a system of governance that protects the integrity of academic processes & standards The applicant can demonstrate that: → academic decision-making reflects the interests of learners and the maintenance of standard; → there is evidence that overall corporate decision-makers within the organisation, be they

owners, shareholders or trustees, do not exercise exclusive authority or undue influence over academic decision-making;

→ academic decision-making independent of commercial considerations; → academic decision-makers are appropriately qualified and experienced.

Specific Finding The panel’s overall finding was that this principle has been met. General Comments The panel is satisfied that Microsoft Ireland has tightly prescribed arrangements pertaining to the management of academic decision-making. They are determined by US-based Microsoft Learning and are appropriate to enable the organisation to deliver high quality higher education/training in the field of studies and to the level sought, NFQ 6, in Ireland. The panel had previously found it difficult to ascertain Microsoft Ireland’s position in the governance arrangements associated with the management of the qualifications concerned as in the 2008 submission and subsequent dialogue there was an absence of a clear articulation of the respective roles and responsibilities of the various parties concerned in this initiative. The panel welcomed the clarification provided on this matter in the course of further discussion with Microsoft representatives in February 2010. The comprehensive documentation supplied by Microsoft Ireland demonstrates an understanding that academic decisions are made in a context different to that of commercial decisions. For the purposes of this evaluation the panel is of the view that both the interests of learners and the maintenance of standards are protected through the model of training provision being deployed by Microsoft. There are many discrete units which design, test and compile programmes within the company. These are separate from the training delivery organisations which distribute the programmes, and from the further third parties that manage the assessments and the collection of feedback. Poor provision of training to learners, or the failure to maintain standards may in themselves result in the undermining of Microsoft products on which the training is provided. An internal tension between product development and the development of training for those products serves to maintain a balance of protected interests, thus ensuring the learner is receiving a good learning experience and a training of high standard. The qualifications are assessed and awarded on the basis of achievement of specified training outcomes. Microsoft Ireland works through a network of seven Microsoft Certified

Page 9 of 26


Partners for Learning Solutions (CPLS) and partners are required to sign an agreement that they will adhere to Microsoft’s learning quality assurance requirements. There are no examination boards, as examinations are conducted in a rigorously managed online environment with the provision of immediate results. This is managed on behalf of Microsoft by Prometric and ensures, along with the Microsoft assessment process, the independence, reliability and validity of results, and guarantees the absence of interference in same. Through these processes decisions about results and certifications are made with fairness and reliability. Whilst the panel notes that Microsoft does not exist to help learners per se, in promoting their own products learners are treated with centrality and importance. Microsoft has an interest in ensuring that its learners are trained to the highest standards and, to that end, it has established a rigorous system of quality assurance.

In addition the panel welcomed the news that the Head of Education, Microsoft Ireland, Dr Kevin Marshall would be the main point of contact with HETAC. Specific Conditions None Specific Recommendations None


Page 10 of 26

B. Principles 2, 3 and 5 encompass all internal academic quality assurance (i.e. the seven elements

of the ESG) including assessment, access, transfer and progression

The applicant has robust quality assurance and quality enhancement policies, procedures and practices, specifically

→ the applicant’s strategy, policy and procedures for quality assurance meet European standards for internal quality assurance within higher education organisations as set out in Part 1 of Standards and Guidelines for Quality Assurance in the European Higher Education

Area (ESG) 5.

• Policy and procedures for quality assurance

• Approval, monitoring and periodic review of programmes and awards

• Assessment of students (Principle 3)

• Quality assurance of teaching staff

• Learning resources and support

• Information systems

• Public information

The applicant’s policy and procedures for Access, Transfer and Progression are consistent with the National Qualifications Authority of Ireland’s Policies, actions and procedures for Access,

Transfer and Progression for Learners (Principle 5), specifically

→ the organisation recognises that any given programme is but one element in an individual’s journey of lifelong learning and puts in place policies and procedures to support that journey;

→ the core elements of the Policies, Actions and Procedures for Access Transfer and Progression of Learners (2003) as determined by the National Qualifications Authority of Ireland (NQAI) being implemented by the applicant, and the applicant capable and open to so applying;

→ the applicant has appropriate procedures relating to credit, transfer and progression routes, entry arrangements and the provision of information.

Specific Finding The panel is satisfied that Microsoft Ireland has robust quality assurance and quality enhancement policies, procedures and practices in place as evidenced by the documentation provided and confirmed at the panel visit. General Comments The development of the certifications, their assessment and their ongoing review are particularly strong and highly commendable in their rigour. Many elements of this process are unique to Microsoft and as such are confidential – the quality assurance process pertaining to the management of tests and testing themselves are proprietary tools, and therefore unlike many quality assurance systems, they are not fully open to public scrutiny on an ongoing basis– they are commercially sensitive. Notwithstanding this the Microsoft team presented information in a fulsome and frank manner with the panel and provided clear assurances that the learner would receive a positive learning experience and that certifications are provided and assessed in a reliable, valid, fit for purpose way.

5 http://www.enqa.eu/files/ESG_3edition%20(2).pdf


Page 11 of 26

Third Party relationships

As indicated in section 4A Microsoft has established a Certified Partner for Learning Solutions (CPLS) network in Ireland, comprising seven authorised CPLS organisations. The Certifications for which validation is sought are provided through these CPLS organisations and thus the relationship between Microsoft and the CPLS is key, and its security, integrity and ongoing monitoring and review is an essential element in the Microsoft quality assurance regime. CPLS partner agreements with Microsoft stipulate that CPLS organisations must use only Microsoft Certified Trainers (MCTs) to deliver Microsoft certified training. Equally, as Microsoft outsources responsibility for the provision of examination testing (to Prometric) and for feedback (to Knowledge Advisors, Metrics That Matter) to third parties these are key relationships and their management is robust. Feedback collected covers a wide variety of matters and can inform certification development, assessment and training provision in a CPLS. Feedback

Information provided by third parties is subject to regular analysis by Microsoft in the USA and feedback is provided to Microsoft organisations globally. Microsoft Ireland conducts monthly reviews of Metrics That Matter which indicates customer satisfaction levels with the quality of the training provided by MCTs in the CPLS organisations. These regular ongoing internal audits, together with a yearly internal audit, serve as a check against quality goals. A check list of items is provided to the auditor at the annual internal audit and he/she indicates, in writing, whether the programme “Fails to Meet or Meets the documented process in each major category (e.g. subcontracting, examination development process and outcomes, use of Subject Matter Experts, etc)”. Assessment

The assessment of learners is rigorously planned and administered and the quality of each certification examination is continually evaluated in terms of subject content and learner performance. Microsoft’s Examination Development Process is well documented (Section 2.0 of its QA submission refers) and the Examination Standards and Procedures Processes are well documented (QA document section 3.0 refers). A matter of concern relating to the issue of HETAC certification was identified. Certain types of abuse/cheating in a test can be identified by Microsoft but might not come to light until it had undertaken its own review of performance outcomes, i.e. when Microsoft has conducted a quarterly monitoring of the patterns of questions and answers given. As such, it was conceivable that, a candidate may already have received a positive test result and been issued with a HETAC award, prior to Microsoft identifying a problem. To counter this possibility, it was proposed that HETAC should only issue certificates to students on receipt of quarterly reports from Microsoft to HETAC indicating those students who had successfully completed their assessments. This would mean that only three months after the test result is issued, on Microsoft’s explicit confirmation, would a HETAC award be made. Teaching Staff

The panel regarded the provision of training leading to the acquisition of Certifications via the CPLS organisations as a good model, but noted that the awards can be earned on the


Page 12 of 26

basis of no teaching input. Whilst Microsoft would like everyone who teaches the use of its products to have the appropriate Microsoft qualification, it cannot prevent others from providing this service. The control point is the examination. It was also noted that, whilst Microsoft was only seeking validation of five certifications obtainable at any of the seven CPLS organisations in Ireland, any successful candidate who attains one of these certifications will be eligible to seek a parallel award directly from HETAC for that certification. Access, transfer and progression

Seeking a HETAC award for a Microsoft Certification may contribute to the courses that a learner can access in other higher education providers, and may provide a pathway for exemptions. Microsoft has published comprehensive information on what a prospective learner needs to know to undertake a certification and has provided on-line self-testing in this regard, but does not limit access – persons can self-select to attend for certifications. One of the strengths is the support for learners through web-based self-study courses and mock examinations. Whilst these are not conventional academic courses/examinations, they should not be regarded as a weakness.

A potential perception of weakness vis-à-vis other types of academic awards, is the fact that the certification may lapse in relevance, whilst the award will not be withdrawn. A learner cannot lose an award made, but, if the product on which it was obtained is withdrawn or replaced (or just stops selling), the award becomes irrelevant – it has no standing. This is unavoidable in this particular business environment. In addition Microsoft makes it clear for which version of the product the award was made, so that there is no danger of potential employers or other parties being misled as to how up- to-date an individual’s qualifications are, provided that individual’s full transcript is available.

Specific Conditions 1 Assessment and Standards (2009) advises that HETAC registered providers normally

appoint (two) external examiners for their programmes, although on initial registration they are appointed by HETAC. It is acknowledged that the confidential nature of some of the assessment processes considered herein, combined with the model of assessment adopted which has limited possibilities for human intervention, requires that the external examiners appointed for the Microsoft provision have a distinctive remit. Given the nature of the assessment and programme development models which are rigorously controlled by Microsoft, the panel advises that HETAC appoint external examiners to Microsoft to perform specific functions. The panel believes that this is essential to ensure that the standards set by HETAC for its awards have been achieved. The panel noted that the standards for Microsoft Certifications are outside the control of either an external examiner or HETAC, and therefore the advice of the examiners, experts in the fields covered by Microsoft certification, will be critical to HETAC in ensuring the maintenance of the standards of HETAC awards.

The responsibilities of the external examiners shall be to:

• consider and advise whether the specified programme continues to meet the standard required to attain a HETAC award by:


Page 13 of 26

o reviewing all changes made to programmes on an annual basis; o reviewing reports on Prometric assessments;

• consider and advise whether the learners are receiving quality education and training o reviewing completion statistics for those registering with CPLS organisations; o reviewing reports received through Metrics That Matter and CPLS

organisations’ responses, if any, to those reports;

• prepare an annual written report, including recommendations, to Microsoft Ireland and to HETAC based on the findings

The external examiners should have a two year term, which may be extended by one year. The examiners will be experts in the fields covered by Microsoft certification and their nomination will be agreed between HETAC and Microsoft. HETAC shall continue to appoint external examiners, but this arrangement is subject to review during monitoring. It is further noted that elements of the External Examiner Report may be deemed confidential, and may be subject to non-disclosure agreements. 2 As indicated above HETAC will only issue parchments to students following an appropriate time-lag and on receipt of quarterly reports indicating those students who had successfully completed their assessments in order to allow Microsoft review the individual psychometric footprint and check for anomalies. Specific Recommendations It is recommended that Microsoft formally and systematically evaluates the relevancy of the National Qualifications Authority of Ireland (NQAI) Policies, Actions and Procedures for Access Transfer and Progression of Learners (2003) for its programmes and their provision, and submits proposals to implement any features identified therein. This is not a precondition to registration.


Page 14 of 26

C. Principle 4 The applicant’s organisational environment promotes learning, specifically

→ the organisation has an open intellectual community that values critical reflection and fosters personal and professional development for learners and staff;

→ staff are appropriately qualified and experienced; → the organisation has processes in place to ensure that the content of programmes reflects

advances in the relevant disciplines; → the pedagogic style incorporates good practice.

Specific Finding

Whilst the primary aim of Microsoft is not an educational one, within the organisational environment of Microsoft Learning, learning is promoted and fostered. Microsoft has an open online community which facilitates learning and provides many supplementary training materials online at no cost. Certification of Microsoft Certified Trainers must be renewed annually. General Comments Microsoft Certified Trainers (MCT) are highly skilled and experienced ICT practitioners who must apply, pay a fee and meet a number of programme requirements in order to renew their certification as MCTs for the following year. Information on this is provided in Section 5.0 of the Quality Assurance manual. Because programme requirements demand that “MCTs consistently satisfy customers” independent audits of the MCTs’ performance are regularly conducted during programmes to ensure compliance with programme requirements. To this end MCTs must meet competency requirements for each course they deliver. To some extent pedagogic style is dependent on CPLS performance. However all CPLS organisations must meet Microsoft quality standards and as all Microsoft Certified Professional material is already designed for home-study or classroom training, some flexibility in delivery can be accommodated.

In addition Microsoft Certified Trainers could benefit from professional links with Higher Education providers, i.e. universities and institutes of technology where state of the art programmes in teaching and learning are being conducted apace at the present time. An initiative such as this would augment the specific recommendation made at D Principle 6 - Peer relationships with the broader community of higher education and training .

It is worthy of note that the examination process is conducted independently of MCTs. In addition, while the examinations are typically conducted at CPLS sites, the conduct of the examinations themselves is tightly controlled by Microsoft and Prometric.

The content of modules is reviewed and updated continually and the Content Development Managers are responsible, inter alia, for the quality and content of course material and managing examination publishing. There is new content for particular certifications, and new certifications and new examinations on an ongoing basis. Specific Recommendations None


Page 15 of 26

D. Principle 6 - Peer relationships with the broader community of higher education and training

→ The applicant can demonstrate an understanding that higher education and training is a collegial, international endeavour;

→ Awards made under the National Framework of Qualifications are intended to promote mutual recognition and confidence in the learning outcomes attained. Other awards or accreditation offered through the organisation are from reputable bodies;

→ Where formal collaborations with other higher education providers or organisations exist these relationships are described and listed in applicant literature and on websites.

Specific Finding

There are significant pockets of strength relating to this principle including the welcome initiative of Microsoft Academy. Amongst other benefits it enables some Irish institutions and their lecturers to prepare learners with the skills they need in today’s job market. Whilst in parallel with this Microsoft Ireland has an under-developed understanding of itself as a provider of higher education/training awards and their associated communities of practice, sufficient evidence was educed to support this principle overall.

General Comments Whilst Microsoft naturally does not have a conventional peer relationship with the educational world, it has made substantial efforts to engage with education providers in a serious manner. They include this application to HETAC; the links to many Irish higher education providers, to whom lecturer training is offered to ensure that Microsoft modules can be taught well; and engagement with international educational contexts such as the USA (ANSI) Scotland (SQA) and South Africa (SAQA), to name just three. Nevertheless a somewhat ad hoc engagement with the higher education and training community in Ireland is a missed opportunity to work more strategically with that community in progressing information and communications technology as a discipline and to facilitate student transfer and progression opportunities between Microsoft and established higher education and training providers Specific Conditions None Specific Recommendations Microsoft, the Microsoft Academy and higher education would benefit from a more developmental mutual cooperation. It is recommended that Microsoft Ireland develop an inclusive national network/forum with higher education providers to consider matters of mutual interest. This forum could promote information and communications technologies and provide an opportunity to exchange information on sectoral developments. It would also be a locus for dialogue on access, transfer and progression issues. It may also be of benefit to Microsoft Ireland to identify all the different units of Microsoft Ireland that engage with higher education and training and both collate that information and develop a coordinated approach to that engagement.


Page 16 of 26

E. Principle 7 - Maintenance of a resource base sufficient to protect learners’ investment of time and money

→ the organisation is adequately resourced to undertake and complete the programmes proposed

→ the physical and electronic infrastructure are provided on a stable basis → the financial plans are based on realistic projections of student numbers and other variables → the administrative infrastructure of the organisation is able to provide timely decision making

to learners → the administrative infrastructure is able to provide a regular flow of information to HETAC

and other stakeholders → there are appropriate transfer or bonding plans in place to protect learners in the event that

the organisation is no longer able to complete delivery of a programme after it has commenced

Specific Finding

This principle is addressed. General Comments Due to the interrelationship of Microsoft with its service partners this principle is somewhat addressed by the different model of provision – learners enrol on training courses and register to sit assessments on a separate basis. Neither the training nor the assessment are directly administered by Microsoft whose role is to quality assure the activities and the relationships with these service partners. The Quality Assurance manual (sections 4 and 6) addresses these matters in detail. Microsoft evaluates and monitors the CPLS organisations where training is provided. As part of the evaluation, the panel visited Global Knowledge Ireland (Dublin) which is both a Microsoft Certified Partner for Learning Solutions and a Prometric Exam Centre and one of seven such approved centres throughout Ireland. The panel was satisfied on the basis of the evidence provided at the visit that the resources, equipment and management expertise were sufficient to protect learners’ investment of time and money. Relating to this principle as well as that of appropriate assessment (see B condition 20) is the need for independent monitoring of the relationship between HETAC and Microsoft. This led to Microsoft suggesting a way forward to address HETAC’s need for information about the development of the provision, student performance and quality assurance related matters. Specific Conditions See B, condition 2. Specific Recommendations None


Page 17 of 26

Part D Programme Validation The objective of this part of the evaluation was to assess whether or not the programmes submitted meet the criteria for validation articulating assessable learning outcomes which meet the standard of the award sought. Formal Recommendation of the Panel to Council regarding programmes to be validated Main Finding In assessing the programmes submitted for validation the panel found that each of the programmes met the criteria for validation, and the NFQ descriptor for Level 6 and were appropriately designated as Special Purpose Awards. Some conditions in respect of the programmes were made and they are detailed below. Microsoft must resubmit amended programme schedules in this regard.

→ Microsoft Certified Systems Engineer (MCSE) 35 Credits

→ Microsoft Certified Technology Specialist (MCTS) 10 Credits

→ Microsoft Certified Professional Developer (MCPD) 15 or 25 Credits

→ Microsoft Certified Systems Administrator (MCSA) 20 Credits

→ Microsoft Certified Trainer (MCT). 10 Credits


Page 18 of 26

Detailed Findings

The overarching criteria for the validation of a programme:

1 Standards: The minimum intended programme learning outcomes must be consistent with the relevant awards standards and the NFQ award-type descriptors.

2 Access: The prerequisite learning for participation in the programme and any other assumptions relating to the programme’s target learners must be explicit.

3 Learning: The programme must enable its target learners to attain the minimum intended

programme learning outcomes reliably and efficiently (in terms of learner effort)

1 Development and publication of explicit intended learning outcomes (ESG)

1.1 The following were explicitly specified: → the target learners’ prerequisite learning6 and any relevant assumptions about programme

participants; → the minimum intended programme learning outcomes and any other educational objectives of

the programme. 1.2 There was evidence that the minimum intended programme learning outcomes are

consistent with the NFQ level indicators 1.3 As awards standards are cumulative, the programme’s prerequisite learning specification (access

learning requirements) includes the knowledge, skill and competence specified at lower NFQ levels in the applicable awards standards for which the programme does not provide learning opportunities.

Specific Finding

The documentation is very clear and detailed describing the intended learning outcomes, despite the fact that the specific term is not used. The panel found that the proposed awards are consistent with the NFQ award type descriptors. Detailed information, along with self-testing options, is freely available on-line to allow prospective candidates establish if they have the requisite knowledge, skill and competence to undertake the courses. General Comments The intended learning outcomes, which are clearly specified, reflect the special purpose nature and associated content of the awards which are geared to industry-specific needs. Detailed discussion took place on the Level proposed and its appropriateness. In particular, there was dialogue on differences in the Certifications proposed and the desire to validate at the same Level. The trainers present demonstrated significant expertise and confidence in explaining the similarity and overlap between the certifications. The panel found that it was appropriate to place each of the certifications at the same Level. Specific Conditions None Specific Recommendations None

6 Prior learning contributes significantly to the determination of what a particular learner can learn in a particular

programme. A change in the pre-requisite learning requirements for a given programme may require a change in the

programme. The relationship between these changes is not linear—a small change in the former could result in a large

change in the latter. This has implications for the programme’s credit profile.


Page 19 of 26

2 Careful attention to curriculum and programme design and content (ESG)

2.1 Evidence was educed to demonstrate that target learners may achieve the intended programme learning outcomes

2.2 The programme’s content and learning environment is appropriate to the programme’s intended learning outcomes

2.3 The programme involves authentic learning opportunities to enable the achievement of the intended programme learning outcomes

2.4 The programme (as a process and considering its intended learning outcomes and other educational objectives) compares well against appropriate benchmarks

2.5 The programme meets genuine education and training needs 2.6 The programme is viable 2.7 The programmes have procedures for assessment of learners which are consistent with HETAC’s

Assessment and Standards 2009

Specific Finding

→ The content of each programme and learning environment are appropriate to each programme’s intended learning outcomes.

→ The CPLS environment and online Microsoft support provide clear opportunities for learners to achieve the outcomes.

→ As a unique set of programmes for a set of proprietary technologies they do not have direct comparators. However the rigour noted in assessing the attainment of intended outcomes is fully substantiated in the processes described.

General Comments The training provided is outcome-, rather than process-driven and there is little reference to formative assessment. The credit weighting proposed and associated justification was not viewed as an accurate/best-fit reflection of the workload associated with the certifications. There is a need to review the number of credits awarded for the 180 hours candidates are required to complete for each programme. The submission indicated that this would equate to seven credits for each 180 hours. The panel’s view was that this should be changed to five credits. Specific Conditions 1 Individual Microsoft Certified Professional examinations (MCP’s) should be valued at 5 European Credit Transfer System (ECTS) each with a minimum of two Microsoft Certified Professional examinations required to achieve the minimum 10 ECTS needed for a HETAC Special Purpose Award. This better reflects the typical learner workload hours and should facilitate transfer and progression opportunities with higher education providers. 2 Microsoft must provide HETAC with revised Programme Submissions to reflect the changes in module credit weighting to include, inter alia, new Programme Schedules for each of the five programmes to correspond to HETAC protocols. 3 Conditions specified in Part C, Section B pertain. 4 In addition to the two MCP minimum requirement, a HETAC Special Purpose award shall not be awarded unless the student has also achieved a Microsoft award of MCSE, MCTS, MCPD, MCSA or MCT. This prevents learners gaining a HETAC award on the basis of two


Page 20 of 26

randomly selected MCPs which would not entitle the student to one of the Microsoft awards in any case. Specific Recommendations It is recommended that Microsoft publish the HETAC programme schedules associated with the certifications.


Page 21 of 26

3 Specific needs of different modes of delivery and types of higher education (ESG)

3.1 There has been appropriate consideration of the modes of delivery including, for example, full- time, part-time, distance learning, e-learning.

3.2 There has been consideration of the type of higher education, for example academic, vocational, professional and any specific features or requirements pertaining to this type of provision in respect of the programme being validated

Specific Finding

The programmes are accessible to learners in a wide variety of modes from home-study, classroom, online – self-selected by the learner; the modes are appropriate to the type of training being offered. The qualifications are vocational in orientation and this is reflected in the training provided and in the assessment model adopted.

Specific Conditions None Specific Recommendations None


Page 22 of 26

Declarations of Interest Simon Woodworth received training from Microsoft and it was funded by them. He undertook four MCTS

certification courses from October 2006 to January 2007 and achieved certification in Windows Application development, Web Application development and Distributed Systems development. These courses were delivered by PFH in Cork and were part-funded by FASA for private sector course participants. Microsoft the course fees of €5000 course fees directly to FAS or PFH. While the BIS department in UCC was fully aware of and supported Simon Woodworth’s training, it was not a party to the financial transaction. The rationale for undertaking these certifications was to enhance material delivered on an existing postgraduate course and to provide a meaningful link between the course content and Microsoft's certification programme.


Page 23 of 26

Appendix One A

Site Visit - Microsoft CPLS - BT Training, Hanover St, Dublin 2 Date: July 24th, 2008

Attended: Rosaleen Barrett, Bryan Maguire, Tara Ryan The HETAC team met with Gareth Fennel of BT Training. The HETAC Group toured the training facilities which consisted of 7 networked pc laboratories & shared social areas. In discussions it was established that there is a close ongoing working relationship between Microsoft and BT Training. CPLS certification is acquired on the basis of fulfilling certain criteria primarily having certain numbers of MS certified staff and running programmes in accordance with the Microsoft specification. A key feature of the ongoing evaluation of the CPLS is the completion of evaluation forms by course participants. Examples were given of the procedure for Corrective Actions based on these evaluations. The completion of a certain percentage of forms was a requirement of Microsoft’s. In addition an important part of this process in that the CPLS has the opportunity to offer Microsoft feedback on programmes and indicated that changes occur to programmes based on feedback collated. BT Training is the largest certified Training Centre for Microsoft in Ireland and they also have an additional site in Belfast. While BT Training carry out various training of their own at the centre the Microsoft training programmes make up about 40% of the total training. Almost the entire client base is that of Corporate / Industry sector. Where necessary, programmes are tailored to the needs of the client and these programmes are designed and delivered by in-house experts, though BT Training have a group of Consultants that they can call on if the expertise is not available internally.

B Site Visit

Microsoft CPLS – PFH, EastGate Avenue, EastGate, Little Island, Co. Cork Date: August 7th, 2008

Attended: Tara Ryan Tara Ryan met with Ms Catherine O’Keeffe, Training Manager, of PFH Technology Group in Cork, at EastGate, Little Island, Cork. PFH is the one of only three Microsoft Certified Partner for Learning Solutions (CPLS) based outside of Dublin on the island of Ireland. Ms O’Keeffe gave Ms Ryan a comprehensive tour of the training facilities and introduced Ms Ryan to the trainers present as well as the Training Support team. There were four networked pc laboratories & some shared social areas, all seeming of a high quality. Ms O’Keeffe gave clear explanations of PFH’s development and relationship with Microsoft. The trainers present described their roles, their certification as Microsoft trainers and their input into the Microsoft programme development process through the provision of feedback. It was explained that there is a close ongoing working relationship between Microsoft and PFH Technology Group. The training provided was set in a professional context and the desire to meet high standards, along with rigorous evaluation of service delivered was well articulated.


Page 24 of 26

Dr Kevin Marshall 23rd October 2008 Education Director EOC Operations Centre Atrium Building Block B Carmenhall Road Sandyford Industrial Estate Dublin 18 Dear Dr Marshall Many thanks for your submission to HETAC for the agreement of Quality Assurance Procedures and the validation of Microsoft Programmes. May I thank you also for your hospitality, assistance and facilitation of the evaluation meeting which took place at your offices on September 12th. I apologise that we have not been able to communicate with you sooner. The day itself was very helpful in identifying matters to be addressed in what has become a rather iterative process towards accreditation of Microsoft programmes. The panel members were cognisant of the high standards associated with Microsoft proprietary awards and the broad strategic focus of Microsoft in pursuing accreditation and a link to both the Irish Framework of Qualifications and the European Qualifications Framework. Given the latter context and the various due diligence processes in which both Microsoft and HETAC must engage on behalf of their various stakeholders, the meeting was very useful in clarifying the context of the application and the nature of our respective responsibilities in any potential relationship. As the chairman indicated on the day, there were some specific points of feedback and the following points are offered for your consideration in the interests of taking matters forward. Conclusions and feedback offered to Microsoft: 1. The application is very welcome and HETAC is happy to continue discussions to find an

appropriate model and framework in which to best address it.


Page 25 of 26

2. Dialogue between industry and academia is to be welcomed and work in this area is to be encouraged and supported. The documentation submitted by Microsoft was considered to be very weak. Panel members noted that it lacked an overview which would have helped (a) to place the submission in an appropriate context and (b) to facilitate a greater understanding of the roles and inter-relationships between Microsoft Corporate, Microsoft Ireland and their respective partners. At the same time, the submission would have benefited from careful editing since it lacked focus, did not address HETAC’s information requirements, and contained many typographical errors, suggesting a lack of oversight.

3. The meeting with key Microsoft staff provided a welcome contrast to the documentation submitted and provided a much clearer indication of the relationship between Microsoft Corporate and Microsoft Ireland, on the one hand, and between them and other partners involved, which is essential in any quality assurance process.

4. A clear case demonstrating correspondence with HETAC computing standards and the standards described in the Minor, Special Purpose and Supplemental Award policy should be made in any subsequent application for recognition or validation of certifications.

5. Quality assurance documentation submitted should explicitly address the HETAC criteria for quality assurance policies and procedures (though it is not necessary to make reference to HETAC itself).

6. Any re-submitted quality assurance documentation will require a clear organisation management diagramme with a description of relationships, authorities and responsibilities in respect of the programmes/certifications being provided in Ireland (the roles of Microsoft Corporate, MS Ireland, CPLS, Prometric, etc. should be included). At the same time, it would be helpful for Microsoft to clarify where, and with whom, responsibility for the management of the application process is to reside.

7. Additional sections are required in the quality manual in a number of areas but, depending on the nature of the application, viz. programme validation or certification recognition, different areas will need clarification.

8. More information is required in the document submitted on partner relationships. Microsoft might consider a general section here on awarding or certification bodies, in general, and the nature of the due diligence actions that might be required in its relationships with such bodies.

9. Should a relationship with HETAC be pursued, it may be useful to appoint an external monitor within the mechanisms for ensuring communication and fulfilment of various stakeholder requirements. This person could be seen as being analogous to an external examiner, but with a brief to advise on a wider range of matters pertaining to or impinging upon the academic quality of the awards being made. This person could be jointly appointed by HETAC and Microsoft.

10. A clear ‘programme schedule’ delineating all examinations required for each certification with

its correct credit weighting is required in any re-submission presented. I think that a helpful next step in the process would be for Microsoft to confirm whether it wishes to submit a formal request to apply either

1) for recognition of all specified certifications issued in Ireland (at the appropriate NFQ level and with the appropriate credit volume) as HETAC special purpose awards


Page 26 of 26

or 2) the accreditation of specified certifications as programmes of study, offered in nominated CPLS, as HETAC special purpose awards.

Clarification on this matter will then enable HETAC to address the application within the appropriate policy and legislative context and ensure that all matters of due diligence are addressed. Yours sincerely Dr Bryan Maguire Director of Academic Affairs cc. Mr Michael Brosnan, MCT

Mr Colin Cassidy, MS Ireland Ms Mairead Cosgrave, Ashling Communication Mr Kevin Cotter, MCT Mr Joe Cox, Programme Validation, HETAC Ms Sinead Doohan MS Ireland Mr David Gibson MS Learning Corp Ms Liberty Munson MS Learning Corp Ms Tara Ryan, Head of Programme Monitoring, HETAC

Documents

HIGHER EDUCATION AND TRAINING AWARDS COUNCIL, … · international Microsoft certification by HETAC, which could result in the parallel issue of an ... • Dr Liberty Munson, Psychometrician