Vida Beresnevičiūtė Arūnas Poviliūnas Rūta Žiliukaitė Beresnevičiūtė Arūnas Poviliūnas Rūta Žiliukait ... Vida Beresnevičiūtė Arūnas Poviliūnas Rūta Žiliukaitė

Guidelines for research of a field of professional activity

Vida BeresnevičiūtėArūnas Poviliūnas

Rūta Žiliukaitė

Development of the Concept of the European Credit Transfer and Accumulation System (ects) at the National Level: Harmonization of the Credit and Implementation of the Learning Outcomes

Based Study Programme Design(VP1-2.2-Šmm-08-V-01-001)

Vida BeresnevičiūtėArūnas Poviliūnas

Rūta Žiliukaitė

Guidelines for research of a fieldof professional activity

Vilnius2012

Reviewed by dr. Almeda Kurienė (Lithuanian University of Educational Sciences) dr. Vidmantas Tūtlys (Vytautas Magnus University)

© Vilniaus universitetas, 2012

ISBN 978-609-462-006-5

Approved by the Council of the Faculty of Philosophy, Vilnius university, April 25, 2012

Vida BeresnevičiūtėArūnas PoviliūnasRūta Žiliukaitė


TABle of ConTenTs

introduction .............................................................................................................................................4

1. team of professional activities’ field research. Brief summary of staGes of the ..........................................................................................7

2. maKinG lists of suBJect-specific and Generic competences .......................9

3. survey of employers ............................................................................................................16

4. focus Groups With Graduates .......................................................................................21

4


Back to table of contents

Introduction

When designing new degree programmes, higher education institutions justify the need for such programmes by labour market demands and by employers’ certification that their companies and institutions need specialists with certain competences, which shall be developed by the new degree programme. Designers of new degree programmes often rely on the research of a field of professional activities of a wider or lesser extent, or that are more or less reliable. However, such study is not a common practice when upgrading degree programmes. Is the research of professional activity necessary? And how can it be useful when designing or upgrading degree programmes?

The main goal of research of professional activities is to define subject-specific and generic competences as a combination of knowledge, skills, capacities, values and attitudes that a specialist of a certain field must have in order to properly perform activities1. Such list of subject-specific and generic competences, which is developed by integrating professional activity field experience rather than based on the unilateral assessment of professional activity needs given by degree programme designers, forms the basis for the definition of learning outcomes. Learning outcomes are statements specifying the level of competences that graduates should have upon the completion of the programme, i.e. what they should know and understand and what skills and capacities they should have upon graduation from the degree programme2. When designing degree programmes, research of the field of professional activities provides an opportunity to move from a more traditional approach, which emphasises knowledge and content related to the preferences of professors of higher education institutions, to the approach of the European Credit Transfer and Accumulation System (ECTS) and the project of Tuning Educational Structures in Europe (hereinafter referred to as the Tuning project), which emphasises the student-centred approach and the development of students’ competences in order to provide the graduates of the degree programme with the maximum opportunities to integrate into the labour market.

Research of the field of professional activities studies professional activity from the point of view of social relations. It means that the characteristics of professional activity also depend on the place the person performing professional activities occupies with regard to social relations. Research of professional activities covers not only certain positions but also the rules regulating professional activities, which can be formalised and non-formalised. Such rules are more or less visible compromises among the key players of the professional activity field: employers, representatives of education institutions and graduates.

Therefore, the research of the field of professional activities should first of all be considered as a tool of a constructive and productive dialogue among higher education institutions, employers and graduates. Unsystematically gathered information about the graduates’ satisfaction with the completed degree programme or about the employers’ opinion regarding which knowledge, skills and capacities the young specialists lack upon their graduation from certain degree programmes

1 Taking into consideration the existing variety of the definitions of the competence’s term, it is important to indicate that hereby the definition of competence provided by TUNING methodology is used: “competences and skills are understood as including knowing and understanding (theoretical knowledge of an academic field, the capacity to know and understand), knowing how to act (practical and operational application of knowledge to certain situations), knowing how to be (values as an integral element of the way of perceiving and living with others and in a social context). Competences represent a combination of attributes (with respect to knowledge and its application, attitudes, skills and responsibilities) that describe the level or degree to which a person is capable of performing them.“ (Tuning, 2003, p. 68). There are two types of competences: competences which are subject-area related, „they are intimately related to specific knowledge of a field of study [...] these give identity and consistency to the particular degree programme“ and generic competences „which could be general to any degree“ (ibid).2 Ibid.

5



will hardly provide the designers of degree programmes with sufficient precise knowledge how to amend degree programmes in order for them to meet the needs of the professional activity field. Systematic surveys of employers and graduates performed applying the methods of social research may provide with reliable information about professional capacities of graduates and prevent the designers of degree programmes from making mistakes resulting from the lack of such information.

On the other hand, if the research of the field of professional activities is viewed as a tool of striking up a dialogue with representatives of the professional activity field, it is important to recognise not only the importance of the input of employers and graduates when designing and upgrading degree programmes, but also the role of teaching staff of higher education institutions in charge of such degree programmes. Not all expectations of employers and graduates are necessarily reasonable and not all of them have to be taken into consideration; therefore, professional activity research systematically performed by designers of degree programmes may be an instrument that helps these social partners understand what a higher education institution can achieve and what it cannot achieve while training specialist of a certain qualification for the labour market, and what cooperation is needed between employers and higher education institutions in order to get the most optimal outcome of studies.

Guidelines of Research of the Field of Professional Activities (hereinafter referred to as the Guidelines) have been prepared during the implementation of national project Development of the Concept of the European Credit Transfer and Accumulation System (ECTS) at the National Level: Harmonisation of the Credit and Implementation of the Learning Outcomes Based Study Programme Design (No. VP1-2.2-ŠM-08-V-01-001) (hereinafter referred to as the ECTS project). The implementation of the ECTS project included the conducting of the research of professional activities the goal of which was to perform the analysis of the professional activity field of the eight study areas stated in the project in order to provide important information to project participants who upgrade degree programmes.

Evaluating the benefits of the research for upgrading degree programmes while summarising the research results, teachers of various higher education institutions who participated in the research of the field of professional activities under the ECTS project indicated the following aspects3:

• The research of the field of professional activities gives an insight into a degree programme as a whole and facilitates better assessment of the objectives of the degree programme.

• The research of the field of professional activities provides strong arguments for upgrading degree programmes: changes in a degree programme (e.g. an updated content of the degree programme, inclusion or non-inclusion of course units in the degree programme, etc.) are made on the basis of systematically collected data; in other words, the research of the field of professional activities facilitates critical evaluation and going beyond the personal experience of the authors of degree programmes.

• The results of the research of the field of professional activities provide information on the professional activities of graduates and facilitate detailed evaluation of the contribution of studies to professional activities.

3 Two focus groups with the teachers of higher education institutions who participated in the research of the field of professional activities (members of study committees) were organised when summarising the research results. The group discussion with the teachers who participated in the research of the field of professional activities was organised and conducted by Vaida Dagytė, a member of the professional field research group.

6



• The results of the research of the field of professional activities contribute to the integration of the intended learning outcomes of course units in the objectives of an entire degree programme.

• The results of the research of the field of professional activities provide arguments for evaluating the importance of the development of relevant competences or, vice versa, facilitate the determination of competences that are not important to the future professional activities of a student.

7



1. TeAm of PRofessIonAl ACTIVITIes’ fIelD ReseARCh. BRIef sUmmARY of sTAGes of The

Professional activity field team

When performing a research of the field of professional activities, it is recommended to have a team consisting of experts and specialists of diverse professional experience. In the ECTS project, the study team consisted of the following:

• professionalfieldresearchexperts in charge of the development of the professional field research methods, the coordination of the project, the analysis of the study data and the reporting;

• professionalfieldresearchers who can gather and generalize information and materials required for the research, and draw up reports;

• professionalfieldresearchers–representativesofsubjectareagroupsoftheECTSproject(hereinafter referred to as specialised researchers) who can prepare the digest of positions held by graduates from degree programmes, also document digests, the list of information sources on professional activity and competences required for such professional activity, and compile initial lists of competences, analyse the research data, provide contact information on graduates who completed differed levels of studies, etc.;

• professional field representatives – employerswho participate – whether directly or indirectly – in the designing or consideration of degree programmes, or who are otherwise familiar with the study process and can represent a certain professional group (professional field), programmes of respective study areas, and who can be involved in the compilation of initial lists of competences and the analysis of the study data.

Representatives of the above four groups may have different functions in various stages of the research, to share different responsibilities with due consideration to the needs of the study, available sources, etc.

Brief summary of stages of the study

The study of professional activity is carried out in stages, each of which can be an independent research in itself or which can be coordinated according to the need and possibility. The following key stages of the study are recommended:

• Theanalysisofdocumentsbasedonwhichtheinitiallistsofsubject-specificandgeneric competences of study areas are developed. The set of documents to be analysed may consist of various Lithuanian and foreign documents, such as regulations of degree programmes, descriptions of the degree programmes, descriptions of a course unit, AIKOS (Open Information, Advice and Guidance System (Internet website), the Lithuanian Classifier of Professions, developed professional standards, applicable regulations of professional regulations, specialist job descriptions, descriptions of qualification requirements, the qualification framework of the sector (e.g. the qualification framework of the European informatics sector), foreign qualification standards, descriptions of foreign degree programmes, documents of the Tuning project and other material related to the analysed study area.

8



• The survey of employers, which aims at finding out the opinion of Lithuanian employers about subject-specific and generic competences of higher education institution graduates employed in their companies who completed programmes of respective study area. Instruments of the study include the developed lists of subject-specific and generic competences. Quantitative and qualitative research methods can be employed in gathering and processing of information.

• Focusgroupsofgraduates, which is advisable to carry out through focus groups with young specialists who completed programmes of respective study areas and who are practising professionally.

9



2. mAkInG lIsTs of sUBjeCT-sPeCIfIC AnD GeneRIC ComPeTenCes

A research of the field of professional activity starts with the analysis of documents. The goal of document analysis is to draw up lists of subject-specific and generic competences. Such lists are later used in other stages of the field of professional activity, such as the survey of employers and focus groups with graduates having professional experience.

Competence is the key term used for the description of the field of professional activity. The analysis of various documents regulating the field of professional activity leads to the formulation of subject-specific and generic competences that are important for the field of professional activity. Not all documents describing the field of professional activity use the concept of competence; therefore, it is important that the researchers who analyse documents know how to describe the content of the documents using the concepts of subject-specific and generic competences.

The analysis of documents consists of three stages. First of all, a list of documents containing relevant information is made, and then subject-specific and generic competences describing the field of professional activity are selected from such documents; finally, the list of selected subject-specific and generic competences is revised by uniting competences selected from various documents. Three groups of professional field researchers, i.e. professional field research experts, professional field researchers and representatives of subject area groups of the ECTS project – specialised researchers that had to upgrade degree programmes, actively cooperate in all stages of document analysis. It is worth while discussing the drawn up initial lists of competences with professional field representatives – employers.

In the ECTS project, the list of documents providing information related to subject-specific and generic competences covered several major groups of sources. Such sources may be divided into three groups.

The first major source of documents consists of various documents regulating the education system and documents of individual educational establishments. This group includes degree programmes of higher education institutions the brief descriptions of which are available in the Open Information, Advice and Guidance System (AIKOS; http://www.aikos.smm.lt/aikos/) and in Internet websites of higher education institutions; regulations of studies of respective study areas; professional training standards (http://www.kpmpc.lt/Standartai/iteisinti.html), other documents related to the activity of higher education institutions (self-evaluations of degree programmes, various studies of graduates, etc.).

The second source of documents consists of documents directly describing the field of professional activity, such as the Lithuanian Classifier of Professions (http://82.135.219.213/mod/klasifikatorius/?p=0), professional standards (e.g. European professional standards of informatics), various job descriptions, professional qualification requirements approved by the state (e.g. Lithuanian Medical Standard MN 14:2005 Family physician. Rights, functions, competences and responsibility (Official Gazette, 2006, No 3-62); a description of qualification requirements for public health care specialists providing health care services in schools (Official Gazette, 2007, No 88-3492), documents, if any, which regulate professional activity and which are approved by national and international professional associations, etc.

The third source of documents has been singled out somewhat artificially, as it covers foreign documents of the types mentioned in the first two groups of sources; therefore, such

http://www.kpmpc.lt/Standartai/iteisinti.html

http://82.135.219.213/mod/klasifikatorius/?p=0

http://82.135.219.213/mod/klasifikatorius/?p=0

10



foreign documents could be included into the first two groups of sources. They have been classified as a separate group because of two reasons. First, in spite of the widely discussed trends of globalisation, the field of professional activity has national character. Second, a set of foreign documents will inevitably be fragmented, as documents will be included by chance rather than because of the goals and objectives of a systematic study. When selecting sources, an important role shall be played by the language that researchers can read, also by the experience of professional activity researchers, the availability of documents, etc. However, in spite of its unsystematic nature, this source is important, as it provides the opportunity to check, learn and look at the performed work from outside. During the document analysis in the ECTS project the language factor determined that the most use was made of documents in English. For instance, when developing a list of subject-specific and generic competences of a social worker, U.K., Irish and Australian professional standards of a social worker have been analysed; in addition to more general standards, learning outcomes of English philology of several universities have been reviewed in order to describe competences of a philologist of English; chemistry – subject-specific competences of the degree programme in chemistry were the only ones of the study fields covered by the ECTS project to be formulated in the Tuning project, etc. (Table 1).

Table1.Examplesofdocumentsourcesofdifferentstudyareas

Englishphilology

Whathasbeendone source

The 2007 self-evaluation summary of the study field of philology of Vilnius Pedagogical University has been analysed

Internal document of Vilnius Pedagogical University

Descriptions of professional bachelor’s, bachelor’s and master’s degree programmes in English philology of Lithuanian higher education institutions have been analysed

Open Information, Advice and Guidance System (AIKOS)

Descriptions of professions falling in the professional subgroup of philologists and translators/interpreters, which correspond to professional education levels 5 and 6, have been analysed

Lithuanian Classification of Occupations (LCP)

A document on language-related industries and professions developed by the European Language Council has been analysed

Languages for language-related industries and professions: the National Report of Lithuania. European Language Council, 2004

A document on English philology studies in Lithuania developed by CQAHE has been analysed

Review report on the study field English philology in Lithuania. Centre for Quality Assessment in Higher Education

Degree programmes in English philology offered by several foreign universities have been analysed

Degree programmes in English philology offered by the University of Helsinki, Clark University (USA), Johannes Gutenberg University (Germany)

Teaching standards developed in the United Kingdom have been analysed

The UK Professional Standards Framework for teaching and supporting learning in higher education

11



Chemistry


Various university degree programmes in chemistry have been analysed

Bachelor’s and master’s programmes of Vilnius University, Kaunas University of Technology and Vilnius Pedagogical University

Descriptions of bachelor’s and master’s degree programmes in chemistry and applied chemistry of Lithuanian higher education institutions have been analysed


The standard of training a technologist of chemical processes has been analysed

The Ministry of Education and Science of the Republic of Lithuania, the Ministry of Social Security and Labour of the Republic of Lithuania (2008)

Descriptions of professions falling in the professional subgroup of chemists (Code 2113), which correspond to professional education levels 5 and 6, have been analysed

Lithuanian Classification of Occupations (LCP)

Documents of the Tuning project have been analysed

Tuning. Reference points for the design and delivery of degree programmes in chemistry. Publicaciones de la Universidad de Deusto, 2009; Tuning. Tuning chemistry subject area brochure. European Chemistry Thematic Network, 2008

Socialwork


Descriptions of degree programmes of three study cycles in social work of Lithuanian higher education institutions have been analysed


Descriptions of various specialisations of the profession of a social worker have been analysed Lithuanian Classification of Occupations (LCP)

The regulation of social work studies has been analysed The Ministry of Education and Science of the Republic of Lithuania (2008)

Documents of the self-evaluation of master’s programme in social work have been analysed

Internal document of Vilnius Pedagogical University

The standard of training a social worker has been analysed

Methodological Centre for Vocational Education and Training (2008)

The Directory of Social Services has been analysed The Ministry of Social Security and Labour of the Republic of Lithuania (2006)

U.K., Irish and Australian standards of training a social worker have been analysed

It is important to note that these lists were made in cooperation. Study experts were aware of the variety of documents regulating the field of professional activity, professional field researchers had skills on analysing such documents and were familiar with documents regulating the field of professional activity, while specialised researchers assessed their relevance to the design and upgrade of degree programmes. Study experts and professional field researchers provided

12



the following documents regulating the field of professional activity to specialised researchers for document analysis: descriptions of professions provided in the Lithuanian Classification of Occupations, descriptions of degree programmes published in the Open Information, Advice and Guidance System, descriptions of various jobs, while specialised researchers shared the experience of their foreign colleagues and documents regulating professional activity of national and international professional associations.

The procedure of making a list of documents regulating the field of professional activity can be summarised as follows: different groups of researchers, study experts, professional field researchers and specialised researchers cooperate and develop a list of documents regulating the field of professional activity, which can be considered as being rather exhaustive or at least as containing more information than one group of researchers initially had on documents regulating the field of professional activity.

When making a list of documents, it is important that in cooperation of all researchers each group substantially broadens its knowledge about documents regulating the field of professional activity and that the set of documents provided for further analysis of the field of professional activity be as comprehensive as possible.

Once lists of documents regulating the field of professional activity were ready, the work was taken over by professional field researchers. It was them who reviewed all documents and selected information based on which the initial lists of subject-specific and general competences were drawn up. The lists were further analysed by specialised researchers and experts of the Tuning project who were international consultants of the ECTS project.

Table2.Changeinthenumberofsubject-specificcompetencesduringtheanalysisofdocuments

Studyfield Initialnumberofsubject-specificcompetences

Finalnumberofsubject-specificcompetences

Subject-specificcompetencesremainingin

thefinallist(%)

Chemistry 62 28 45

English philology 79 23 29

Art 50 26 52

Informatics 57 24 42

Medicine (general practitioners) 110 23 21

Music 46 (73 in Version 2) 30 41

Public health 106 18 17

Social work 110 38 35

Theaveragenumber 35

Table 2 shows the change in the number of subject-specific competences during the analysis of documents regulating the field of professional activities. In all cases, the initial list of subject-specific competences was the longest. On average the final list of the subject-specific competences includes 35% of the subject-specific competencies of the initial list.

The comparison of lists of competences shows that the initial list, which was fragmentary, random and full of repetitions, has been transformed into the conceptual and justified final list of

13



competences. This improvement of the list was an important intellectual exercise for the entire group of researchers4. During the intense analysis, which lasted for over a month, the analysis, synthesis and classification of competences has been mastered.

Table 3 shows the change in competences related to teaching chemistry in higher education institutions and in secondary schools. The initial list features as many as 13 competences describing this activity. Professional field researchers who compiled this list were not competent to decide whether the organisation of activity outside school time, the knowledge of the teacher’s work, the ability to prepare textbooks, the knowledge of child and teen psychology, etc., should be included into the list of subject-specific competences. After all, they analysed documents from the approved list.

Table3.Changeofchemistryteaching-relatedsubject-specificcompetencesofthechemistrysubjectarea

Firstlistofsubject-specificcompetences

To be able to teach the subject, to develop the skills of chemistry knowledge of schoolchildren and students

To be able to organise work in class and exercises in class

To be able to examine and assess the knowledge of schoolchildren and students

To be able to organise activity outside school time

To be able to draw up teaching plans

To be able to communicate with colleagues, parents and schoolchildren

To train students for final theses and examinations

To prepare students and schoolchildren for future studies and career

To prepare textbooks

To know the methodology of the teacher’s work

To be able to select and apply the methods of the teacher’s work

To know the subject of teaching

To have knowledge of child and teen psychology

Secondlistofsubject-specificcompetences

To be able to teach chemistry in general education schools

To be able to teach chemistry in higher education institutions

To be able to prepare and apply methodological materials and guides

To know the methodology of the teacher’s work

To know the subject of teaching

Thirdlistofsubject-specificcompetences

Ability to teach chemistry in general education schools

Ability to prepare and apply methodological materials and guides in chemistry

Source: Results of the professional field research of chemistry study area: Guidelines for upgrading degree programmes (http://www.ects.cr.vu.lt/Files/File/Chemija_tyrimu%20technine_ataskaita.pdf).

4 More detailed information about the improvement process of competences is available at the Internet website of the ECTS project. The results of the research of the field professional activities are available under Project results at the website (http://www.ects.cr.vu.lt/Projekto_rezultatai). Each of the eight study reports contains chapter Analysis of documents of professional activity, which provides the lists of analysed documents and at least two tables of the lists of competences – one containing the initial lists of competences and the other – the final lists of competences.

http://www.ects.cr.vu.lt/Files/File/Chemija_tyrimu technine_ataskaita.pdf

http://www.ects.cr.vu.lt/Files/File/Chemija_tyrimu technine_ataskaita.pdf

http://www.ects.cr.vu.lt/Projekto_rezultatai

14



It is important to note that the initial list of competences is not surplus information. The final list of competences covers and integrates the entire experience of drawing up the list, thus also including competences that were later omitted. If there is a need to specify competences of the final list in more detail, the reconstruction of the procedure of drawing up the final list should be useful – a major part of the omitted competences would become elements of competences of the final list.

Initial lists of competences were made with reference to national, foreign and international sources. National sources provide information that describes professional activity, which is inconsistent and in need of systematisation. Furthermore, the major part of information contained in such documents is irrelevant; therefore, it was criticised and corrected during discussions with specialised researchers. The experience of other countries helped systematise national information. Such cooperation with specialised researchers and reference to better developed documents resulted in the second list of subject-specific and generic competences. This list was discussed with experts of the Tuning project. The discussions led to the development of the third list of subject-specific and generic competences, which was employed in further professional field studies.

The specification of competence lists was aided by all experts of the Tuning project who participated in the ECTS project and have vast international experience. Thus, the national lists of competences of the field of professional activity integrated the international experience in line with the priorities of the common European higher education area.

The summary of the document analysis performed within the framework of the ECTS project reveals several points:

First, professional field researchers noted that information contained in national documents directly regulating the field of professional activity is inconsistent and in it needs to be systematised more than information contained in international documents. A major part of information stated in national documents is dated and irrelevant. Information contained in foreign documents was valuable for the analysis of professional activity and its synthesis into individual activity areas, which were later united into competences by specialised researchers.

Second, the descriptions of professions provided by the Lithuanian Classification of Occupations and the descriptions of degree programmes provided by AIKOS differ from European sources; the formulation of competences lack points that are more global and related to more general things (socially responsible activity, equal opportunities, etc.). In comparison with sources of other European universities, the Lithuanian sources focus more on subject-specific competences, yet they discuss generic competences to a lesser degree.

Third, professional standards that regulate the educational system were more of a benefit. They contained information that was better structured, was more explicit, detailed and more useful for the drawing up of lists of subject-specific and generic competences. The experience of other European universities and the Tuning project was especially valuable. These sources were especially systematic and exhaustive.

Four, the analysis of documents regulating the field of professional activity was carried out by combining individual work and group discussions, which was a big advantage. Discussions revealed that the lists of competences, which were drawn up individually and based on the analysed documents, could be specified in more detail. Group discussions not only helped better understand the core and the correlation of individual competences, but also set the guidelines

15



for further work. Such methods of document analysis revealed the following advantages of the analysis: a shorter duration of the analysis of documents, faster assimilation of information, accelerated grouping of synonymous competences, more efficient group work.

However, if the work of any group of researchers was not so good, it influenced the joint result. For instance, the failure of specialised researchers to participate made it more difficult to identify areas of professional activity, to reasonably group competences or to assess which competences were more important than others. This confirms again that the analysis of the field of professional activity is a complex study requiring subject-specific knowledge, the knowledge of how to carry out a social research, as well as purposeful cooperation between specialised researchers and professional field researchers, which alone can ensure as adequate reconstruction of the field of professional activity as possible.

each committee of the deGree proGramme can:• draw up the list of the most important national and international sources defining

the field of professional activity and related to the degree programme and its graduates;

• analyse information contained in the sources by singling out competences to be developed, summarise and classify such competences;

• discuss the developed list of competences with a most wide circle of academics and representatives of the professional activity field, i.e. with professors, members of the research staff and employers;

• draw up the final list of competences developed by the degree programme.

16



3. sURVeY of emPloYeRs

The goal of the survey of employers is to find out the opinion of Lithuanian employers about subject-specific and generic competences of higher education institution graduates employed in their companies who completed certain degree programmes.

This goal can be achieved with the help of both quantitative and qualitative research methods. Subject to the selected research strategy, the results and the nature of conclusions, which can be drawn from the research data, will differ.

When conducting the quantitative research, i.e. the representative survey of employers according to the standardised questionnaire, answers to the following questions of the designers of degree programmes may be given: which subject-specific and generic competences the employers point out as being important for the activities of their employees, and which are considered to be less important; which competences the employers agree on, and regarding which competences their opinions differ; whether the employer’s assessment of the relevance of competences is subject to the area of his/her company’s operations; how employers assess the level of the competence possessed by the employee when starting his/her work at the company; whether the assessment given by the employer is subject to the qualification (professional bachelor, bachelor or master) acquired by his employee. The results of the quantitative research are numerical values of the evaluation of subject-specific and generic competences. In turn, the designers of degree programmes may employ these results for justifying the list of competences used for the definition of learning outcomes. Such research allows drawing a conclusion whether and to what extent the list of competences drawn up by the designers of degree programmes and used by them when designing or upgrading degree programmes is accurate.

The qualitative research, which may employ the methods of a qualitative interview or a focus groups, may also reveal the opinion of employers regarding which subject-specific and generic competences the specialists employed in their companies need. The qualitative research allows drawing up as exhaustive a list of subject-specific or generic competences as possible, and being sure that no competence of importance for the field of professional activities, which can be developed by the degree programme, was omitted from the initial list of competences discussed with the employers during the research. However, unlike the application of the methods of the quantitative research, this research will not allow drawing conclusions regarding quantitative differences of evaluation of competences and ranking competences according to the importance attributed to them by employers. Due to a small sample of employers participating in the qualitative research, each competence included into the list, each proposal and each opinion of the employers will have equal weight. Conclusions about the trends of the field of professional activity may be provided only as assumptions the validity of which could be proven by a subsequent quantitative research.

Designers and/or deliverers of degree programmes may choose to what extent they wish to study the opinion of employers. Several approaches can be employed in this respect, viz. both quantitative and qualitative, or either quantitative or qualitative research can be performed. Also, several strategies may be used when combining research methods: researchers may perform a qualitative research during the preparatory stage of a quantitative research in order to develop a more accurate instrument for measuring (i.e. the questionnaire), or a qualitative research may be performed after a quantitative research in order to get better understanding of the obtained results.

17



A quantitative survey of employers was performed within the framework of the ECTS project during which employers were provided with lists of subject-specific and generic competences developed by the researchers from the study programmes (representatives of different subject areas) participating in the project jointly with foreign experts. The qualitative interviews of employers were conducted during the final stage of the research in order to better understand the results of both the quantitative survey of employers and focus groups of graduates.

The goal of the quantitative survey of employers performed within the framework of the ECTS project was to find out the opinion of Lithuanian employers about subject-specific and generic competences of higher education institution graduates employed in their companies who completed programmes of eight study areas.

The research covered the following study areas: 1. English Philology, 2. Social Work, 3. Informatics, 4. Chemistry, 5. Public Health, 6. Medicine, 7. Arts 8. MusicWith due consideration to its goal, the distinguishing feature of the research of the field

of professional activities lies in the fact that when researching the opinion of employers it is impossible to perform a single representative survey of employers of a certain country using the questionnaire, which is the same for all employers, the results of which could be applied to the designing of different degree programmes. Subject-specific competences are subject to a specific area or areas of professional activity, which serve as targets when training specialists. It is impossible from the methodological point of view to develop a single comprehensive list of competences suitable for all areas of activity from which employers would select competences relevant to the operations of their company. Therefore, the research of the opinion of employers regarding subject-specific and generic competences that should be developed by a certain degree programme will inevitably include two tasks:

1. the definition of a sample of potential employers of graduates who completed degree programmes of a certain study area, e.g. who are the employers of graduates who complete degree programmes in English Philology in Lithuanian higher education institutions (translation services, general education schools, companies providing tourist services, governmental institutions, etc.);

2. the development of a questionnaire customised for the employers of graduates completing degree programmes of a certain study area.

During the implementation of the ECTS project, eight independent samples of employers were interviewed using a questionnaire adapted to each specific study area.

A survey agency carried out the survey using the methods developed by research experts of the ECTS project.

Employers were surveyed using a face-to-face interview method and Internet survey. A respondent was asked to do a face-to-face interview first; if the respondent objected or if she/he requested so, she/he was provided with an e-questionnaire.

18



A properly formed sample of respondents is one of the main conditions of the quality of quantitative research data. The representativeness is the key quality criterion of sampling. A sample shall be considered to be representative if the survey of sample units (in this case employers selected for the research) yields similar results to those that would be obtained if the entire population (in this case all potential employers of graduates of a certain study area) is surveyed.

There are different ways of sampling; however, not all of them provide the same guarantee regarding the representativeness of the sample. The representativeness of the sample is best ensured by probability sampling. However, the application of such sampling requires a comprehensive and accurate list of population. Unfortunately, a list grouping national companies according to study areas of employed graduates is not available to researchers, and the development of such list would have increased the cost of research considerably. Non-probability sampling has been chosen during the implementation of the ECTS project. Just as in the case of probability sampling, the quota sample is simulated in order to make it correspond to the population with regard to the parameters of importance for the research, and that the sample survey would allow drawing conclusions pertaining to the entire population. However, unlike in the case of a probability sample, there is a risk of the researcher having inaccurate data on population parameters that would be used for the quota sampling, which means that the sample will not be representative of the entire population. This must be taken into consideration when analysing data and formulating conclusions of the research.

The ECTS project used four criteria of the quota definition for the sampling of employers, viz. the degree programme completed by employees (the qualification of employees), the area of corporate operations, the sector of corporate operations, the location of corporate headquarters.

It is also important to note that sampling for the survey was extended only to companies and institutions that have employees with qualification degrees of a professional bachelor, bachelor or master from a certain study area acquired in higher education institutions not earlier than five years ago (i.e. degree programme graduates who graduated in 2005-2009).

Employers or employer representatives (direct managers or persons informed about subject-specific and generic competences of the employee) from the selected companies were surveyed.

The questionnaire consisted of the following four question groups:• factual questions about the employee and the company;• the assessment of the general employee’s qualification acquired at higher education

institution;• subject-specific competences (the importance of each competence to professional

activity and the employee’s level of each competence at the start of employment);• generic competences (the importance of each competence to professional activity and

the employee’s level of each competence at the start of employment).Questionnaires of the Tuning project were employed for the development of questions

aimed at finding out the employers’ assessment of the importance of subject-specific and generic competences to professional activity and the employee’s level at the start of employment. The wording of these questions is presented in Fig. 1.

19



Q1. A list of subject-specific skills and capacities that may be useful for professional activity (e.g. of a chemist) is provided below.

Please asses each capacity:Q1_A. Is this capacity completely unimportant, unimportant, important or very

important for the employee’s professional activity in your company?Q2_B. What was the level of this capacity that the employee had at the start of

employment at your company: the capacity was non-existent, low, average or high?

instructions for the person conductinG the poll: please shoW card 1 featurinG the list of suBJect-specific sKills and capacities.

please asK Q1_a and Q1_B reGardinG each capacity.

Skillsandcapacities

Q1_aImportanceofthecapacitytotheemployee’sprofessionalactivityinthecompany

Q1_B Capacitylevelofthe

employeeatthestarofhis/heremploymentatthecompany

Com

pletely

unimportant

Unimportant

Important

Very

important

Non-existent

Low

Average

High

1 Competence 1 1 2 3 4 1 2 3 42 Competence 2 1 2 3 4 1 2 3 43 ... 1 2 3 4 1 2 3 4

Q2. A list of generic skills and capacities that may be useful for successful professional activity of a graduate from a higher education institution (e.g. interpersonal competences, technological skills, language skills, cognitive capacity) is provided below. Please asses each capacity:

Q2_A. Is this capacity completely unimportant, unimportant, important or very important for the employee’s professional activity in your company?

Q2_B. What was the level of this capacity that the employee had at the start of employment at your company: the capacity was non-existent, low, average or high?

instructions for the person conductinG the poll: please shoW card 2 featurinG the list of Generic sKills and capacities.

please asK Q2_a and Q2_B reGardinG each capacity.

Skillsandcapacities

Q2_a Importanceofthecapacitytotheemployee’sprofessionalactivityinthecompany

Q2_B Capacityleveloftheemployeeatthestarofhisemploymentat

thecompany

Com

pletely

unimportant

Unimportant

Important

Very

important

Non-existent

Low

Average

High

1 Competence 1 1 2 3 4 1 2 3 42 Competence 2 1 2 3 4 1 2 3 43 ... 1 2 3 4 1 2 3 4

Fig.1.Wordingofquestionsaboutsubject-specificandgenericcompetences

20



The assessment of subject-specific competences included in the survey of employers conducted within the framework of the ECTS project allowed ranking these competences according to their importance to professional activity of employees and the preparedness of employees for employment5. In turn, the designers of degree programmes could employ these data, on one hand, to justify the need to include or omit the development of this or that subject-specific competence by the degree programme, and on the other hand – to consider which competences are regarded by employers as being less irrelevant.

each committee of the deGree proGramme can:• conduct a quantitative survey of employers with the help of survey agencies

or persons experienced in conducting and qualified to conduct social research, provided there are sufficient financial resources. The committee of the degree programme must provide the persons conducting the research with the list of competences that it wants to assess. The cooperation with other committees of degree programmes in the same study area in the same or another higher education institution is recommended in order to reduce financial costs;

• arrange a focus group or conduct qualitative interviews with the employers of the graduates of degree programmes and with representatives of their associations, if any, with the help of persons experienced in conducting and qualified to conduct social research. The focus group should be limited to the maximum of 10 representatives of employers (the recommended size of the group is 6-8 participants) with whom the developed list of competences and the preparedness of graduates for their professional activity could be discussed.

5 Detail results of the survey of employers are provided by the reports of the professional field research available at the website of the ECTS project http://www.ects.cr.vu.lt/Projekto_rezultatai


21



4. foCUs GRoUPs wITh GRADUATes

Focus groups with graduates of study areas aim at revealing the graduates’ opinions and assessment of degree programme and learning outcomes, their application in professional activity and in the labour market. They also seek to discus subject-specific and generic competences of graduates that they acquired during studies, the need or absence of certain professional knowledge, skills and capacities, proposals regarding the upgrading of the completed degree programmes.

It is recommended to invite the following professionally active graduates of degree programmes to attend focus groups:

• graduates who graduated not earlier than five years ago;• young specialists engaged in professional activity and holding various positions in

different sectors;• graduates of degree programmes of the same level (professional bachelors, bachelors

or masters).8-10 graduates of degree programmes can take part in the focus group. The specialised

researchers (who compile information about further professional activity of graduates) will have the best chances to ensure a constructive contact with graduates and their interest in committing some of their time for discussions in the focus groups.

The results of the survey of employers as well as the lists of subject-specific and generic competences applied in the professional activity research can be used in discussion.

In the ECTS project it was the professional field researchers who developed guidelines for the focus groups and moderated discussions with due consideration to the progress of the research and interim results. The key topics and issues discussed with graduates of degree programmes of the surveyed study area were as follows:

• the assessment of the preparedness for employment and professional activity (the assessment of degree programme and learning outcomes);

• expectations of employers regarding graduates of certain degree programmes: on what competences the employers focused when hiring; what competences the employers demand from their newly hired employees;

• knowledge, skills and capacities acquired on-the-job: what competences were acquired on-the-job; whether they think it would have been better to acquire these competences at the higher education institution; whether it is possible to acquire these competences at the higher education institution, or whether they are specialised competences that can and must only be acquired on-the-job;

• lacking knowledge, skills and capacities: what competences they missed or currently miss at work;

• proposals for the upgrading of degree programmes of the study area: what proposals they have regarding any supplementation/improvement of the degree programme that they themselves completed; maybe course units should be supplemented with new topics, or the degree programme should be supplemented with new course units, maybe the methods of teaching should be slightly amended; what competences should get more attention according to their opinion;

• the discussion of lists of subject-specific and generic competences: which competences are required in their employment, and which competences are less relevant; whether they think that during their studies they acquired the required and relevant competences.

22



Data of the qualitative section of the professional field study carried out within the framework of the ECTS project provided valuable information about the results of degree programmes in the labour market6. Even though it was difficult for all participants of focus groups to discuss degree programmes at the competence level, the developed lists of subject-specific and generic competences provided the discussion with a certain direction and content, and they were like a tool for structuring the discussion. It was relatively easy for graduates of degree programmes to identify knowledge and skills that they miss in the labour market or in professional activities, and to state the areas in which they feel professionally competent. Recent graduates are able to constructively reflect on the expression of degree programme outcomes within the context of their professional experience. Data from this stage of the professional field study complemented the results of the quantitative survey of employers with the graduates’ perspective: they assessed degree programmes within the context of both further education and work experience, and discussed competences of relevance to their professional activity. It should be noted that graduates were inclined to attribute relatively more relevance to the development of generic competences during studies.

each committee of the deGree proGramme can:arrange a focus with graduates of degree programmes with the help of persons

experienced in conducting and qualified to conduct social research. The focus group should be limited to the maximum of 10 graduates (the recommended size of the group is 6-8 participants) with whom the developed list of competences and the preparedness of graduates for their professional activities could be discussed. It is recommended to invite a person that the graduates are not familiar with to moderate the discussion.

6 Data of the focus groups with graduates are provided by the reports of the professional field research available at the website of the ECTS project http://www.ects.cr.vu.lt/Projekto_rezultatai. Reports of the professional field research summarise the insights and results of focus groups held with graduates of degree programmes of one study area.


puBlished By Vilniaus universitetas

Universiteto str. 3, LT-01513 VilniusTel.: (8 5) 268 7001, fax: (8 5) 268 7009

E-mail: [email protected], www.ects.cr.vu.lt

prepared By Publishing company „Kriventa“V. Pietario str. 5-3, LT-03122 VilniusTel./fax: +370 5 265 0629E-mail: [email protected]

www.ects.cr.vu.lt