ΥΠΟΕΡΓΟ 2: Επιμόρφωση Αξιολογητών και Ανάπτυξη Εκπαιδευτικού Υλικού για την Εξ αποστάσεως Εκπαίδευση Εξεταστών της Αγγλικής

INTRODUCING THE KPG

SPEAKING TEST

NEW ORAL EXAMINER INFORMATION PACK

APRIL 2012

2

1. AN OVERVIEW OF THE KPG EXAMINATION SYSTEM

1.1 Background information

The KPG (Kratiko Pistopiitiko Glossomathias) is a state certification system of language proficiency established on the basis of a 1999 law. The Law was put into effect in 2002 and the first exams (at B2 level) were run in April 2003.

The KPG examination battery conforms to the principles of testing, considering local and international research findings, as well as the principles of the Common European Framework for Languages (CEFR) of the Council of Europe. Thus, it is viewed as a glocal rather than a local or global examination system. The KPG exam battery views all languages as equal and designs the exams in all of its languages on the basis of common specifications. Moreover, the exams’ global scale descriptors and language use descriptors relate to those of the CEFR.

The Ministry of Education, Life Long Learning and Religious Affairs is the legal copyright owner of all documents containing information about the KPG assessment system and is responsible for the administration of the exams.

The system has been designed and developed by groups of foreign language specialists in the language departments of the universities of Athens and Thessaloniki, selected to carry out the project and relevant research, and who are appointed by the Ministry of Education.

Development of and research for the KPG exams in English is carried out by the Research Centre for English Language Teaching, Testing and Assessment (RCEL) of the Faculty of English Studies, School of Philosophy, University of Athens under the direction of Prof. B. Dendrinos.

Presently, candidates can take exams in English, German, Spanish and Turkish (developed by groups of experts at the University of Athens) and French and Italian (developed by groups of experts at the University of Thessaloniki). In the future exams will be designed for other European languages as well.

Thus, the bodies involved in the design and implementation of the system are: o The Ministry of Education, Life Long Learning and Religious Affairs o The Directorate for the Certification of Foreign Language Proficiency o The Central Examination Board (appointed by the Ministry) o The Departments of Foreign language and Literature of the University of Athens and the

University of Thessaloniki. Since March 2007, the University Departments involved have been awarded European funding for

the further development of the KPG examination system in order to ensure the sustainability of the system in the future, the training of oral examiners and script markers and many other projects.

1.2 Approach to language use adopted by the KPG exam battery

All the KPG exams adhere to a functional approach to language use and set out, throughout all the modules, to evaluate socially-purposeful language knowledge and literacy. The view of language that the KPG exam battery, and in particular the examinations, adhere to is that language is a social phenomenon, and that to use language for a variety of purposes means to take part in social practices. Language is a dynamic, interactive, social phenomenon between the speaker and listener (or the reader and writer). We convey meanings not by single, isolated sentences which occur in a vacuum, but by more complex exchanges produced in a specific context in which the participants’ beliefs, expectations, the knowledge they share about each other and of the world, and the situation in which they interact play a crucial part and determine the linguistic choices they will

3

make. Language is not used to express grammar or vocabulary; it is used to perform or fulfil certain functions or purposes (e.g. to agree or disagree, to invite, to congratulate, to advise, to promote, to convince, to request, etc.) and the functions we want to perform will determine the shape of the text that emerges as we communicate with one another. As far as the KPG exams in English are concerned, it should be noted that English is viewed as a pluricentric language and the KPG exams deal with World Englishes. As a result, oral and written texts are not exclusively in British or American English but may be, for example, in Australian or Canadian English.

1.3 Overall aims of the KPG examination system

The KPG exams represent a “proficiency assessment” (rather than diagnostic or competences measurement) examination system which aims to test the candidates’ ability to make socially purposeful use of the TL in Greece and abroad. The KPG exams aim to measure candidates’ competence in comprehending and producing oral and written discourse, as well as their ability to act as mediators across languages and their awareness of how the target language works to create socially purposeful meanings. In other words, they aim at assessing how well candidates use the language to understand oral and written texts or to produce socially meaningful messages and not at assessing their knowledge about the foreign language, about grammar and vocabulary. More specifically, depending on the level of the exam, KPG candidates are expected to show that they can use the target language with different degrees of competence to:

• comprehend authentic written and oral texts,

• express themselves in standard forms of English using written and oral forms of contextually appropriate language,

• make linguistic choices which presuppose the development of language awareness,

• perform mediation tasks and relay information from Greek into the target language,

• understand the rubrics of the exams and be able to carry out exam tasks as required.

1.4 Levels certified in the KPG examination system

The levels certified are:

• A1-A2 (Basic User), • B1-B2 (Independent user) and • C1 (Proficient user). C2 (Proficient user) is not available at the moment, but will be introduced

in November 2012. The primary objective of the KPG exam at the B1, B2 and C1 levels is to respond to candidates’ social needs, by granting them certification which can be of use for occupational purposes. Certification at levels A1 and A2 aims mainly at assessing knowledge and skills developed by candidates in the course of their language training within or outside the state educational system. The A1/A2 and B1/B2 levels are examined through integrated-graded exams, which contain an equal number of items at each level. C1 level is examined on its own but will be examined together with C2 through an integrated-graded exam as of November 2012.

4

1.5 Candidate profile

KPG exams at B and C levels are designed for candidates 15+, who are EU citizens, and candidates from other countries who are living, studying and/or working in Greece, whereas the A1 and A2 levels are mainly addressed at young learners, although older candidates also sit for the exam at these levels. Greek is considered the common language of the candidates since they are required to understand instructions and rubrics in Greek, for the A level exam, and to function in the role of mediator transferring information from Greek in the TL for the B and C1 level exams. The KPG exam system takes additional measures to help candidates with special needs. It adapts the tests to cater for different types of learning difficulties and disabilities, depending on the candidate applications, and provides two special Examination Centres, one in Athens and one in Thessaloniki. Examiners trained in dealing with special needs are used in the two examination centres and specially trained script raters are also used in the Rating Centres to grade the written scripts of candidates with special needs.

1.6 Overall Structure of the KPG exams

Each exam, regardless of level and language, consists of 4 modules: Module 1 tests Reading comprehension and Language Awareness. Module 2 tests Writing production and written mediation skills (no mediation activities for A1/A2 level). Module 3 tests Listening comprehension. Module 4 tests Speaking production and oral mediation skills (no mediation activities for A1/A2 level).

1.7 The KPG examination system and other language certification systems

The KPG examination system resembles other national/state language systems for the certification of language proficiency (e.g. the Finnish National Language Exam System). However, it has its own characteristic features. As this is a national exam system, with validity in the Greek and the European social context, it differs markedly from large-scale international examination systems (such as the Cambridge ESOL or the Michigan English exams).

• It does not have commercial interest and it is subsidized by the state.

• There is no interest in material profit which is often the case with international exams concerned with assessing and certifying proficiency in one of the “strong” European languages.

• It is not addressed to a “global” audience but to candidates who live in Greece and need to use English in Greece and abroad for a variety of purposes.

• It has common specifications for all the languages examined.

• It presupposes that candidates have good knowledge of the Greek language and reality.

• It considers non-native like pronunciation (in listening texts and candidates’ oral production) as acceptable as long as the language produced is intelligible and communication is not impeded.

• It is the only examination system which assesses candidates’ mediation skills, an aspect included in the Common European Framework of Reference.

5

• There is a greater variety of discourses, genres and registers than in the tests of other exams.

• A variety of multimodal texts are used in the exams which require distinctly different test items than those of monomodal texts.

• A variety of authentic texts recorded for the exams are used to test listening comprehension.

1.8 KPG and formal education

KPG is linked with school education:

• It is linked directly with the National Foreign Languages Curriculum, language exams in schools will be similar in type to the KPG exams, and KPG preparation has already been introduced in school support programmes. To view the National Foreign Languages Curriculum visit: http://rcel.enl.uoa.gr/xenesglosses/.

• It aims to have a positive backwash effect on language teaching/learning practices in schools, it shapes attitudes to language and language learning and it motivates the development of strategies for speech comprehension and production.

• In the future, it will also be linked with tertiary education, adult education, ‘Second chance’ secondary schools and workplace education.

1.9 Further information about the KPG examination system

1.9.1 Information and materials available on-line

Interested parties may find on-line, at the Ministry of Education’s site: http://www.kpg.minedu.gov.gr/, documents about:

• the KPG exam system and its implementation,

• the rules and regulations regarding the examination system,

• how to register for the exams and how to apply to become a KPG assessor,

• the Common KPG Framework for all languages,

• specifications for the A1, A2, B1, B2 and C1 level (soon also for the C2 level),

• sample exams and past papers in all the examined languages. For information in English about the exams, the site http://rcel.enl.uoa.gr/kpg/ of the Research Centre for English Language Teaching, Testing and Assessment (RCEL) of the University of Athens provides:

• documents about the KPG exam system and its implementation, • the KPG exam specifications, • KPG English exam past papers, • information about the training of the oral examiners and script raters of the KPG English

exams, • oral examiner and script rater training materials, including exam simulations, and • information about research, publications and KPG funded projects.

1.9.2 Publications available for language teachers preparing candidates and for candidates themselves

6

Most of the information and materials on-line have also appeared in booklets published by the Ministry.

Video recordings have been made for the preparation of orals examiners.

Commercial local and international publishers have produced books for the preparation of KPG exam candidates.

Handbooks for KPG examiners and script raters have been produced by the KPG teams and published by RCEL.

1.9.3 Future developments of the KPG exam system

The C1/C2 integrated-graded exam will be introduced in November 2012.

In the near future, a KPG web based platform will be made available. It will address candidates, parents, teachers and examiners offering information tailored to each group, and providing past papers, exam preparation tips and materials, examiner training materials, etc. KPG will soon be able to have web based communication with interested parties and e-training for its evaluators through a new sophisticated portal.

Finally, KPG will soon be making available an on-line diagnostic test, and, in the future an e-test version to supplement rather than replace the pen-and-paper exam.

2. OVERVIEW OF THE EXAMS In this section, an overview of the structure of the exams per level is presented. Each exam consists of 4 modules or tests, Reading comprehension, Writing, Listening comprehension and Speaking. Each module aims at assessing specific communicative uses of the language. Past papers from each exam can be found at http://rcel.enl.uoa.gr/kpg/, the site of the Research Centre for English Language Teaching, Testing and Assessment (RCEL) of the University of Athens.

2.1 OVERVIEW OF THE A LEVEL EXAM

Module 1: Reading comprehension This module tests candidates’ (a) ability to understand written (multimodal) texts, and (b) language awareness with regard to lexical and grammatical elements as used in utterances and brief texts. The test consists of 50 items, 40 multiple choice (20 for A1 and 20 for A2) and 10 short answer items (5 for A1 and 5 for A2).The 40 multiple choice items are awarded a maximum of 40 points (1 mark per item) and the 10 short answer items are awarded a maximum of 10 points (1 mark per item), the total being 50 marks. The duration of this test is 65 minutes. Module 2: Writing This module tests the candidates’ ability to produce messages and short texts in writing, given instructions and cues. Writing at this level is very controlled. It consists of 4 activities, 2 activities for A1 and 2 for A2. The first A1 activity (1.1) and the first A2 activity (2.1) are marked with a maximum of 6 points, whereas the second A1 activity (1.2) and the second A2 activity are marked with a maximum of 9 points. The maximum total marks the candidate can obtain from all four written activities are 30 points from marker A and another 30 points from marker B, in other words a maximum of 60 points. The duration of this test is 40 minutes. Module 3: Listening comprehension This module tests candidates’ ability to understand spoken language which is linguistically simple. The messages candidates are asked to listen to are on predictable, everyday topics, and they are all studio-

7

recorded so that speech is slow and clear –with no background noise interfering. Candidates always have the opportunity to listen to the recording at least twice before responding. This test consists of 10 multiple choice items (5 for A1 level and 5 for A2 level) and 10 short answer items (5 items for each level). All 10 items of the test are awarded a maximum of 50 points (2 marks per item for the multiple choice items and 3 marks per item for the short answer items). The duration of this test is 20 minutes. Module 4: Speaking This module tests candidates’ ability to deliver a message orally and specifically, (a) to respond to personal questions (two questions for each level, i.e., four questions in total), (b) to describe or talk about something that s/he sees a picture (two questions for each level, i.e., four questions in total) and (c) to answer questions about one or more multimodal texts belonging to the same thematic category (2 questions for A1 level) and to ask questions relating to missing information in one of the multimodal texts (3 questions for A2 level). The duration of this test is 20 minutes. The minimum score for achieving the A1 level certification is 60/200, whereas that for the A2 is 120/200. In modules 1, 2 and 3 of the A level exam, all instructions are provided in both English and Greek. Additionally, one activity in module 1 and one in module 3 check candidates’ reading and listening comprehension, respectively, through the use of items written in Greek.

2.2 OVERVIEW OF THE B LEVEL EXAM

Module 1: Reading comprehension Module 1 tests reading comprehension skills and language awareness. It aims at assessing candidates’ understanding of the overall or partial meanings in a text, to make inferences or intelligent guesses on the basis of the text. Moreover, some items are designed to assess their ability to make language choices that are correct and appropriate to the linguistic, discursive and social context. The test consists of 60 items, 50 multiple choice items (25 for B1 and 25 for B2) and 10 short answer items (5 for B1 and 5 for B2).The 50 multiple choice items are awarded a maximum of 40 points (0,8 marks per item) and the 10 short answer items are awarded a maximum of 10 points (1,0 marks per item). The duration of this test is 85 minutes. Module 2: Writing This module tests candidates’ ability to produce written discourse and function as mediators through written production. It consists of 4 activities: 2 activities for written production (1 for B1 and 1 for B2) and 2 activities for written mediation (1 for B1 and 1 for B2) based on a Greek text. Just like the integrated-graded A level exam, the candidate is required to do all four activities if she/he wants to be certified for B2 level proficiency. The candidate who wants to be certified for B1 level language proficiency only needs to complete the activities marked as B1. The first activity, which is of B1 level (B1.1), and the second activity, which is of B2 level (B2.1), are marked with a maximum of 7 points each, whereas the third activity, which is a B1 level mediation activity (B1.2), and the fourth activity, which is a B2 level mediation activity (B2.2), are marked with a maximum of 8 points each. The maximum total marks the candidate can obtain from all four written activities is 30 points from marker A and another 30 points from marker B. In other words, the successful candidate can obtain a maximum of 60 points for module 2. The duration of this test is 80 minutes. Module 3: Listening comprehension This module aims to assess candidates’ ability to understand standard spoken language on both familiar and unfamiliar topics normally encountered in personal, social, academic and vocational life. Candidates always have the opportunity to listen to the recordings twice before responding. This test consists of 15 multiple choice items (7 for B1 level and 8 for B2 level) and 10 short answer items (5 items for each level). All 25 items of the test are awarded a maximum of 50 points (2,0 marks per item). The duration of this test is 25-30 minutes. Module 4: Speaking

8

This module aims to test candidates’ speaking production and specifically, candidates’ ability to (a) answer questions about him/herself and his/her environment (two sets of questions, each containing a B1 and a B2 level question, i.e., four questions in total), (b) develop a topic on the basis of a visual prompt (1 task for B1 level and 1 for B2 level) (c) relay in English a message conveyed in a Greek text and respond to two tasks related to this text (1 task for B1 level and 1 for B2 level, which belong to the same set). The duration of the test is 25 minutes. The maximum total marks the candidate can obtain from all three oral activities are 20 points from examiner A and another 20 points from examiner B. In other words, the successful candidate can obtain a maximum of 40 points for module 4. The duration of this test is 20-25 minutes. The minimum score for achieving the B1 level certification is 60/200, whereas that for the B2 is 120/200.

2.3 OVERVIEW OF THE C1 LEVEL EXAM

Module 1: Reading comprehension Module 1 tests reading comprehension skills and language awareness. It aims at assessing candidates’ understanding of the overall or partial meanings in a text, to make inferences or intelligent guesses on the basis of the text. Moreover, some items are designed to assess their ability to make language choices that are correct and appropriate to the linguistic, discursive and social context. The test consists of 60 items, 50 multiple choice and 10 short answer items. The 50 multiple choice items are awarded a maximum of 20 points (0,4 marks per item) and the 10 short answer items are awarded a maximum of 5 points (0,5 marks per item), the total being 25 marks. The duration of this test is 80 minutes. Module 2: Writing This module tests candidates’ ability to produce written discourse and function as mediators through written production. It consists of 2 activities: 1 activity for written production where candidates are asked to interact with and respond to an English text and 1 for written mediation based on a Greek text. The maximum total marks the candidate can obtain from both activities are 30 points. The duration of this test is 90 minutes. Module 3: Listening comprehension This module tests candidates’ ability to understand standard spoken language on both familiar and unfamiliar topics normally encountered in personal, social, academic and vocational life. Candidates always have the opportunity to listen to the recordings twice before responding. This test consists of 20 multiple choice and 5 short answer items. All 25 items of the test are awarded a maximum of 25 points (1 mark per item). The duration of this test is 25 minutes. Module 4: Speaking This module, which consists of two activities, tests candidates’ speaking production and specifically, a) to state with precision, justify and support their views, ideas, opinions on social issues and practices, (Activity1), and b) to exchange complex information and discuss a difficult subject, using information from a Greek source text, negotiating a course of action, a decision, a conclusion, a solution to a problem, etc. (Activity 2). The maximum total marks the candidate can obtain from both activities are 20 points. The duration of this test is 25 minutes. The minimum score for achieving the C1 level certification is 60/100.

9

3. OVERVIEW OF THE SPEAKING TESTS

In this section, an overview of the content and structure of the oral tests per level is presented.

3.1 CONTENT AND STRUCTURE OF THE A, B AND C1 LEVEL SPEAKING TESTS

A LEVEL B LEVEL C1 LEVEL Duration 15-20 minutes 25 minutes 20-25 minutes

Mode of participation Candidates are tested in pairs but do not converse with each other. They interact with the Examiner / Interlocutor.

Candidates are tested in pairs but do not converse with each other. They interact with the Examiner / Interlocutor. In activity 1, the examiner interacts with each candidate in turn. In activities 2 and 3 the examiner does not interact with candidates; s/he sets the task and the candidates respond.

Candidates are tested in pairs and converse with each other in Activity 2.

Warm-Up (Candidates are not assessed)

There isn’t a separate warm up stage but Activity 1 functions as a warm up.

Introducing Ourselves: (2 min for both candidates) Addressing each candidate separately, the Examiner asks a few general questions (where do you live, which school do you go to, what do you do for a living) to break the ice and get to know the candidates. The questions also provide information in relation to the candidate profile, enabling the examiner to choose more appropriate tasks.

Introducing Ourselves: (1 minute – per candidate) The Examiner asks each candidate a few icebreaking questions (e.g. age, studies/work or hobbies). The questions also provide information in relation to the candidate profile, enabling the examiner to choose more appropriate tasks.

Activity 1 Interview (5 minutes – 2½ per candidate) This is a “getting to know you” task, which requires interaction between Examiner and candidate. This activity also functions as a

Dialogue (6 minutes - 3 minutes per candidate) Dialogue between the examiner and each candidate. The candidate answers two sets of questions, each containing a B1 and a B2

Open-ended response to an opinion question: (4 minutes – 2 minutes per candidate) In this activity, each candidate responds to a single question posed by the Examiner,

10

warm up but unlike other level oral exams it is assessed. Each candidate is asked 4 questions – two for A1 and two for A2 level – which are signposted for the examiner.

level question about him/herself and his/her environment (i.e., 4 questions in total), posed by the examiner.

expressing and justifying her/his opinion about a particular issue or topic. The Examiner asks candidate A an opinion question from a list of different questions. Candidate A responds, and then the Examiner asks candidate B a different question.

Activity 2 Talking about photos (5 minutes – 2½ per candidate) This activity essentially involves a guided description of a photo or series of photos (or other visuals e.g. sketches, drawings) which are thematically linked. Candidates are asked to describe people, things and situations, talk about what they see in one or more pictures, find the differences, between pictures, talk about something that happened, tell a simple story, predict what is going to happen, etc. The activity comprises four (4) questions – two for A1 and two for A2 level.

One-sided talk (8 minutes– 4 minutes per candidate) Each candidate responds to two tasks (one B1 level task and one B2 level task) and develops a topic on the basis of a visual prompt.

Mediation (15 minutes for both candidates) In this activity, candidates carry out a conversation in order to complete a task based on input from a Greek text. The candidates are given two Greek texts that are different but in some way related to one another. Candidates are asked to read the texts silently for two minutes before they start exchanging and negotiating information and completing the task. After exchanging relevant information from their respective texts, candidates are required to negotiate a course of action, reach a decision or conclusion, or find a solution to a problem. The Examiner asks candidate B to start the conversation.

Activity 3 Giving and asking for information (6 minutes – 3 minutes per candidate)

This activity is based on multimodal texts and also consists of two parts. Candidates are asked two A1 questions which they answer using as a source one or more multimodal texts such as a poster, an invitation, an announcement etc. For the A2 level, candidates pose three questions to the

Mediation (10 minutes - 5 minutes per candidate) This activity tests candidates’ ability to relay in English a message conveyed in a Greek text and answer questions about this text. Each candidate responds to two tasks (a B1 and a B2 level task belonging to a set) based on input from a Greek text.

There is no activity 3.

11

Examiner relating to one multimodal text guided by question prompts, and the examiner answers (the answers are provided in the examiner pack).

NOTE Grading in the A level oral test occurs on many levels. There is an equal number of A1 level and A2 level questions throughout all activities. In addition, the activities themselves are also graded in terms of difficulty; thus, Activity 1 (dialogue in the form of an interview) is considered linguistically and cognitively less demanding than activity 2 which is in turn considered linguistically and cognitively less demanding than Activity 3.

Grading in the B level oral test occurs on many levels. There is an equal number of B1 level and B2 level questions throughout all activities. B1 level questions and tasks are cognitively and linguistically less complex and require less complex language production than B2 level tasks.

3.2 MATERIALS FOR THE A, B AND C1 LEVEL SPEAKING TESTS

Materials A LEVEL B LEVEL C1 LEVEL The Candidate Booklet • information about the test and

guidelines on how to carry it out,

• the Speaking Test Assessment Criteria Grid,

• six pages of colour photos consecutively numbered. Each page has a title reflecting the thematic and contextual arrangement of the photos, which constitute the visual prompts for Activity 2, and

• information about the test and guidelines on how to carry it out,

• the Speaking Test Assessment Criteria Grid,

• five pages of colour photos consecutively numbered. Each page has a title reflecting the thematic and contextual arrangement of the photos, which constitute the visual prompts for Activity 2, and

• information about the test and guidelines on how to carry it out,

• the Speaking Test Assessment Criteria Grid, and

• 11 pages of texts in Greek. There are five different sets of texts. Each set consists of one or more texts which are different but in some way related to one another. The heading on each page indicates the theme and context of talk prompted by these texts for Activity 3. In case there are three

12

• five pages of multimodal texts consecutively numbered. Each page has a title reflecting the thematic and contextual arrangement of the multimodal texts, which, along with the question prompts for each multimodal text, constitute the prompts for Activity 3.

• six pages of texts in Greek, consecutively numbered. The heading on each page indicates the theme and context of talk prompted by these texts for Activity 3.

candidates to be examined together, there is one thematic category which consists of three sets of texts.

The Examiner Pack • the Interlocutor Frame, • potential trouble spots during the

Speaking Test and how to deal with them,

• the A level Speaking Test Rating Scale,

• the test items, i.e., the questions for Activity 1 and the tasks for Activities 2 and 3,

• the responses to the questions that the candidates ask the examiner in the A2 part of Activity 3.

• the Interlocutor Frame, • potential trouble spots during the

Speaking Test and how to deal with them,

• the B level Speaking Test Rating Scale,

• the test items, i.e., the questions for Activity 1 and the tasks for Activities 2 and 3.

• the Interlocutor Frame, • potential trouble spots during the

Speaking Test and how to deal with them,

• the C1 level Speaking Test Rating Scale,

• the test items, i.e., 14-20 opinion questions for Activity 1, half of which are more appropriate for younger candidates and half for adults, and the tasks for Activity 2.

The Examiner Training Pack the handouts distributed at the A level Examiner seminars.

the handouts distributed at the B level Examiner seminars.

the handouts distributed at the C1 level Examiner seminars.

13

Examiner - Interlocutor Examiner -

Rater

Candidate B

Candidate A

Alternative position 1 Alternative position 2

Examiner - Rater

4. SPEAKING TEST PROCEDURE AND ROLE OF THE EXAMINER

This section presents an overall description of the exam room setting for the speaking test, the procedures carried out by the examiners before, during and after the test and suggestions on how to tackle any problems that might occur during the test.

4.1 SPEAKING TEST GENERAL PROCEDURE

Throughout the exam, there are two assessors and two candidates in the exam room. Both assessors evaluate the candidate and fill in both parts of the “Oral Production Evaluation

Form”. The assessor who has the role of the Examiner-Rater sits on the side and is silent. S/he listens,

observes, takes notes, and rates each candidate’s performance on the spot, using the “Oral Production Evaluation Form”. S/he is Evaluator 1.

The assessor who has the role of Examiner-Interlocutor is the one who sits facing the two candidates and who conducts the test, interacting with them. S/he rates candidates when they have left the exam room. So, besides being the Examiner, s/he has the role of Evaluator 2.

The candidates are examined in pairs. The candidates take turns in doing the required task first. This means that if candidate A is the first

one asked to respond to the questions of Activity 1, it is candidate B that is asked to begin Activity 2 and candidate A that is asked to begin Activity 3 and so on.

Assessors should change roles frequently. It is recommended that they change roles as Evaluator 1 and Examiner/Interlocutor (Evaluator 2) when they have conducted the test with 2-3 pairs of candidates. However, the frequency of role changing is up to them.

Both examiners conducting the speaking test are equally responsible for carrying out the test successfully.

In using the exam material: You are expected to use a variety of activity pages, texts and task types, keeping the candidate’s profile in mind. You are not expected to use all photos, texts or tasks, nor to use all of them more than once. If you feel that one does not work for you, don’t use it.

14

4.2 EXAMINER ROLE AND CONDUCT

Responsibilities of Examiners • To assess the candidates’ oral performance by taking into account the level and purpose of the

exam.

• To create a comfortable atmosphere providing the candidates with sufficient encouragement enabling them to perform to the best of their abilities.

• To ensure test condition consistency by following the guidelines for examiner conduct.

Examiner conduct throughout the test • Speak clearly, loudly and slowly enough for candidates to understand.

• Always use English (even if the candidate slips into his/her mother tongue).

• Smile and have a friendly, cheerful attitude towards the candidates to make them feel at ease.

• Do not interrupt candidates while they are speaking as this might affect their confidence and their flow of ideas.

• Always assess taking into account what the candidate has managed to do rather than what the candidate has done wrong. Remember that this is a stressful and unfamiliar experience for the candidates and the language produced during the exam will not resemble a natural conversation.

Before the test begins • Arrive at the Examination Centre at least two hours prior to the speaking test in order to

prepare fully and go to the special room made available for the examiners. • Study carefully the Candidate Booklet and the Examiner Pack. • Work with a colleague and discuss the tasks and texts, helping each other understand what

each task requires. • Role-play one or two tasks and see how they work for you. • Acquaint yourself thoroughly with the Interlocutor Frame provided in the Examiner Pack. • Examiners are expected to have prepared themselves for the test by having become familiar

with the information contained in the Handouts developed for the Oral Examiner Seminars. However, study carefully the Rating Scale in the Examiner Pack to remind yourself of the assessment criteria.

• Go to the exam room assigned to you and your colleague and together arrange the desks, as shown in the diagramme above.

• Bring the candidates into the exam room, ask them to hand you their evaluation forms and give them to your colleague, show them where they are supposed to sit, introduce yourself and your colleague, and ask for the candidates’ names.

• Explain the duration and procedure of the test (use body language to make your language more comprehensible) in order to prepare the candidates and to familiarize them with your speech pattern and the English speaking environment. Use the Interlocutor Frame throughout the test..

During the test • Always speak clearly and slowly enough for candidates to understand. If you sense that they

have not understood your instructions, repeat them more slowly and use body language to make instructions comprehensible.

15

• Demonstrate sensitivity and allow candidates time to respond, moving on to the next task or candidate, when it is obvious that there will be no further response. Long pauses should be avoided since they will make the candidate feel uncomfortable.

• When each task is assigned, make sure the candidates understand what they are expected to do. You may check their understanding by asking “Do you understand this task?” “Would you like me to repeat?”.

• Do not show that you disapprove of their performance when they give a wrong response, but do provide positive feedback (“OK”, “that’s fine”) when they manage to respond, despite errors.

• If the candidate gets stuck or seems unable to continue, try encouraging him/her by repeating the last part of his/her previous utterance, or prompting him/her by saying “anything else?”

• Do not correct candidates’ errors.

• Be supportive. Never show displeasure, disappointment, surprise, even when the candidate has provided an unexpected answer.

• Use conversational signals (e.g., nodding of head, phrases like “uh hum”) to mark participation in the exchange and to help it flow.

At the end of the test • Thank the candidates for their participation.

• Do not give any information regarding the candidates’ performance.

• Say goodbye and lead them out of the room.

• Take the completed evaluation form from the second examiner and fill in your marks.

4.3 POTENTIAL TROUBLE SPOTS DURING THE SPEAKING TEST

PROBLEM SOLUTION

What do I do if ….. Follow suggestions below

…the candidate is hesitant, makes long pauses or produces little output?

1. Repeat the candidate’s last phrase with rising intonation.

2. Ask a few prompting questions (Is there anything else you would like to add…?).

3. Try to break down the task into simpler questions.

...the candidate is nervous and has difficulty in speaking?

1. Smile, use body language, facial expressions and intonation to make the candidate feel more comfortable.

2. If you asked the nervous candidate to begin first, switch to the other candidate and then come back to the nervous candidate.

…the candidate draws a blank and seems unable to answer (for activities 2 and 3)?

1. Repeat the question/task. 2. If the candidate still hesitates, change the task

but stick to the same visual prompt/ text.

NOTE:

Any extra help or guidance (as described above) given to a candidate should be taken into account in the assignment of the final mark.

16

Remember that you are there to assess candidates and not to help them or teach them. It is very easy to slip into the teacher’s role and supply the candidate with a word s/he is searching for or to complete the candidate’s phrase in order to keep the candidate going. Avoid providing unnecessary help as much as possible.

Care has been taken to phrase task instructions using simple lexis and sentence structure. Do not change or add more information to task instructions.

If a candidate does not understand a word in the task instructions, you may give a synonym if you are asked to.

5. THE INTERLOCUTOR FRAME

In order to ensure that all KPG candidates are treated fairly in the Oral Tests and undergo the same test-taking experience, and in order to reduce the variation in the talk of interlocutors (and the kind of accommodations they make), an Interlocutor Frame has been introduced for each level of the Oral Test. Other examination systems have introduced the same practice (e.g. Cambridge Local Examinations Syndicate), which they call Interlocutor Scripts. In essence, an interlocutor frame (or script – just like a script an actor reads from) spells out exactly what the interlocutor should say from the moment the candidates enter the room till the moment they depart. Research has shown (see Fulcher 2003) that interlocutor frames do minimize the differences between the talk of interlocutors and result in a fairer test.

In Appendix 2, you can find the Interlocutor Frame developed for the B level speaking oral test. Please acquaint yourself thoroughly with the frames and use them to conduct the Oral Tests.

6. USING THE OFFICIAL ASESSMENT FORM

The official assessment forms (see the assessment form for the B level speaking exam provided as an example in Appendix 1) have two columns for marking. Column 1, on the left-hand side, is filled in by Evaluator 1 who assesses while the candidates are taking the test, and Column 2, on the opposite side, which is filled in by Evaluator 2 (the Examiner/Interlocutor), as soon as the candidates leave the exam room. Make sure the candidates’ names and code numbers are on their respective forms. Both Evaluators mark the boxes in the columns in ball point pen (black or blue). Since Evaluators record their marking separately, their ratings may differ. Do not mark the form before you have made your final decision on the mark you will award. The overall rating for each candidate – the total mark on the test – is electronically computed, so there

is no space provided on the Form for this. Make sure you are clear about how you place your marks on the Assessment Form. Corrections are

not permitted on the Form, which should not be wrinkled or damaged in any way. However, if you do make a mistake and must correct it, take it to the Exam Centre Committee and they will help you deal with the problem.

Fill in the appropriate boxes for each scale like this:

17

7. OVERALL ORAL LANGUAGE ABILITY OF THE A, B AND C1 LEVEL CANDIDATE

This section presents the expectations for oral language ability at A, B and C1 levels. These have been adapted from the expectations described in the Common European Framework of Languages (CEFR).

The A level candidate is a basic user of English. S/he has a limited command of basic grammatical structures and sentence patterns, and a limited vocabulary. On the basis of this knowledge, s/he is expected to be able to understand and to use everyday language familiar to him/her to satisfy his/her basic communicative needs. S/he can introduce him/herself and others, ask and answer questions about personal details, such as where s/he lives, people s/he knows, etc, which are clearly formulated. Finally, s/he is expected to be able to interact in a simple way provided that his/her interlocutor talks slowly and clearly and is prepared to help. The B level candidate is an independent user of English. S/he is expected to sustain reasonably fluently a straightforward description of one of a variety of subjects within his/her field of interest and immediate environment (e.g., family, friends, work/studies, hobbies/interests) and situations which are cognitively familiar (e.g. leisure activities, environmental issues, personal characteristics etc), presenting it as a linear sequence of points at B1 and more coherently, using a variety of connectors at B2. Moreover, s/he is expected to communicate with some confidence on familiar routine and non-routine matters related to everyday issues, to exchange, check and confirm information, deal with less routine situations and explain why something is a problem. Also, s/he is expected to be able to express thoughts on more abstract, cultural topics such as films, books, music etc., to exploit a wide range of simple language to deal with most situations likely to arise whilst traveling and, finally, to enter unprepared into dialogue on familiar topics, express personal opinions and exchange information on topics that are familiar, of personal interest or pertinent be to everyday life (e.g. family, hobbies, work, travel and current events). His/her language must be appropriate to the communicative situation while his/her pronunciation and intonation must not obstruct the communication of his/her message in the target language. The B level candidate is also expected to act as mediator by using the target language (in this case, orally) to convey the overall meaning or specific information in a Greek text. The C1 level candidate must demonstrate that s/he has the ability to talk about or discuss a subject and to exchange information by producing language in/with a comprehensible accent, using grammatically accurate and communicatively effective language. Producing language, which abides to the functional rules of the linguistic system under given conditions of interactive communication, s/he will be expected to respond to questions and to discuss topics, based on information provided in a Greek text, with his/her interlocutor/s.

18

TASK 1: THE EXPECTATIONS FOR ORAL LANGUAGE ABILITY

Read carefully the expectations for oral language ability at A, B and C1 levels, presented above, and decide which level each of the specific language ability descriptors (can do statements), presented below, belongs to.

Can Do Statements Level (A, B or C1)

1. Can express ideas and thoughts/opinions about films, books, music, etc. reasonably fluently.

2. Can ask and answer questions related to common aspects of everyday life, such as travelling, food, shopping, entertainment, means of transport, television programs, school, etc., or to convey and retrieve information pertaining to quantities, figures, prices, etc.), and to exchange ideas and information concerning familiar issues in predictable everyday situations.

3. Can make simple comparisons between persons, objects and daily activities.

4. Can present and justify an opinion or express and explain why they wish for or intend to do something.

5. Can use the language with relative fluency on a wide range of everyday topics related to work, studies or leisure, marking clearly the relationships between ideas.

6. Can handle numbers, quantities, qualities of things, simple directions and the concept of time (e.g. what I am going to do next week or at 3:00 o’clock, what I did last Friday, etc.).

7. Can propose the solution to a problem and defend her/his proposal by justifying why they think it would be effective

8. Can communicate spontaneously with good grammatical control without much sign of having to restrict what s/he wants to say, adopting a level of formality appropriate to the circumstances.

9. Can take the floor by intervening elegantly using formal or informal language, depending on the communicative situation (using emotional language or indirectness when it is required), and give the floor to their interlocutor or make a concluding statement.

10. Can exchange and check information, deal with routine situations and explain why something is a problem.

19

TASK 2: THE EXPECTED ORAL PRODUCTION FROM ACTIVITY 1

After having worked on the expectations and the indicative ‘can do’ statements for each level presented in Task 1, read through the example questions from Activity 1 of various speaking tests from previous exam periods, presented in the table below, and decide which level, A, B or C1, they are designed for.

Activity 1 questions Level (A, B or C1)

1. How many people are there in your family? Who are they?

2. Do you agree or disagree with Nelson Mandela’s view that Education is the most powerful weapon which you can use to change the world?

3. What do you usually eat in order to have a healthy diet?

4. What can you see from your bedroom window?

5. What do you like to buy when you go shopping and why?

6. What do you think can be done so that children stop eating junk food?

7. Do you talk about your problems to other people or do you prefer to keep things to yourself and why?

8. Do you think that the saying (by Ralph Waldo Emerson) “The only way to have a friend is to be one” is always true? Explain why or why not.

9. If you could change one thing in your neighbourhood what would it be and why.

10. Tell us what you’re going to do tomorrow evening.

20

APPENDIX 1: OFICIAL ASSESSMENT FORM FOR THE B LEVEL SPEAKING TEST

21

APPENDIX 2: INTERLOCUTOR FRAME FOR THE B LEVEL SPEAKING ORAL TEST

Part 1 (warm-up & Activity 1) Introducing ourselves (about 2 minutes for both candidates) [NOT MARKED]

Examiner: Good afternoon. Welcome. Can I have your evaluation forms, please? (Take them and give them to your co-assessor, making sure you don’t mix up the two candidates.)

Thank you, please take a seat. My name is ________ (and) this is my co-assessor ________. S/he will be observing us.

Please speak in English, loudly and clearly, throughout the test. You may ask me to repeat task instructions or to explain something you don’t understand, but only in English.

So… What is your name? (Write it down.) And yours? (Write it down.) Examiner: (Addressing candidate A) So________ (his/her NAME), what do you do? / where do

you live? / what are your plans for the future? etc. (any general questions to break the ice and get to know the candidate.)

Examiner: (Addressing candidate B) And what about you ________ (his/her NAME), what do you do? / where do you live? / what are your plans for the future? etc. (any general questions to break the ice and get to know the candidate.)

Activity 1 (6 minutes for both candidates- 3 minutes each) Examiner: Ok. Let’s start with Activity 1. I will ask each of you some questions. So, ________ (candidate A’s NAME). Choose TWO sets of questions from the list below

and ask him/her. (When your exchange with the candidate has finished.) Thank you. Examiner: Now, let’s go on with ________ (candidate B’s NAME). Choose TWO sets of questions DIFFERENT from the ones you asked candidate A, from

the list below. (When your exchange with the candidate has finished.) Thank you.

Part 2 (Activity 2) Activity 2 (8 minutes for both candidates - 4 minutes each)

Examiner: We can now go on with Activity 2. I will give each one of you one or more photos and I will ask you 2 questions. You each have about 4 minutes to answer.

Start with candidate B this time. Examiner: Let’s start with________ (candidate B’s NAME). ________ (his/her NAME), turn to

page________ (Select a page from the Candidate Booklet) and look at picture(s) ________ (Select one or more pictures from this page) and ________ (choose and read out a B1 task from the ones given below).

Examiner: (When the candidate has finished) Now, look at picture(s) ________ (Select one or more pictures from the same page) and ________ (choose and read out a B2 task from the ones given below).

Examiner: (When the candidate has finished.) Thank you. Examiner: Now, ________ (candidate A’s NAME), it’s your turn. Please, go to page________

(Select a different page from the Candidate Booklet) and look at picture(s) ________ (Select one or more pictures from this page) and ________ (choose and read out a B1 task from the ones given below).

22

Examiner: (When the candidate has finished) Now, look at picture(s) ________ (Select one or more pictures from the same page) and ________ (choose and read out a B2 task from the ones given below).

Examiner: (When the candidate has finished.) Thank you.

Part 3 (Activity 3) Activity 3 (10 minutes for both candidates- 5 minutes each)

Examiner: Now, let’s move on to Activity 3. I will give each one of you a Greek text to read and two tasks to do. After reading your texts, you will each have about three minutes to perform these tasks. Start with candidate A this time.

Examiner: Let’s start with ________ (his/her NAME). Go to page ________ (Select a page from the Candidate Booklet) and look at (the) text (number ____) (Select a text) and ________ (choose and read out the B1 level task).You can read the text for about two minutes and then, before you start, I will repeat the task for you. Now, ________ (candidate B’s NAME), Go to page ________ (Select a different page from the Candidate Booklet) and look at (the) text (number____) and ________ (choose and read out the B1 level task). You can read the text for about two minutes and then, before you start, I will repeat the task for you. After about two minutes Let’s start with ________ (candidate A’s NAME).

Ready? Ok (Repeat the B1 task). Examiner: Now, ________ (candidate B’s NAME), let’s continue with you. Ready? OK (Repeat

the B1 task). (When the candidate has finished.) I will give you some time to read the text again to do another task (give him/her the B2 level task).

Examiner: Now, ________ (candidate A’s NAME), let’s continue with you. I will give you some time to read the text again to do another task (give him/her the B2 level task).

Examiner: Now, ________ (candidate B’s NAME), it’s your turn to do the second task. Ready? Ok (Repeat the B2 task.) (When the candidate has finished.) Thank you.

Examiner: Now, ________ (candidate A’s NAME), it’s your turn to do the second task. Ready? Ok (Repeat the B2 task.) (When the candidate has finished.) Thank you. This is the end of the examination. Have a nice afternoon/ evening.