China English: Issues in Constructing and Localising an EAP Test

Wenxia Zhang （张文霞） Dept. of Foreign Languages and Literatures Tsinghua University wxzhang@mail.tsinghua.edu.cn

Outline

Background Research questions Research approach The study: constructing and localising the test Discussion: challenges and issues Conclusion

1

Background Globalisation: World Englishes, Asian Englishes and English in

China (B. Kachru, 1982, 1994, 2005; Y. Kachru & Nelson, 2006; Graddol, 2013)

The Asian region comprises the largest English-knowing population in the world: the estimated users of English in India and China alone comprise over 533 million people. (Kachru, 2005)

China English (Li, 1993; Du & Jiang, 2001) - Definition: English used by Chinese EFL learners with

culture-specific features - Distinction between Chinese English (Chinglish) and China

English

2

Background World Englishes and EFL teaching “Certainly there has long been an emphasis in the

ESL/EFL profession in general on externally imposed models and pedagogies that are in various ways, as has been discussed in the literature, not relevant to the Asian situations.” (Y. Kachru & Nelson, 2007:135)

World Englishes and EFL assessment - Norms in EFL/ESL Tests (Davies et al., 2003, 2004; Brown, 2004) - Contrastive Rhetoric and rater characteristics

(Hamp-Lyons & Zhang, 2001; Zhang, 2004)

3

Background Brown (2004:318) states: Even in considering a topic as narrow as the effects of Englishes on test bias, we must remember the possibility that multiple Englishes may be involved in even a single EFL/ESL test. I think it is fair to say that the following Englishes might have some impact on tests: 1. the English(es) of the test takers’ local community; 2. the dominant English of the test taker (which may not be the same as the local community); 3. the English(es) of the test content; 4. the English(es) of the test proctors; 5. the English(es) of the test scorers/raters; 6. the English(es) of the decision target community; 7. the English(es) of the decision target purpose; 8. the English(es) of the decision makers.

4

Background Brown and Lumley (cited in Hamp-Lyons & Davies, 2008:29) claim to have had several aims in view in their Indonesian test development, all of which were fulfilled. -the judicious selection of tasks relevant to teachers of English in Indonesia; - the selection of culturally appropriate content; - an emphasis on assessing test takers in relation to local norms; - the use of local raters, that is non-native speakers of English (whose proficiency was nevertheless of a high standard). (Brown and Lumley 1998: 94)

5

Research context International English tests in China (e.g., TOEFL-ibt, IELTS,

PTE, etc.) National English tests (e.g., NMET, CET, PETS, etc.) Change in curriculum (individualised learning and

teaching) and need for school-based tests and/or classroom-based assessment

6

Research objective To investigate/explore the impact of China English on the construction and/or localisation of an EAP test in the Chinese context. Research Questions - How does China English affect the test construction at various stages? - What are the challenges of China English for test developers, administrators and raters? China English in the present study refers to the characteristics of English used for assessment purposes in the Chinese context.

7

Research Approach A qualitative approach(exploratory, descriptive) , or rather, a social ethnographic approach was adopted “to build up a comprehensive picture of the context”.(Graddol, 2013:6) Data: documents (test syllabus, test papers, minutes, etc.) and interviews (with decision-makers, test developers, EFL teachers and students) Data analysis: on-going, preliminary, descriptive, theme-based.

8

The Test: Tsinghua English Proficiency Test (TEPT) Purpose: to assess students’ EFL proficiency in academic contexts Components: Written (paper and pencil) paper (listening, reading and writing) and Oral (computer-based) Test-takers: university students (year 2 and above) An overview of TEPT: Tsinghua English Proficiency Test - overall structure and sample test paper.docx

9

Test construction: needs analysis Who are the decision makers? 1996 version – university administrators

2005 version – test developers

2011 version – joint decision (test-takers, school authorities, EFL teachers and test developers)

10

Test construction: setting standards

Whose norms? Defining the construct: - Theoretical framework: Bachman (1990, 1996, 2010) - National College English Curriculum Requirements (2007) - CEFR - Tsinghua English Teaching Syllabus (2010) Deciding on the test structure: - Cambridge tests, TOEFL-ibt, PTE, CET, PETS, …

11

Test construction: designing and developing the test Whose English? Components: role of translation? Test material: genre, topic, length, etc. Task type: MC, reading-aloud?

(Beyond language assessment: social, political, cultural, etc.)

12

Test administration: rater training

Whose norms to follow?

Scale development: general vs. task-specific

Rater training: identifying and handling China English

13

Test administration: score reporting

Assessment of learning (AOL) or assessment for learning (AFL)?

Nature of assessment: diagnostic, proficiency, screening, …?

Single score vs. analytical profile

14

Challenges and issues

Nature of an EAP test – whose voice?

English used in the test – whose norms?

Globalisation/Internationalisation vs. localisation – where is the boundary?

15

Implications - test development and rater training in EFL setting - role of China English in EFL assessment

Limitations - generalisability - data and method

16

Conclusion “It is crucial, therefore, to understand the contextual

dynamics and to set realistic policy goals in order to raise standards over time in local environments which truly value the benefits of multilingualism.” (Saville in Graddol, 2013:8)

17

Selected References Bachman, L. (1990). Fundamental Considerations in Language Testing. Oxford: Oxford University Press. Bachman, L. (1996). Language Testing in Practice. Oxford: Oxford University Press. Bachman, L. and Palmer, A. (2010). Justifying Assessment Use Argument in the Real World. Oxford: Oxford University Press. Brown, J.D. (2004). What do we mean by bias, Englishes, Englishes in testing, and English language proficiency? World Englishes, 23(2), 317–319. Davies, A., Hamp-Lyons, L. and Kemp, C. (2003). Whose norms? International proficiency tests in English. World Englishes, 22(4), 571–84. Du, R.Q. and Jiang, Y.J. (2001). A review of “China English” research in the past twenty years. Foreign Language Teaching and Research, 33(1), 31-47. Graddol, D. (2013). Profiling English in China: The Pear River Delta. Cambridge: Cambridge English Language Assessment Hamp-Lyons, L. and Davies, A. (2008). The Englishes of English tests: bias revisited. World Englishes, 27(1), 26–39. Hamp-Lyons, L. and Zhang, W. (2001). World Englishes: Issues in and from academic writing. In J. Flowerdew & M. Peacock (eds.), Research Perspectives on English for Academic Purposes (pp.101-116). Cambridge: Cambridge University Press. 18

Higher Education Dept., Ministry of Education. (2007). College English Curriculum Requirements. Shanghai: Shanghai Foreign Language Education Press. Kachru, B. (1982). (ed.) The Other Tongue: English Across Cultures. Urban, IL:University of Illinois Press. Kachru, B. (1994). Englishization and contact linguistics. World Englishes, 13, 135-154. Kachru, B. (2005). Asian Englishes: Beyong the Canon. Hong Kong: Hong Kong University Press. Kachru, Y. and Nelson, C. (2006). World Englishes in Asian Context. Hong Kong: Hong Kong University Press. Kacrhu, Y. and Nelson, C. (2007). Response to Kirkpatrick by Yamuna Kachru and Cecil L. Nelson. International Journal of Applied Linguistics, 17(1), 135-137. Li, W.Z. (1994). China English and Chinese English. Foreign Language Teaching and Research, 4. TEPT group. (2012). TEPT (Tsinghua English Proficiency Test) Syllabus. Beijing: Tsinghua University Press. Zhang, W. (2004). The Rhetorical Patterns Found in Chinese EFL Student Writers’ Examination Essays in English and the Influence of these patterns on Rater Response. Beijing: Tsinghua University Press.

19

20