Jalt shinshu20140830pvkey

JALT Shinshu Chapter, Aug. 30, 2014 Seisen Jogakuin College, NaganoCLT and Assessment in English Education in Junior and Senior High School

Are the goals for exam preparation and CLT mutually exclusive?

亘理陽一 Yoichi Watari

[email protected] Shizuoka University

mailto:[email protected]

I’d like to say …

• the goals for exam preparation and CLT are not mutually exclusive; Rather, it could, or has to, build a mutually complementary relationship

• based on

• two studies on university entrance exams (Seki et al., 2011; Watari et al., 2012) and

• a study on high school entrance exams (Watari et al., 2013)

Introduction

• Two typical claims:

• [1] “Students and their parents’ first priorities in English classes are to prepare for English entrance exams.”

• [2] “Grammar translation style of studying is most efficient way to pass the exams.”

Introduction

• Nagakura et al. (2006): A survey

• In Aichi, Ishikawa, Toyama, and Shizuoka

• For 712 JHS students, 764 SHS students, 372 university students,

• 244 teachers (79 in JHS &165 in SHS),

• 431 parents (285 of JHS stu & 146 SHS stu)

Introduction


• 11 questions about goals of English education in schools with 4 possible answers

• Results: The No. 1 for JHS/SHS Students are…

• “to gain high marks in the entrance or employment exams”

• “to communicate with people in/from other countries in English”

Introduction


• Results: For parents

• No. 1: “to communicate with people in/from other countries in English

• No. 2: “to develop good communication abilities in both English and Japanese.”

• No. 3: “to gain high marks in the entrance or employment exams”

Introduction

• Grammar Translation Method is …

• a method of foreign language teaching which makes use of translation and grammar study as the main teaching and learning activities. (Richards & Schmidt, 2010, p. 252)

• characterized by emphasizing: 1) grammar teaching, 2) reading rather than speaking, 3) translating, 4) using grammatical terms of the students’ L1, 5) trying to understand the language analytically (Shirahata et al., 2009, p. 126)

Seki et al. (2011) survey

• Research Object: The questions of 91 entrance exams

• By 33 universities (/765 in 2009)

• National: 9 (/86)

• Private: 24 (/589)

• Of humanities and the science

• In 2008 and 2009



• 33 universities, humanities and the science, 08 & 09

• Classified each question in terms of the text content, text style, text field, question type, answer form, total number of words, and so on.

• To examine if each exam question is answerable in grammar-translation style of studying

Figure 1. Items of analysis• Question types: Filling-in of blanks / Reading

comprehension / Grammar and Usage / Words with similar meanings / Word-sorting / True-False / English-Japanese translation / Japanese-English translation / Error Correction / Reference relation / Word definitions / Pronunciation / Rewriting / Tasks / Listening / Partial English-Japanese translation / Organizing paragraphs / Writing / Summarizing

2

2 Research method

2.1 The background of the subjects

The project group chose well–known universities in regions all over Japan whose entrance examinations were

available for the research. In total thirty-three universities including nine national and twenty-four private

universities were chosen as follows: Hokkaido Univ., Tohoku Univ., Tokyo Univ., Hitotsubashi Univ., Nagoya

Univ., Kyoto Univ., Osaka Univ., Kyusyu Univ., Tokyo Institute of Technology., Sophia Univ., Waseda Univ.,

Keio Univ., Meiji Univ., Rikkyo Univ., Aoyama Gakuin Univ., Chuo Univ., Hosei Univ., Nihon Univ., Toyo

Univ., Komazawa Univ., Sensyu Univ., Nanzan Univ., Cyukyo Univ., Meijyo Univ., Aichi Univ., Kansai Univ.,

Kwansei Gakuin Univ., Ritsumeikan Univ., Doshisya Univ., Kyoto Sangyo Univ., Kinki Univ., Konan Univ., and

Ryukoku Univ. National universities usually give their entrance examinations twice in a year. The first exam is

held in the middle of January with the greater numbers of admittance and the second one is given in the middle

of March. This project group analyzed the entrance examinations of the national universities in the first term.

Regarding private universities, the group analyzed examinations of the department of each university with the

highest quota of students in principle. If the universities had both art and science departments, two examinations

from both departments of which had the highest enrollment were analyzed respectively. Totally ninety-one

college entrance examinations conducted in 2008 and 2009 were analyzed in this research.

2.2 Analyzed Item

Figure 1 is the analysis sheet used for the research. First, the analysis items in the class ‘General’ show the

basic information of the examination questions analyzed, such as the serial number among the universities

analyzed, the university’s name, the department’s name, the year in which the examination was given, the

examination type, the test time allotment, and the question section number.

Second, the items in the class ‘Readability’ show the number of words in English sentences in each question,

the number of paragraphs (or turns in conversations) and the reading ease and Kincaid Grade Level calculated by

Flesch- Readability formula1). In the Reading ease scale, the number closer to 100 is easy to read and the number

under 60 is considered difficult. Kincaid Grade Level shows which grade of the American educational system

1) Reading Ease=206.835 (1.015×α) (84.6×β) α=average words per sentence, β=average

syllables per word For further information, refer to Kiyokawa 2000 .

General Readability Category Questions

Ser

ial n

umbe

r

Uni

vers

ity

Dep

artm

ent

Yea

r

Exa

min

atio

n ty

pe

Tes

t tim

e al

lotm

ent

Que

stio

nse

ctio

n nu

mbe

r

Num

ber

of w

ords

Num

ber

of p

arag

raph

s

Fle

sch

Rea

ding

Eas

e

Fle

sch-

Kin

caid

Gra

de L

evel

Sty

le

Cat

egor

y

Top

ic

Que

stio

n nu

mbe

r

Tes

t dir

ection

Ran

ge o

f re

fere

nce

Que

stio

n ty

pe

Lan

guag

efo

ran

swer

ing

Way

s of

ans

wer

ing

Num

ber

of m

ultipl

e ch

oice

item

s/as

sign

ed w

ord

num

bers

A t

ype

B t

ype

Figure 1 analysis sheet

• Questions in the group A

• solvable by grammar-translation style of studying

• characterized by translation, syntactic and/or semantic accuracy, and phonetic rules within isolated sentences.

• Questions in the group B

• requiring some higher-level skills and strategies such as skimming, making inferences, summarizing and organizing paragraphs, expressing their opinions within limited words, solving some tasks using English

• Questions in the group O

• other than type A and B questions

Table 1 Descriptive Statistics (1)

n=91� Test time�

Total number of words�

��

N. Of Questi

ons�

N. Of Group

A�

N. Of Group

B�

�� 83.85� ( )�( ) 39.80�( )�( )�

�� 19.05� 815.35� 244.28 12.81� 9.55� 8.27�

�� 140� ( )�1336.00 62� 39� 32�

�� 60� 767� 126.70 10� 1� 0�

Table 2 Descriptive Statistics (2)Styles�Expository� 67.12%�

Academic� (39.35%)�

Essay� (27.76%)�

Narrative� 5.12%�

Dialogue� 18.87%�

Other� 8.89%�

Instructed in��Both� �1.86%��English� �22.06%��Japanese� �76.08%� To Answer in��English �90.79%��Japanese� �9.21%� To Answer by��Writing� �11.36%��Citing� �0.47%��Choosing ��88.17%�

3626 questions in 91 exams

Figure 2. The ratio of question forms in number

0% 5% 10% 15% 20% 25% 30%

"��&�

)��&�

��"&�

writing %��

210�

��3(��

��(�

( �

error correction �-�,�

�.�,�

��$��

�#��3$��

!-/�

�+�

��

��*��'�

� �'�

Figure 3. The number of questions by group

A�,#1485#

O�,#1508#

B�,#633# 17.46％

40.95％

41.59％

Figure 4. Group B questions by question type

0 50 100 150 200 250 300 350 400

��"�

�'�)�

error correction �(�&�

!��

�'�&�

��%��#�

�� - ��

�$�

writing ,+*�

��

��

��#�

Skimming�Making Inferences�

Scanning�

Problem-solving�

Expressing�

Understanding paragraphs�

Summarizing� Organizing�

Figure 5. Languages used/to use by group

0% 20% 40% 60% 80% 100%

Answer

Instructed

Answer

Instructed

Answer

Instructed

A

O

B

in Japanese in both in English

Figure 6. The scope to solve a question by group

0%# 20%# 40%# 60%# 80%# 100%#

A#

O#

B#Graph(s)#

Words/phrases#

Clause#

Sentence#

Sentences#

Passage#

Passage#and#more#

Figure 7. (B-ratio − A-ratio) × total number of words by exam

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

0 500 1000 1500 2000 2500 3000 3500 4000 4500


• Those questions compatible with grammar-translation type of learning: 40.41% as a whole.

• The group B questions are

• giving instructions in English and requiring answers in English

• demanding understanding of sentences, paragraphs, or the whole text

• steadily increasing.

• The more B-oriented an exam becomes, the more information students have to deal with.

Watari et al. (2012) analysis


• 33 universities, humanities and the science, 08 & 09

• Content analysis of the group O questions in terms of

• types: lexical, referential, paraphrasing, thematic, organizational, inferential, other

• reference to contexts: linguistic (immediate words/association)/situational (at hand/background)


• Reference to linguistic contexts:

• Preceding/following words

• Word association

• E.g., Ritsumeikan, 2008, [3(1)]

4

2008È Ŝŷ� Ň�Ƥ±·� 2/2,ƷdŠƷùÍ� þ~� ŏ¤� ŏĢ� §ƙR(§ƙ)� §ƙR(īČ)� §ƙR(òŋ)� §ƙR(Ĝ)� §ƙƖ|� īČĿu� Û«ĨÅ� röR(ö)� òŋł·� Ī�ł·� ö� öĨR(ŏ¤)� öĨR(ŏĢ)� öĨR(ĨÅ)� Ĝ� ĨÅ� Š �ƭ�� ƫ� Đ:(A)ƹ(C); AdBqr:¾ų5�K�-L.L:( )9�LK:9E26EƋÐ8ţħO(1)ƹ(4)�If3ƌ=ƪ-:İ�OũŊď9]eU(8&�� (A) �()��&' +� AƴI bought this jacket here the other day, and it’s too big. BƴWould you like to return it or exchange it for another item? Aƴ( ) (1) I’ll bring it back. (2) I believe it was in the chest. (3) I’d like my money back, please. (4) I don’t have any change on me. (B) � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �()�� !� �, AƴI can’t believe I failed my driving test! Bƴ( ) Most people don�t pass on the first try. (1) You’re right. (2) Don’t worry. (3) Not that way. (4) That’s the way! (C) � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �()�� !� �, AƴI�m thinking about buying a purple car. What do you think? Bƴ( ) you�d be better off with a white car. (1) In my opinion, (2) Lucky for you, (3) On the other hand, (4) Exactly as I thought, �Ƭ� Đ:uų:(�)ƹ(�)-L.L:ņå9�LK:9E26EƋÐ8ţħO(1)ƹ(10)�If3ƌ=ƪ-:İ�OũŊď9]eU(8&�� In a dentist’s office AƴWhat seems to be the problem? BƴI’ve had a toothache for the last few days. It really hurts when I eat cold things. Aƴ( (�) ) Does it bother you then?� � � � � � � � � � � � � � � �()�� &' +� BƴNot really. Mostly it’s just things like ice cream. AƴLet’s have a look. ( (�) )� � � � � � � � � � � � � � � � � � �()�� %� BƴOn the right side. In the back on the bottom. AƴHave you had problems with this tooth in the past? Bƴ( (�) )� � � � � � � � � � � � � � � � � � � � � � � � � � � �()�� &' +� AƴI see. ( (�) ) We’d better clean it out and fill it. � �()�� &' +� (1) Which one is it? (2) What about frozen food? (3) Where did I put my light? (4) I believe this is the first time. (5) How about when you eat hot food? (6) I still have some of my baby teeth. (7) Have you been in pain for a long time? (8) You can brush your teeth after every meal. (9) There seems to be a pretty large hole in it. (10) The two front teeth look like they are loose.


• Reference to situational contexts:

• At hand

• Background (including general knowledge)

• E.g., Ritsumeikan, 2008, [3(1)]

4

2008È Ŝŷ� Ň�Ƥ±·� 2/2,ƷdŠƷùÍ� þ~� ŏ¤� ŏĢ� §ƙR(§ƙ)� §ƙR(īČ)� §ƙR(òŋ)� §ƙR(Ĝ)� §ƙƖ|� īČĿu� Û«ĨÅ� röR(ö)� òŋł·� Ī�ł·� ö� öĨR(ŏ¤)� öĨR(ŏĢ)� öĨR(ĨÅ)� Ĝ� ĨÅ� Š �ƭ�� ƫ� Đ:(A)ƹ(C); AdBqr:¾ų5�K�-L.L:( )9�LK:9E26EƋÐ8ţħO(1)ƹ(4)�If3ƌ=ƪ-:İ�OũŊď9]eU(8&�� (A) �()��&' +� AƴI bought this jacket here the other day, and it’s too big. BƴWould you like to return it or exchange it for another item? Aƴ( ) (1) I’ll bring it back. (2) I believe it was in the chest. (3) I’d like my money back, please. (4) I don’t have any change on me. (B) � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �()�� !� �, AƴI can’t believe I failed my driving test! Bƴ( ) Most people don’t pass on the first try. (1) You’re right. (2) Don’t worry. (3) Not that way. (4) That’s the way! (C) � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �()�� !� �, AƴI’m thinking about buying a purple car. What do you think? Bƴ( ) you�d be better off with a white car. (1) In my opinion, (2) Lucky for you, (3) On the other hand, (4) Exactly as I thought, �Ƭ� Đ:uų:(�)ƹ(�)-L.L:ņå9�LK:9E26EƋÐ8ţħO(1)ƹ(10)�If3ƌ=ƪ-:İ�OũŊď9]eU(8&�� In a dentist’s office AƴWhat seems to be the problem? BƴI’ve had a toothache for the last few days. It really hurts when I eat cold things. Aƴ( (�) ) Does it bother you then?� � � � � � � � � � � � � � � �()�� &' +� BƴNot really. Mostly it’s just things like ice cream. AƴLet’s have a look. ( (�) )� � � � � � � � � � � � � � � � � � �()�� %� BƴOn the right side. In the back on the bottom. AƴHave you had problems with this tooth in the past? Bƴ( (�) )� � � � � � � � � � � � � � � � � � � � � � � � � � � �()�� &' +� AƴI see. ( (�) ) We’d better clean it out and fill it. � �()�� &' +� (1) Which one is it? (2) What about frozen food? (3) Where did I put my light? (4) I believe this is the first time. (5) How about when you eat hot food? (6) I still have some of my baby teeth. (7) Have you been in pain for a long time? (8) You can brush your teeth after every meal. (9) There seems to be a pretty large hole in it. (10) The two front teeth look like they are loose.

Figure 11. The reference to contexts by question type

0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#

Other#

Inferen5al#

Thema5c#

Organiza5onal#

Referen5al#

Paraphrasing#

Lexical#

None# Immediate#words# Word#associa5on# Situa5on#at#hand# Background#

Figure 12. The question types by question form

0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#

Gap#filling#

Comprehension##check#

Words#with##similar#meanigs#

TrueDfalse#

Reference#relaGon#

SorGng#

Listening#

RewriGng#

Lexical# Paraphrasing# ReferenGal# OrganizaGonal# ThemaGc# InferenGal# Other#

Figure 13. The reference to contexts by question form

0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#

Gap#filling#

Comprehension##check#

Words#with##similar#meanigs#

TrueDfalse#

Reference#relaGon#

SorGng#

Listening#

RewriGng#

None# Immediate#words# Word#associaGon# SituaGon#at#hand# Background#

Figure 14. The question types by O-orientation

0%# 20%# 40%# 60%# 80%# 100%#

O*type#in#total#(n#=#1508)#

O*oriented#(n#=#925)#

O*oriented#top#10#(n#=#315)#

Lexical#

Paraphrasing#

ReferenEal#

OrganizaEonal#

ThemaEc#

InferenEal#

Other#


• In dialogue,

• linguistic contexts 60%; situational contexts 40%

• In expository texts, more than 60% linguistic, mostly in a restricted range

• Comprehension check: Thematic & inferential about 60%

• Gap filling: Inferential about 80%

• Students have to read a lot of sentences and grasp the gist in the shortest possible time, with enough vocabulary

Figure 16. The number of questions by group

0% 20% 40% 60% 80% 100%

A�� O�� B��

Watari et al. (2013) survey

• 高校入試研究:

• 亘理陽一・石井亨・小川登子・奥住桂・加藤由美子・吉池陽子・根岸雅史

• 2013上智大学・ベネッセ英語教育シンポジウム（2013年12月1日、上智大学）

�Benesse%Corpora+on�


• Research Object: The questions of entrance exams for public high schools

• 47 prefectures, in 2003 & 2013, 3174 questions

• Classified each question in terms of the knowledge of English/abilities to use English, authenticity/task-likeness, levels of thinking

• To examine if there has been any change over a decade



• No basic difference in communicative orientation over a decade; More than 80% of questions already measured abilities to use English, and 20% required multiple skills at 2003.

• Tend to be more task-like: The questions with some settings, more often in someone’s positions

• But in most cases asking literal comprehension and controlled production


Conclusion: I’d like to say …

• the goals for exam preparation and CLT are not mutually exclusive; Rather, it could, or has to, build a mutually complementary relationship

• because actually

• university entrance exams ask students to deal with the B-type and O type questions more and more,

• and high school entrance exams ask them abilities to use English through task-like questions.

Conclusion

• “[T]he catch is that the test must offer students a genuine intellectual challenge, and teachers must be involved in designing the test if it is to be an effective point of leverage.” (Wiggins, 1989, p. 704)

References• ARCLE(編)（2014）『第５回研究会レポート上智大学・ベネッセ応用言語学シンポジウム2013報告: これからの中学校・高校での英語の指導と学びを考える -全国の高校入試分析と中高生の英語学習実態をもとに-』ベネッセ教育総合研究所.

• 門田修平・氏木道人・野呂忠司(編) (2010)『英語リーディング指導ハンドブック』大修館書店.

• 三浦孝 (2010)/「大学入試に関する2つの主張を検討する」『英語教育』59, 5, 24-25.

• 永倉由里 (2006).「英語教育の目的は何か：中学・高校・大学の生徒・学生と教師へのアンケート調査から」犬塚章夫・三浦孝編『英語コミュニケーション活動と人間形成』(pp. 55-66) 成美堂.

• Nuttall, C. (2005). Teaching reading skills: In a foreign language. Oxford: Macmillan.

• Richards, J. C., & Schmidt, R. (2010). Longman Dictionary of Language Teaching and Applied Linguistics (4th Ed.). London: Longman.

• 清水真紀 (2005).「リーディングテストにおける質問タイプ: パラフレーズ、推論、テーマ質問と処理レベルの観点から」『STEP Bulletin』17, 48-62.

• 白畑知彦・冨田祐一・村野井仁・若林茂則 (2009).『改訂版英語教育用語辞典』大修館書店.

• 卯城司 (編) (2009).『英語リーディングの科学: 「読めたつもり」の謎を解く』研究社.

• Wiggins, Grant (1989). A True Test: Toward more authentic and equitable assessment. Phi Delta Kappan 70, 9, 703-13.