学位論文発表 michal mazur 修正

A Study on English Language Tutoring System Using Code-Switching Based Second

Language Vocabulary Acquisition Method

Graduate School of Information Science and Technology Language Media Laboratory

Michal MAZUR, DC3

2016.1.27

Doctoral Dissertation

HOKKAIDO UNIVERSITY

A Study on English Language Tutoring System Using Code-Switching Based Second Language Vocabulary Acquisition Method

DISSERTATION CONTENTS

1. Introduction

2. Conversational Agents

3. Second Language Vocabulary Acquisition

4. Emotions and Learning

5. The Co-MIX Project

6. Conclusions and Future Works

2

Section 1. Introduction

HOKKAIDO UNIVERSITY

A Study on English Language Tutoring System Using Code-Switching Based Second Language Vocabulary Acquisition Method 4

1. Background

• There is an increasing demand from a global society to learn foreign languages

• English is most widely taught as second language [1]; highest number of L2 speakers [2]; third most spoken language in the world [3]

• Knowing English is increasingly valued in Japan (e.g. job hunting, getting into university, communicating with growing number of tourists)

[1] Lewis, M. Paul, Gary F. Simons, and Charles D. Fennig (eds.). 2015. Ethnologue: Languages of the World, Eighteenth edition. Dallas, Texas: SIL International. Online version. Retrieved from: http://www.ethnologue.com (accessed 2016/1/3).

[2] The Concept of Global Human Resource Development Focusing on the East Asian Region. Retrieved from: http:// www.mext.go.jp/english/highered/1303540.htm (accessed 2015/3/14).

[3] No. of foreign visitors to Japan in 2014 highest ever. Retrieved from: http://www.japantoday.com/category/national/view/no-of- foreign-visitors-to-japan-in-2014-highest-ever (accessed 2014/2/4).

HOKKAIDO UNIVERSITY


• The expectations of English teachers towards students were found very low

➡ However, 79.0% of Japanese students would like to be able to use English; 58.6% are interested in reading books in English [4]

• Students: an issue of insuff ic ient vocabulary understanding among L2 learners; motivation problems [5]

• Teachers: the problem of preparing of valuable study materials for second language teaching

[4] Honna, N. (2008). English as a Multicultural Language in Asian Contexts: Issues and Ideas, 121-122. Tokyo: Kuroshio Publishers.

[5] http://www.japantimes.co.jp/opinion/2009/02/05/commentary/whats-wrong-with-the-way-english-is-taught-in-japan/#.Vos3rLzpYkA

2. Common problems

http://www.japantimes.co.jp/opinion/2009/02/05/commentary/whats-wrong-with-the-way-english-is-taught-in-japan/#.Vos3rLzpYkA

HOKKAIDO UNIVERSITY


“The most essential part of second language learning is vocabulary“ [7]

• Outdated approach (test-oriented; memorization techniques)

• Low retention of vocabulary (easily forgotten; problem of usage)

• Wilkins: “Without grammar very little can be conveyed, but without vocabulary nothing can be conveyed.” [6]

[6] Wilkins, D. A. (1972). Linguistics in language teaching. London: Edward Arnold [7] Ling, Z. (2010). Effectiveness of L2 Vocabulary Learning and LT-WM. Journal of Xinjiang University, 6, 135-139.

3. The importance of vocabulary

HOKKAIDO UNIVERSITY


• Many techniques have emerged to facilitate word acquisition, but none of them has proven to be universally effective

• Technological innovations are bringing the motivation for introducing new ways to enhance existing learning methods

• Tackling the problem of second language vocabulary acquisition in the field of computer science

• Code-switching > used in an e-learning to help expanding students’ second language vocabulary

4. My research topic

HOKKAIDO UNIVERSITY


• Proposing a new code-switching-based method in the field of computer science

• Approaching the problem of vocabulary acquisition among second language learners (insufficient vocabulary understanding)

• Successfully Implementing the proposed method into an E-learning system

• Gathering evidence to confirm the effectiveness of the proposed approach

5. Purpose and Significance of Research

8

HOKKAIDO UNIVERSITY


6. The novelty of this research

• A code-switching based vocabulary acquisition method is presented for the first time in an e-learning system. It has not yet been studied in the field of computer science

• Combining the phenomenon of code-switching with other existing methods of teaching vocabulary

• Using different ways of encouraging users to actively participate in a learning task

Section 2. Conversational Agents

HOKKAIDO UNIVERSITY


1. Introduction

Conversational agents (chatterbots): “Artif icial constructs that are designed to converse with human beings using natural language as input and output” [8]

• Chapter presents the past and the present of chatterbots

• It discusses the classification of conversational agents

• It presents a broad idea of a conversational system that would act as a language tutor

• Problem of encouraging users’ interaction is discussed

[8] Brennan, K. (2006). The managed teacher: Emotional labour, education, and technology. Educational Insights, 10(2), 55-65.

HOKKAIDO UNIVERSITY


• The potential application of conversational agent for education

• Some insight on the roles that conversational systems play in the modern society

• The idea of a conversational agent using code-switching is presented

• The need of the conversational system to react to changing emotional states of users is discussed

2. Some discussed problems

HOKKAIDO UNIVERSITY


Figure 1. The overview of code-switching-based system with modules

Co-MIXMODULE

Section 3. Second Language Vocabulary Acquisition

HOKKAIDO UNIVERSITY


1. Introduction

• In a spoken language about 1,800 words represent about 80% of the spoken corpus [9]

• Some frequent words are often repeated > learners can understand a large portion of foreign language conversation with a relatively small vocabulary [10]

• The way that new words are presented to students usually depends on teachers’ teaching strategies

[9] McCarthy, M. J. (2004). Touchstone: From Corpus to Course Book. Cambridge: Cambridge University Press.

[10] McCarten, J. (2007). Teaching Vocabulary. Lessons from the Corpus. Lessons for the Classroom. Cambridge: Cambridge University Press.

HOKKAIDO UNIVERSITY


• Even one time exposure to a target word is able to contribute to a student’s initial vocabulary knowledge [11]

• Language learners can learn new vocabulary and spelling more efficiently by receiving comprehensible feedback while they read [12]

• Humans tend to be good at inferring meaning from context

➡ Single words can be understood from the context of the surrounding text [13]

[11] Nagy, W. E., Anderson, R. C., & Herman, P. A. (1987). Learning word meanings from context during normal reading. American Educational Research Journal, 24(2), 237-270. [12] Krashen, S.D. (1985). The Input Hypothesis: Issues and Implications. New York: Longman. [13] Labutov, I., & Lipson, H. (2014). Generating Code-switched Text for Lexical Learning. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: Volume 1: Long Papers (pp. 562–571). Baltimore: Association for Computational Linguistics.

HOKKAIDO UNIVERSITY


2. Definitions

• Incidental learning

• High frequency word list

• Extensive reading

• Graded readers

• Repetition Example 1. Typical extensive reading textbook

HOKKAIDO UNIVERSITY


• Code-switching- „The TENKI forecast is great for the coming days.” - Japanese word within an English

sentence.

- „Boku ne, petto hoshii na. Rabbit toka, gerbil toka ne.” (I want to have a pet. For example, rabbit or gerbil.) - English words within a Japanese sentence

• a brief insertion of a word from one language into another

• An agreement on the positive effects of code-switching in learning [14]

• Recently a target of interest for researchers in different fields (psychologists, linguists and computer scientists)

= code mixing; code changing

[14] Celik, M. (2003). Teaching vocabulary through code-mixing. ELT Journal, 57(4), 361-369.

HOKKAIDO UNIVERSITY


• It can be used by less proficient people to fill linguistic gaps

• When planned ahead of use, it can greatly contribute to more efficient understanding of a given topic [15]

Example 2. Code-switching in a real life situation

[15] Cook, V. (2001). Using the first language in the classroom. Canadian Modern Language Review/La Revue canadienne des langues vivantes, 57(3), 402-423.

Section 4. Emotions and Learning

HOKKAIDO UNIVERSITY


1. Introduction

• Chapter continues exploration of code-switching method, as a way of acquiring second language vocabulary

• The foundation of the work on a system that would give students an opportunity to expand and practice their knowledge of second language vocabulary

• Check how example sentences that appeal to emotions will affect the learning process of test subjects and their performance when studying with code-switching method

HOKKAIDO UNIVERSITY


2. Related background

• Ekman’s list of six basic emotions [16]

Example 3. Six basic emotions (Source: The Grimace Project)

[16] Ekman, P. (1999). Facial expressions. Handbook of cognition and emotion, 16, 301-320.

e . g . “ T h e r e w a s a n

announcement that my

f r i e n d w i l l t r a n s f e r

schools, so I’m sad.”

s

HOKKAIDO UNIVERSITY


• The existence of a link between learning and some affective states [17]

➡ Emotions can modify thinking and problem solving ➡ Emotional stimuli tend to be better remembered than

unemotional [18]

• Emotional selectivity in human memory [19]

➡Items associated with emotional reactions tend to be learned better

➡There seems to be a memorizing advantage for affective material

[17] Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004). Affect and learning: an exploratory look into the role of affect in learning with AutoTutor. Journal of educational media, 29(3), 241-250 [18] Bower, G. H. (1992). How might emotions affect learning. The handbook of emotion and memory: Research and theory, 3, 31. [19] Parrott, W. G., & Spackman, M. P. (2000). Emotion and memory. Handbook of emotions, 2, 476-490.

HOKKAIDO UNIVERSITY


• Teach students new English words with proposed code-switching method

• Check if the students are able to understand the meaning of the word from context of the whole sentence using presented method

• Examine how sentences that appeal to emotions will affect the learning process of test subjects and their studying performance

3. Preliminary experiment (Purpose)

HOKKAIDO UNIVERSITY


4. Method

• Conducted with 15 Japanese students (age 10-13)

• The participants were asked to study sentences with code-switching (in Japanese and English). Sentences were presented on iOS tablet

• Selected nouns were replaced. Each round of the experiment introduced two English words

• Participants comprehension of selected words was double-checked before and after the experiment

HOKKAIDO UNIVERSITY


• Two parts of the experiment:

1. Learning part (4 pattern sentences with proper context; 1 minute for each subject)

• Example sentences:

a) Kaigai ni sunde iru indojin ni totte wa, boukoku no culture wo ikasu no ga juuyou desu. (For Indians who live overseas, keeping their home culture is important.)

b) Gakkou no tomodachi ga tenkou ni naru to iu announcement ga kite, ima wa kanashii desu. (There was an announcement that my friend will transfer schools, so I’m sad).

HOKKAIDO UNIVERSITY


2. Testing part (8 sentences; 4 with proper context and 4 with improper context)

• Two tests (emotional and unemotional sentences) • Participants were asked to select proper sentences

• Example sentences:

a) Resutoran de announcement wo chuumon shitara, oishikatta no de ureshikatta desu. (I ordered announcement in the restaurant and it was tasty, so I was glad.)

b) Kare no sightseeing ga warukatta no de, jugyou wo yasumimashita. (Because his sightseeing was bad, he was absent in the class).

HOKKAIDO UNIVERSITY


5. ResultsTable 1. The percentage of correct answers for each question in both phases of

the experiment

Table 2. The percentage of correct answers in both test phases

HOKKAIDO UNIVERSITY


6. Discussion

• Percentage of correct answers in both tests was high -> indicates the effectiveness of code-switching method

• Participants successfully understood most of second language words without direct translations

• The difference between the number of correct answers in both phases was 3.4%, in favor of unemotional sentences

• Code-switching method is suitable to be implemented in the second language vocabulary teaching system

Section 5. The Co-MIX Project

HOKKAIDO UNIVERSITY


1. Co-MIX Project

• Create applications that utilize code-switching method to teach second language vocabulary to students

• Major contribution: Co-MIX Japanese, a teaching system using code-switching for English vocabulary acquisition

• A preliminary evaluation of another system, aimed at English-speaking users who want to study Japanese

HOKKAIDO UNIVERSITY


2. My assumptions

• Code-switching will expand students second language vocabulary

• The user attitudes towards the proposed system will be generally positive

• The users will benefit from code-switching method and the scores in language tests while using the proposed system will be higher

• Users will be able to successfully obtain the meaning of a number of second language words from context

HOKKAIDO UNIVERSITY


3. Co-MiX Japanese• Harnesses the benefits of code-switching, incidental

learning and advantages of learning vocabulary in context

• Two modules:

1. Study material generator 2. Language quiz generator

• Two systems are used in this research (proposed and baseline)

• Both systems are provided with a selection of graded articles in Japanese, and their translations in English

HOKKAIDO UNIVERSITY


VOCABULARY STUDYING METHODS

BASELINE PROPOSED

Incidental learning Incidental learning

Extensive reading Extensive reading

Graded readers (English) Graded readers (mixed language)

Repetition Repetition

High frequency words list High frequency words list

Code-switching method

Example 4: Vocabulary studying methods used by the baseline and the proposed systems

HOKKAIDO UNIVERSITY


4. Research questions

• Main question: “Can code-switching phenomenon be successfully employed in e-learning to expand students’ second language vocabulary?”

• Two sub-questions: “Can the code-switching method enable students to understand words without given definitions?”

“What are the user attitudes towards the proposed system?

HOKKAIDO UNIVERSITY


Figure 4. Process of study material generator

(Co-MIX Japanese)

5. Study material generator

HOKKAIDO UNIVERSITY


e.g.

(1)昨日は日曜日で、とてもいい(2)天気でした。あんなに素晴らしい(2)天気は本当に珍しいです。(17)交換留学生の木村さんは日本語(4)学科のジャックさん一緒に(5)自転車で新宿へ行きました。そして、新宿のショッピングセンターで(6)買い物をしました。

(1)昨日 | yesterday | 1 (2)天気 | weather | 2

(3)交換留学生 | exchange student | 2 (4)学科| department | 2 (5)自転車 | bicycle | 4 (6)買い物 | shopping | 6 —— 3.93

1. Indexed

vocabulary

2. Editable

vocabulary list

3. Average

difficulty ofarticle

• Format of the dictionary:

[kanji] | [English translation] | [difficulty]

Example 4. The output file of study material generator (Co-MIX English)

HOKKAIDO UNIVERSITY


6. Language quiz generator

Figure 5. Flow process chart of the language

quiz generator (Co-MIX Japanese)

HOKKAIDO UNIVERSITY


Figure 8. Process of third phase of the test

(Co-MIX Japanese)

HOKKAIDO UNIVERSITY


7. Evaluation experiment: (Breakdown of 40 participants)

Gender:Male: 21 Female: 19

Average age: 28.3

Education:University students: 24 Graduates: 16

Studying English from:Junior high school: 38 Elementary school: 2

Continue studyingafter graduation:Less than 2 years: 8 3-5 years: 3 More than 5 years: 5

HOKKAIDO UNIVERSITY


8. Procedure

1. Respondents used two presented systems. User results were automatically saved in a result file to be later used in data analysis

2. Respondents filled out a paper survey after finishing the learning tasks with both systems

3. Respondents rated both systems by marking 14 word pairs on a 7-point semantic differential scale

4. Respondents ranked both systems based on their overall performance

HOKKAIDO UNIVERSITY


9. Experiment method

• Semantic Differential (SD) scale [17]

➡Used in a variety of studies and contexts to measure people’s attitude towards stimulus words and concepts

➡Ratings on SD scale are often correlated

➡SD scale can be used to measure users’ perceived satisfaction with an e-learning system and to assess its value [18]

[17] Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. Urbana: Univer. of Illinois Press.

[18] Levy, Y. (2006). Assessing the value of e-learning systems. IGI Global, pp. 93-94.

HOKKAIDO UNIVERSITY


Figure 10. Semantic differential scale for proposed and baseline systems (Co-MIX Japanese)

HOKKAIDO UNIVERSITY


10. Factor analysis

• Two separate sets of statistical tests were carried out on the baseline and proposed systems using the result data acquired from the SD analysis

• The purpose was to evaluate each set of data separately and then compare their characteristic features

• Data from the system evaluations were submitted to factor analysis (using SPSS 21*)

* IBM SPSS 21: http://www-01.ibm.com/software/analytics/spss/products/statistics/

HOKKAIDO UNIVERSITY


Table 4. Factor analysis results for baseline

system. The highest significance in bold font.

Table 3. Factor analysis results for the proposed

system. The highest significance is in bold font.

HOKKAIDO UNIVERSITY


11. User rankings

• Respondents were asked to rank both systems based on their performance and then state which of the systems they prefer

Example 5. User rankings for three questions asked after performing the experiment

HOKKAIDO UNIVERSITY


12. Analysis of user mistakes

Table 5. Average number of mistakes for proposed and baseline systems. Lower number in bold.

• The average number of mistakes the participants made was higher for the baseline

➡transfer mistakes (e.g. „conbersashon”; „curassu”, „libing”) ➡vocabulary mistakes (e.g. „creation” [criterion];„becoming” [beginning]) ➡spelling mistakes (e.g. „univercity” [university]; „bycicle”[bicycle])

HOKKAIDO UNIVERSITY


13. Discussion

• A higher number of correct answers in the proposed system with code-mixing

• Time spent with the proposed system was longer (the average answering time for the proposed system was 11.6 minutes and in case of the baseline 8.8 minutes)

• Smaller number of mistakes > better memorization

• Positive user opinions, e.g: „the proposed system was more encouraging to interact with”

HOKKAIDO UNIVERSITY


14. Co-MIX English

• Target group: English-speaking people with some background in Japanese language education

• For English-speaking foreigners who want to study Japanese vocabulary

• Another implementation of code-switching based method

• Purpose: to find more evidence that the code-switching-based method used in the previous system can be adapted in other languages

HOKKAIDO UNIVERSITY


15. Study material generator

Figure 11. Process of study material generator (Co-MIX English)

HOKKAIDO UNIVERSITY


TV is a (1)thief of time. You can come (2)home tired, and when you feel like you don't want to do anything else, you turn the TV on. It generates the (3)noise and fills the (4)room with rush, repeatedly emitting (5)news, and the results of the sports from throughout the (6)day. It feels like a (7)dream.

(1)thief | 泥棒 | どろぼう | 4 (2)home | 家庭 | かてい | 5 (3)noise | 騒音 | そうおん | 3 (4)room | 部屋 | へや | 5 (5)news | 情報 | じょうほう | 3 (6)day | 日 | ひ | 5 (7)dream | 夢 | ゆめ | 4

• Format of the dictionary:

[English word] | [kanji] | [reading] | [difficulty]

1. Indexedvocabulary

2. Editable

vocabulary list

Example 6. The output file of study material generator (Co-MIX English)

HOKKAIDO UNIVERSITY


16. Language quiz generator

Figure 12. Process of language quiz generator (Co-MIX English)

HOKKAIDO UNIVERSITY


*”Tanaka Corpus of parallel Japanese-English sentences”; http://www.edrdg.org/wiki/index.php/Tanaka_Corpus

1. Correct

word

2.typing

mistake

Supports input in three

Japanese writing systems

[Test 1] Please fill in the missing word:

[Question 1/20] Please type the Japanese word for "tea" (o _ h _):

> ocha

Correct!

[Question 2/20] Please type the Japanese word for "center" (c _ _ _ _ _ i _):

> 中心

Correct!

[Question 3/20] Please type the Japanese word for "place" (t _ _ _ r _):

> ところ

Correct!

[Question 4/20] Please type the Japanese word for "exchange" (k _ _ _ a n ):

> kokan

Incorrect. It should be: '交換 (こうかん)'. Example sentence:

"このセーターを大きいのと交換していただけますか。" "Will you exchange this sweater for a larger one?"

[Question 5/20] Please type the Japanese word for "food" (t _ _ _ m _ n _):

> たべ

3. example

sentence*Example 7: The example of language test (Co-MIX English)

http://www.edrdg.org/wiki/index.php/Tanaka_Corpus

HOKKAIDO UNIVERSITY


17. Evaluation experiment (Breakdown of 16 participants)

Gender:male: 12 female: 4

Studying Japanese:less than 6 months: 2 1-3 years: 3 2-5 years: 10 (62.5%) more than 5: 1

Occupation:employee: 8 student: 6 self-employed: 1 housewife: 1

Education:university students: 6 graduates: 10

Age:20+: 10 30+: 6

HOKKAIDO UNIVERSITY


18. Experiment Method

• University students and company employees participated in a preliminary evaluation experiment and interacted with the proposed system (Co-Mix English)

• Participants were asked to fill out a paper survey after they finished the learning tasks with a given system

• Samples from proposed and baseline system experiment results were compared to check test scores and user attitudes towards baseline and proposed systems

HOKKAIDO UNIVERSITY


19. SD Scale

Figure 15. Semantic differential scale for proposed and baseline systems (Co-MIX English)

HOKKAIDO UNIVERSITY


20. Discussion

• A higher number of correct answers in the proposed system with code-mixing

• Time spent with the proposed system was longer (the average answering time for the proposed system was 11.8 minutes and in case of the baseline 8.8 minutes)

• Experiment demonstrated more evidence that the proposed method is useful and effective way of expanding students’ second language vocabulary

Section 5. Conclusions and Future Works

HOKKAIDO UNIVERSITY


1. Conclusions

• The participants improved their vocabulary scores after using the Co-Mix Japanese, and the Co-MIX English systems

• High percentage of correct answers, and positive feedback from the participants suggest that presented method was well received

• The extension to support another language was successful

HOKKAIDO UNIVERSITY


2. Future Works

• Higher user satisfaction with the proposed system also provides motivation to improve and refine it in a number of different ways in the future

• Another series of tests to compare Co-MIX Japanese and Co-MIX English systems

• Advance the project towards the initial idea to provide a chatterbot able to process user queries in mixed languages

HOKKAIDO UNIVERSITY


ご清聴ありがとうございました。

„Thank you for your attention!”

Do you have any 質問? 何かquestionはありますか？

HOKKAIDO UNIVERSITY


*”Tanaka Corpus of parallel Japanese-English sentences”; http://www.edrdg.org/wiki/index.php/Tanaka_Corpus

1. Correct

word

2.typing

mistake

Supports input in three

Japanese writing systems



> ocha

Correct!


> 中心

Correct!


> ところ

Correct!


> kokan




> たべ

3. example

sentence*Example 7: The example of language test (Co-MIX English)

http://www.edrdg.org/wiki/index.php/Tanaka_Corpus

63

1. Correct

word

2.typing

mistake



> ocha

Correct!


> 中心

Correct!


> ところ

Correct!


> kokan




> たべ

3. example

sentence*

Documents

学位論文発表 michal mazur 修正