Using Web-based Speech Recognition Technologies to Improve English Pronunciation

Howard Chen 陳浩然 hjchen@ntnu.edu.tw

English Department 師大英語系National Taiwan Normal University

Better Listening and Speaking Skills: GEPT and new TOEFL

Writing Section: There will be one 30-minute and one 20-minute essay.

Speaking Section: Six open-ended speaking questions require test takers to speak into a microphone.

Integrated Language: Some sections of the new test will combine four basic communication skills. For example, a test taker might listen to a lecture and read a passage, then write or speak about it.

Score: 1-120 (human graders for essay and speaking section).Time: 3.5 hours

Different Solutions

Some universities hire more native English speakers and provide student with more opportunities to interact with these teachers.

Some colleges have reduced the class size and expect that there will be more teacher-student interactions in the target language.

Some universities also begin to explore the power of new computer technologies-pronunciation tutor and ASR.

The Development of ASR in CALL

A few years ago, very few CALL (computer assisted language learning) programs claimed that they incorporated new speech recognition technologies. ‘

But within the past 5 years, automatic speech recognition (ASR) technologies are widely used in various language teaching and learning programs.

Some Programs Available in Taiwan

ASR Provides the Following Benefits:

Students will have more opportunities to produce the target language and have extensive interaction with computers.

Students will have individual attention, and they do not need to compete with other classmates.

Students can learn to communicate under a less threatening environment and they often can get feedback from computers quickly.

Students are provided with various direct or indirect feedback from computers.

Students can hear the models provided by different native speakers. Student can have better control about their learning pace and might have enhanced their confidence.

TRACI Talk

TeLL me More pro

The Scores Provided by MyET Program-via web

Prices Are Too High and Limited Access Although these PC-based speech recognition

software programs are quite useful, many students cannot afford to buy these products. (The price of Traci Talk or TeLLmeMore will be around 100 US dollars).

Some schools have purchased these software programs but students can only use these software programs in language center or language labs and they can not have access to these programs from dormitory or home.

Microsoft Speech Applications Developmental Kit The new Microsoft speech applications

developmental kit allows programmers and researcher to develop different web-based speech applications. The speech development kit requires a new programming environment based on the integration of the following software programs.

Microsoft .NET Speech SDK Microsoft Internet Information Services (IIS). Microsoft Internet Explorer 6.0 or later version. Microsoft Visual Studio .NET Professional Microsoft .NET Framework

The Flowchart of Interaction

NTNU Pronunciation Practice Web Site

NTNU Dept of English: ASR Technologies

Repeat

Conversation

Choose the Right Answer

Write and Say

Flash and Speech Recognition

Personal Learning Records

Checking Students’ Performances

Learners’ Evaluation The online system was completed after

extensive tests around January, 2005. We then invited 25 students (non-English

majors) who were taking Freshman English course to use this web site in the spring semester.

All of these college students were graduated from vocational high schools and most of them have difficulties in English pronunciations.

74% of these students felt that their pronunciations were poor in a survey.

Users Survey

Other Useful Information

most students indicated that they chose to try 3-5 times before they gave up trying.

If they find they were not sure about the pronunciations of a certain word, many students (65%) would choose to listen to the audios provided by the system.

It is also interesting to note that students (61%) find that they can pass the shorter sentences more easily. They also found that the “identify the object” is the easiest exercise (39%), followed by “listen and repeat”(35%) and “choose the appropriate response”(26%)

Some Positive Comments

Suggestions for Improvement

Reflection 1 The user interface can be made more interesting and

attractive. Some of the sections with TTS (text to speech) sounds can be replaced with human voices. The hardware of the web site server can be enhanced to provide better and reliable performances.

If the learning environment can be changed to game-like environment, that would be more attractive for users. We are currently developing a 2-D learning environment and use the Flash animation to make this site more interesting.

As for the problematic items, there seems be some bugs for certain items, we would need to find out the solutions these items. To identify and correct these items quickly will help students to reduce their frustrations in interacting with this site.

Reflection 2: Feedback Quality- Lower Threshold? In addition, the system so far can only give a pass

or fail judgment. For students at the lower proficiency level, sometimes they might find the system very demanding about their pronunciations.

Furthermore, the system cannot clearly pinpoint the problems of individual speakers. The students can only listen to the models carefully and try again and again. Perhaps we can figure out a better way of make the speech recognition system less demanding (e.g. lowering the sensitivity of speech recognition engine); however, we will then face the problem of setting the reasonable threshold.

How to Use the System in English Learning? Diagnosis: A screening device One possibility of utilizing this ASR system is to use

the system to assess large numbers of students’ pronunciation abilities. This system might be able to help to quickly identify students who are relatively weak in pronunciation. Then teachers and tutors might provide extra help.

Motivating learners to listen and speak. A tool for oral practice. (with the animation and other multimedia support )- Practice with any kind of sentences and phrases. (not confined to some fixed patterns).

Thank you! Questions and Feedback Welcome!