Exploring the Effect of the Self-Directed English Learning on the English Speaking Test

Scores of Korean College Students

Jungtae Kim, Ph.D.Department of TESOL,

Pai Chai University

Backgrounds

• Criticism of English courses at Korean colleges–Ambiguous goals: What to teach?–Mass instruction: more than 35 ss– Teacher centered–Mismatch between students’ needs and

English education at colleges– TOEIC L/R?– Requiring English speaking scores from

companies

1. The need of communicative competence

Students’

needs

Social

needs

Commu

nicative

compete

nce

2. What do they really want?

• Students’ needs– Achieving high speaking ability

– Practical English > academic English

– TOEIC L/R?

• Social needs– English speaking skills?

– OPIc, TOEFIC Speaking

> Writing skills?

> Reading skills?

3. Colleges’ Reaction to the needs

• TOEIC L/R

• Conversations courses

• Teaching strategies for high scores of commercials tests

• Adopting TOEIC Speaking test, OPIc

• Mass instruction

• Passive students

4. Student Interview & Survey

• High TOEIC score

• Conversation skills

• Fluency

• Good pronunciation

• Reading (TOEIC Listening > Reading scores)

• Feedback

5. Students’ SDL capability

Many students are

• Low motivation

• Low self-directedness

• Low self-esteem

• Low self-confidence

• High English anxiety

• Test wiseness

=> Self-Directed English Learning

Theoretical Foundations

• Self-directed language learning relies on the student to have acquired learner autonomy.

• Little, D. (2014)

Autonomous learners understand the purpose of their learning program, explicitly accept responsibility for their learning, share in the setting of learning goals, take initiatives in planning and executing learning activities, and regularly review their learning and evaluate its effectiveness (cf. Holec 1981, Little 1991)

• Lee and Son (2007)

- A difficult situation of learning English in Korea

- Limited English learning opportunities: limited time to learn English in the classroom

- The concept of self-directed learning is quite necessary in addressing common challenges uniquely associated with EFL learning.

• Dam & Legenhausen 1996, Legenhausen1999.

Autonomisation = strategy training

• Autonomy in language education covers

Learning strategies, self-regulation, motivation, individual differences, sociocultural approaches, teacher development

• In sum, mostly language learning strategies and attitudes to language learning were researched.

• The relationship between SDEL capability

and language ability improvement –elementary level, a little research on college level

• A gap between self-directed language learning and L2 speaking ability

Research Question

To what extent are students’ SDEL capability correlated with their English speaking scores?

• SDEL capability vs. English speaking scores

PCU SDEL Model

Learning

AssessmentCoaching

1. English Consulting System

Assessment -

Diagnosis

- Online survey

- L/S+R/W

- Rater Training

Placement-

Learning

-Different levels

-Self-directed

learning

-Computer-

Assisted Learning

Self-directed

learning

Coaching

-Training coaches

-Management of

learning

– Weekly group

meeting

-

2. Self-directed English Learning System

SDEL

• Online self-directed English

learning survey

ESWT

• English Speaking & Writing

Tests

DynEd

Real life Conversation

Improving speaking

proficiency

30min/day, 4days/week

-> 25 hrs/semester

Graduation Requirement

Reading Assistant

Reading aloud

Improving fluency +

reading speed

Methodology

1. Participants

• 90 Korean English learners at PCU, Daejeon

• 37 male, 53 female

• 20-30 years old

• Diverse English levels

• TESOL or double major

2. SDEL Survey

• Developed by Y. W. Cho based on – Griffiths, C. (Ed.). (2008). Lessons from good language learners.

Cambridge University Press.

– Dornyei, Z., & Ushioda, E. (Eds.). (2009). Motivation, language identity and the L2 self. Multilingual Matters.

• The survey consists of 10 features:(1) interpersonal ability, (2) self-esteem, (3) self-confidence,

(4) resisting to anxiety with English, (5) goals, (6) motivation,

(7) self-directedness, (8) information process ability,

(9) self-understanding, and (10) overall level.

• Reliability: Cronbach’s alpha =.76

3. ESWT

• ESWT(English Speaking & Writing Test)

• Providing English speaking/writing scores per semester

• Preparing for TOEIC Speaking or OPIc

• Monitoring score improvement over years

4. Test administration

• Spring, 2014

• Mass admin: 60 Ss per time

• Computer-Based Test

5. Introduction to ESWT

• Speaking Part (4 items)– Self introduction

– Picture story telling with 6 pictures of students’ real life

– Describing a chart

– Giving opinions

– Preparation time: 30 sec

– Response time: 1 min

– 2 chances for answering

• Writing Part (1 item)

– Giving opinions

– 15 min for preparation and response

6. Rating

• Two raters, Analytic+Holistic Scoring

• Rater training – 3 time, 6 hours per semester

• 10 scales: 0, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5

• Scoring Rubric based on ACTFL Scoring Guidelines

ESWT Standard Setting

ESWT 토익스피킹 OPIc 토익L/R

5 8 AH 950-990

4.5 7 AM 915-945

4 7 AL 880-910

3.5 6 IH 780-875

3 6 IM 675-775

2.5 5 IL 535-670

2 3,4 NH 345-530

1.5 2 NM 0-345

1 1 NL 0-345

0 0 0 0

Scoring Constructs

Feature Descriptors

Task

Completion

The speaker’s ability to complete tasks by selecting language functions to reas

onably address the task and to select the language needed to carry out require

d functions in academic settings.

Fluency The speaker’s natural flowing speech including hesitation, repetition,

re-structuring, inappropriate words and sentences in academic settings.

Accuracy The speaker’s appropriateness of pronunciation, grammar, and language use i

n academic settings.

Coherence The speaker’s clear and logical organization of his/her utterances in academic

settings

Level Descriptors

Score Descriptors

5 The speaker’s communicative competence is almost always effective in terms of fluency, functional compete

nce, accuracy, and coherence of their speech in academic settings. They perform given tasks very competently

. It is near-native speaker’s English proficiency.

4 The speaker’s communicative competence is generally effective in terms of fluency, functional competence,

accuracy, and coherence of their speech in academic settings. They perform given tasks competently.

3 The speaker’s communicative competence is somewhat effective in terms of fluency, functional competence,

accuracy, and coherence of their speech in academic settings. They perform given tasks somewhat competentl

y.

2 The speaker’s communicative competence is NOT generally effective in terms of fluency, functional compet

ence, accuracy, and coherence of their speech in academic settings. They do NOT perform given tasks general

ly competently.

1 The speaker’s communicative competence is extremely poor in terms of fluency, functional competence, acc

uracy, and coherence of their speech in academic settings. They can NOT perform given tasks at all.

0 No Response

Experimental Rating Procedure

• Top-down rating

1. Task completion

2. Holistic Score (allow 1 scale difference)

3. Coherence

4. Fluency & Pronunciation

5. Language use

6. Grammar

Results

1. ESWT Scores Stat

Variable Mean S.D. Min Max Range

S_level 2.50 0.93 1.00 5.00 4.00

Task 2.94 1.10 1.00 5.00 4.00

Coher 2.38 1.09 1.00 5.00 4.00

Fluency 2.41 1.01 1.00 5.00 4.00

Pronun 2.60 1.03 1.00 5.00 4.00

LangUse 2.37 0.95 1.00 5.00 4.00

Grammar 2.35 0.88 1.00 4.75 3.75

2. SDEL Survey Stat

Variable Mean S.D. Minimum Maximum Range

InterP 5.59 1.11 1.50 7.00 5.50

SelfES 5.15 0.98 3.29 7.00 3.71

SelfConf 5.26 0.91 3.29 7.00 3.71

EnAnxi 4.45 1.17 1.86 7.00 5.14

Goals 4.93 1.07 3.00 7.00 4.00

Motiv 5.74 0.97 3.67 7.00 3.33

SelfDir 4.57 1.19 2.63 7.00 4.37

InforPro 4.92 0.95 2.25 7.00 4.75

SelfUnder 5.72 1.04 2.33 7.00 4.67

SDLevel 5 0.83 3.57 6.59 3.02

Crosstab = ESWT Level vs. SDEL Level

ESWT_level

1 1.5 2 2.5 3 3.5 4 4.5 5 Total

SDEL

Level

4 6 7 10 1 4 0 0 0 0 28

5 4 3 9 4 5 4 1 1 1 32

6 1 2 3 4 9 1 3 1 2 26

7 0 1 0 1 2 0 0 0 0 4

Total 11 13 22 10 20 5 4 2 3 90

ESWT – beginner level (46), intermediate (35), advanced (9)Beginner ESWT+Mid SDEL=10, Beginner ESWT+Level 5 SDEL=9 Intermed ESWT+Level 6 SDEL=9

Spearman Correlation Coefficients

Variable ESWT_level SDEL_level

ESWT_level 1.00

SDEL_level 0.45 1.00

Mean 2.40 5.08

S.D. 0.93 0.83

Spearman Correlation Coefficients – ESWT

Variable S_level Task Coher Fluency Pronun LangUse Grammar

S_level 1.00

Task 0.94 1.00

Coher 0.96 0.94 1.00

Fluency 0.96 0.91 0.96 1.00

Pronun 0.88 0.78 0.84 0.85 1.00

LangUse 0.96 0.91 0.91 0.92 0.86 1.00

Grammar 0.94 0.89 0.90 0.91 0.83 0.97 1.00

Mean 2.40 2.94 2.38 2.41 2.60 2.37 2.35

S.D. 0.93 1.10 1.09 1.01 1.03 0.95 0.88

Spearman Correlation Coefficients – SDEL Survey

Variab InterP SelfES SelfC EnAnxi Goals Motiv SelfDir InforPr SelfUn SDLev

InterP 1.00

SelfES 0.682 1.000

SelfConf 0.510 0.723 1.000

EnAnxi 0.349 0.646 0.575 1.000

Goals 0.360 0.558 0.622 0.616 1.000

Motiv 0.350 0.539 0.580 0.490 0.627 1.000

SelfDir 0.339 0.591 0.675 0.576 0.737 0.643 1.000

InforPr 0.473 0.636 0.728 0.661 0.737 0.631 0.689 1.000

SelfUn 0.358 0.553 0.548 0.414 0.482 0.550 0.546 0.546 1.000

SDLeve 0.637 0.841 0.841 0.778 0.827 0.757 0.829 0.867 0.624 1.000

Mean 5.592 5.154 5.265 4.451 4.925 5.735 4.573 4.922 5.722 5.076

S.D. 1.111 0.984 0.911 1.167 1.073 0.966 1.187 0.947 1.043 0.829

Spearman Correlation Coefficients – ESWT vs. SDEL Survey

Variable S_level Task Coher Fluency Pronun LangUse Grammar

InterP 0.24 0.26 0.27 0.22 0.30 0.24 0.16

SelfES 0.33 0.34 0.37 0.35 0.34 0.33 0.30

SelfConf 0.44 0.43 0.46 0.45 0.41 0.46 0.44

EnAnxi 0.31 0.30 0.34 0.32 0.26 0.30 0.30

Goals 0.38 0.35 0.40 0.39 0.35 0.37 0.35

Motiv 0.38 0.34 0.37 0.39 0.37 0.35 0.32

SelfDir 0.44 0.42 0.45 0.45 0.40 0.45 0.42

InforPro 0.38 0.38 0.40 0.37 0.35 0.39 0.36

SelfUnder 0.28 0.28 0.29 0.25 0.31 0.27 0.25

SDLevel 0.45 0.44 0.48 0.46 0.44 0.45 0.42

Mean 2.40 2.94 2.38 2.41 2.60 2.37 2.35

S.D. 0.93 1.10 1.09 1.01 1.03 0.95 0.88

Spearman Correlation Coefficients- SDEL criteria vs. ESWT Level

• The correlation coefficients between the SDEL criteria and the overall ESWT score: mostly low - 0.22 to 0.45.

• The lowest correlation coefficient was 0.22 between motivation and the overall ESWT score

• The highest coefficient was 0.45 between self-directedness and the overall ESWT score.

Spearman Correlation Coefficients- ESWT criteria vs. SDEL Level

• The correlation coefficients between the overall level of the SDEL and the scoring criteria of the ESWT - 0.38 to 0.47

• The lowest correlation coefficient was 0.38 between the overall level of the SDEL and the grammar score

• The highest coefficient was 0.47 between the overall level of the SDEL and the fluency score

• A post-hoc test with High and Low groups - low and moderate correlations

• the students’ SDEL capability might not affect their English speaking scores.

Case 1=High SDEL w/ Low ESWT LevelGrade 3, Female, Married to American

S_level Task Coher Fluency Pronun LangUse Gramma

2.8 4.25 3.25 3.2 2.4 3 3.5

SDLevel InterP SelfES SelfCon EnAnxi Goals Motiv SelfDir InforPr Self_Un

6.52 4 6.14 7 7 7 7 7 7 7

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

S_level Task Coher Fluency Pronun LangUse Grammar

- Lack of interaction w/ classmates- Low Inter personal ability - Slow speaker- Fossilization on her pronunciation- High Task completion- Unique pattern

Case 2=Low SDEL w/ Low ESWTGrade 1, Female, Low motivation

S_level Task Coher Fluency Pronun LangUse Grammar

1 1 1 1.2 1.2 1.5 1.75

SDLevel InterP SelfES SelfConf EnAnxi Goals Motiv SelfDir InforPro SelfUnd

4 5 4 4 3 5 5 5 5 6

0

1

2

3

4

5

6

7

- Mid SDEL w/ low ability to overcome English anxiety

- Mid self-esteem & self-confidence - Very beginner level in L2 speaking - Not expect high achievement in

the future in English speaking due to low SDEL

Case 2=High SDEL w/ Mid ESWTGrade 4, Female, high motivation

S_level Task Coher Fluency Pronun LangUse Grammar

3 3.5 3 3 4 3 2.5

SDLevel InterP SelfES SelfConf EnAnxi Goals Motiv SelfDir InforPro SelfUnd

6 7 6 7 6 6 7 5 6 7

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

S_level Task Coher Fluency Pronun LangUse Grammar

- High SDEL w/ relatively low ability of self-directedness

- High GPA- Expected advanced level, but

intermedaite speaking scores- Low accuracy - Maybe nervous on taking

speaking test

Discussion

• Low reliability from the SDEL Survey with different students – Cronbach alpha: 0.76, but moderate correlation among the features of the SDEL Survey.

• High SDEL ability does NOT guarantee high English speaking scores.

• High interest in English learning does NOT guarantee high English speaking scores.

• Dominant low level ESWT scores w/ various SDEL levels

• Troubles to make connections between low level ESWT scores w/ mid SDEL levels

• Caring low level SDEL levels along with improving English speaking ability

• Without maintaining high SDEL capability, NO English speaking improvement!

Thank you for listening.

Contact: kim.jungtae@gmail.com