1

Qualitative Inquiry & Digital EFL Learning

A Theory-Driven QI in Response to Recent Paradigm Shifts

in TESOL and on the Internet

Chin-chi ChaoAssociate Professor

National Chengchi University國立政治大學副教授招靜琪

Presented at the National Chung Cheng University, December 30, 2008.

2

Agenda

1. Paradigm shift in TESOL: more open to QI

2. Paradigm shift on the Internet: Web 2.0

3. The case of a theory-driven QI4. The contributions of QI

3

1. Recent Paradigm Shift in TESOL

4

Recent Debate Paradigm Shift

Applied Linguistics (Special issue, 1993), Modern Language Journal (1994), TESOL Quarterly (1997), Paradigm Shift: Understanding and Implem

enting Change in Second Language Education (Jacobs & Farrell, 2001).

The Social Turn in Second Language Acquisition (Block, 2003),

Cognitive and sociocultural perspectives: Two Parallel SLA Worlds? (Zuengler & Miller, 2006)

5

Key components of the PS Jacobs, G. M., Farrell, T. S., (2001).

• Focusing greater attention on the role of learners rather than the external stimuli learners are receiving from their environment. Thus, the center of attention shifted from the teacher to the student. This shift is generally known as the move from teacher-centered instruction to learner-centered or learning-centered instruction.

• Focusing greater attention on the learning process rather than on the products that learners produce. This shift is known as a move from product-oriented instruction to process-oriented instruction.

• Focusing greater attention on the social nature of learning rather than on students as separate, decontextualized individuals.

6

• Focusing greater attention on diversity among learners and viewing these differences not as impediments to learning but as resources to be recognized, catered to and appreciated. This shift is known as the study of individual differences.

• Focusing greater attention on the views of those internal to the classroom rather than solely valuing the views of those who come from outside to study classrooms, evaluate what goes on there and engage in theorizing about it. This shift led to such innovations as qualitative research - with its valuing of the subjective and affective, of the participants' insider views and of the uniqueness of each context.

• Along with this emphasis on context came the idea of connecting the school with the world beyond as a means of promoting holistic learning.

7

• Helping students to understand the purpose of learning and develop their own purposes.

• A whole-to-part orientation instead of a part-to-whole approach. This involves such approaches as beginning with meaningful whole texts and then helping students understand the various features that enable to texts to function, e.g., the choice of words and the text's organizational structure.

• An emphasis on the importance of meaning rather than drills and other forms of rote learning.

• A view of learning as a lifelong process rather than something done to prepare for an exam.

8

8 Changes as Part of the Paradigm Shift in Second Language Education

• Learner autonomy • Cooperative learning • Curricular integration • Focus on meaning • Diversity • Thinking skills • Alternative assessment • Teachers as co-learners (Jacobs & Farrell, 2001)

9

Paradigm Shift Observed by 2006 Zuengler & Miller, 2006

“Ontological” – “basic questions about the nature of reality (Denzin & Lincoln, 1998, p.185)

1. Cognitive vs. Sociocultural Understanding of Learning

2. Positivists vs. Relativists over how to construct SLA theory

10

Traditional Scientific Understanding

• establishment of laws or patterns

• that exist across contexts,

• as a deductive system of reasoning

• that is rule-based

• and thus independent of the forces(Linda Watkins-Goffman, 2006, p.2)

11

Contrasts between positivism and post-positivism (Jacobs & Farrell, 2001)

Positivism Post-Positivism

Emphasis on parts and decontextualization Emphasis on whole and contextualization

Emphasis on separation Emphasis on integration

Emphasis on the general Emphasis on the specific

Consideration only of objective and the quantifiable

Consideration also of subjective and the non-quantifiable

Reliance on experts and outsider knowledge--researcher as external

Consideration also of the "average" participant and insider knowledge--researcher as internal

Focus on control Focus on understanding

Top-down Bottom-up

Attempt to standardize Appreciation of diversity

Focus on the product Focus on the process as well

12

Sociocultural Perspectives on LL

• View language use in real-world situations as fundamental, not ancillary, to learning.

• Language not as input, but as a resource for participation in daily activities

• Participation as both the product and process of learning (Zuengler & Miller, 2006)

13

The Changing Focus of ResearchIt is about:• the attempt to adapt the self into a new context and

a new world• the struggle for participation in a new social

environment• participation as a metaphor for learning a new

language• participation as described by the individual’s

narrative can be interpreted as a metaphor for acquiring a new identity

NOT just• in the usual sense where the lexical, grammatical, a

nd semantic systems are learned or acquired (Watkins-Goffman, 2006,p. 1)

14

Comparison

15

4 QIApproaches

16

12 QI Characteristics

17

2. Paradigm Shift on the Internet

18

19

20

參與Participation

使用者就是貢獻者User as Contributor

使用者自主Trust &

Decentralization

遊戲式的經驗Play &

Rich Experience

態度，而非科技An Attitude

not a technology

Web 2.0

21

Networks of Personalized Learning

• Blogs• Podcasts• MySpace• Flicker• RSS• Delicious

22

Tools for Collaborative Language Learning

• Skype• Google Talk• Chinese pod• Live Mocha• Voice Thread• Yack Pack• Dotsub• Chinswing, etc.

23

貢獻分

我的學習點數

跟我一樣在學這個語言的人

24

有人看過我 po出去的功課

25

26

27

28

29

Community

togetherness

Dialogue Discourse

Identity

Culture

30

Toward an Understanding of Computer-mediated EFL Writing Experience Through Vygotiskian Perspectives

Chi-chi ChaoNational Chengchi University

Chao, C. (2007). Toward an understanding of computer-mediated EFL writing experience through Vygotskian perspectives: A monograph published by Taiwan Journal of TESOL. Taipei, Taiwan: Crane.

31

Purposes

1. to explore how Vygotsky’s socio-cultural-historical theory (SCT) could serve as a useful framework to the study of EFL learning experiences

2. to understand how EFL learning is supported by automated writing evaluation (AWE) programs: first without an inquiry community and later with one

32

The Philosophical Underpinnings and QI Tradition

The philosophical underpinning of the research reported here is interpretivist in nature. The aim is not to offer causal explanations, but to understand the experience by way of “reconstructing the self-understandings of actors engaged in the action” (Schwandt, 2000, p. 191).

In other words, the focus is on grasping the meanings that constitute the action of learning to write in English in the AWE mediated learning environment, seeking to discover some of the essence of that experience through intensive study of individual cases.

Taking a theory-driven approach

33

Social Cultural Historical Theory (SCT)

Lev Semenovich Vygotsky (1896-1934)

a Russian Jewish developmental psychologist and the founder of cultural-historical psychology

his major works span 6 volumes, written over roughly 10 years Mind in society: The development of higher psychological processes (1978)

34

Definition of SCT

[D]espite the label “sociocultural” the theory is not a theory of the social or of the cultural aspects of the human existence. …it is,… rather, a theory of mind… that recognizes the central role that social relationships and culturally constructed artifacts play in organizing uniquely human forms of thinking (p. 1). (Lantolf, 2004, cited in Lantolf & Thorne, 2006)

35

The Social Formation of Mind

“We want to be clear on this point. The argument is not that social activity influences cognition, but that social activity is the process through which human cognition is formed” (Lantolf & Johnson, 2007, p. 878).

36

An Important Quote

Any function in the child’s cultural development appears twice, or on two planes. First it appears on the social plane, and then on the psychological plane. First it appears between people as an interpsychological category, and then within the child as an intrapsychological category. This is equally true with regard to voluntary attention, logical memory, the formation of concepts, and the development of volition — [I]t goes without saying that internalization transforms the process itself and changes its structure and functions. Social relations or relations among people genetically underlie all higher functions and their relationships (Vygotksy, 1978, p. 57).

37

Key Concepts

Mediation Internalization Zone of proximal

development (ZPD)

38

Q1. Describe as thoroughly as possible the image you have for the word “apprentice.”

Describe her work environment, tools she uses, and the people she might work with. Compare and contrast this image with that of a pupil in the classroom and then that of an EFL learner on the Internet – What situation would an EFL learner be an apprentice?

Piaget: the child as lone scientist vs. Vygotsky: the child as apprentice

39

窯廠學師仔囝仔工又稱「學師仔」 ( 學徒 ) ，一般師仔工 ( 學徒工 ) 大都負責工廠較不重要工作，或是跟在作瓷仔 ( 陶瓷 ) 師傅身邊當助手，工作的同時邊學習，一些較靈巧的囝仔工二、三年出師 ( 學成 ) 了，擔任師傅。而有些師傅則向老闆請負 ( 包工、分租 ) ，等較有資本後自行設立小窯廠，全家人投入陶瓷製作生產，他的小孩又成了囝仔工，生意較好時另又招了一些囝仔工。

台灣陶瓷數位典藏計畫http://digital.ceramics.tpc.gov.tw/Web/yingo200/story/work/work.htm

40高雄市立美術館圖片檔 ( 古勒勒‧達比烏蘭 )http://www.kmfa.gov.tw/Data/Image/ 古勒勒與他的學徒 Engzngz.JPG

Development occurs when we incorporate tools for thinking from our society.

41

Development occurs when we incorporate tools for thinking from our society.

Q2. Describe an experience (i.e., yours or your students’) that can verify this statement. Then, describe an experience in learning a foreign language which can also respond to this statement.

42

• Vygotsky’s emphases: First. the symbol systems come to us from others rather than within ourselves. Second, the symbol systems are not just used in our thinking but completely reorganize our thinking.

• …It [language] is no longer speech for communication; it is now a personal psychological tool that changes all her thought processes.

43

• In traditional learning theory (note: i.e., behaviorism) and social learning theory, society is thought to influence and shape the child, but in Vygotsky’s theory, the child is a part of society and a collaborator in his learning with adult mentors.

• For Vygotsky, we can’t function on an adult level without the culture of which we are a part bringing us along and providing what is necessary. This conceptualization acknowledges a deeper level of social interaction than the simple social influence and conditioning envisioned by learning theory (note: i.e., behaviorists).

44

Influences (just a few)

Activity theory (Leontiv, Engestrom) Anchored instruction (Bransford et al) Cognitive apprenticeship (Collins, Brown & Newman, 19

87) Distributed cognition (Hutchins, 1995) Dialogic inquiry (Wells, 1999) Dynamic assessment (Holt & Willard-Holt, 2000; Lantolf) Situated Cognition (Brown, Collins, & Duguid, 1989; Lav

e & Wenger, 1991 ) Social constructivism (bringing together the work of Pia

get with that of Bruner and Vygotsky)

45

Some of the authors

Jerome Bruner, Andy Clark Michael Cole, James Lantolf James V. Wertsc

h, Gordon Wells,

46

Features of SCT-inspired research• Emphasizing the context: taking a holistic view on th

e “ecology” of language learning environment and attempting to capture “multiple dimensions of context and multiple levels of discourse.”

• The complexity of the language learning environment is elucidated by adopting theoretical frameworks from disciplines outside of TESOL.

• Issues such as power, identity, culture and gender are investigated. Agency is another such issue interested to researchers. (Tsui, 2008, p. 41)

• Attempting to understand how technology transforms human action (e.g., learners’ and teachers’) and what activities people are engaged in with the assistance of technology in the particular social cultural context.

47

Many studies have adopted SCT-related theoretical framework in designing and examining

activity structures for CALL

• Self-access language learning (Hoven, 1999), • CMC (computer-mediated communication) (Mes

kill, 2005; Shin, 2006), • Telecommunication projects (Belz, 2002; Lee, 20

04), • Project-oriented CALL (Jeon-Ellis, Debski, Wiggl

esworth, 2005)

48

The Mediation/Tool: AWE AWE (Automated Writing

Evaluation) programs is a kind of computer software designed to evaluate compositions, supposedly as efficiently as human raters.

49

Previous research has mostly centered on psychometric aspects of the software, comparing the quality of machine generated feedback and evaluation with that of human raters (Warschauer & Ware, 2006).

50

Nowadays AWEs have been advocated as “web-based writing instructional tool” (Vantage), expanding from the original purpose of evaluation to that of instruction.

There is thus a need to understand how it supports learning in the sociocultural context of language learning environments.

51ETS Criterion http://criterion1.ets.org/cwe/

Demo: http://www.ets.org/Media/Products/Criterion/tour2/critloader.html

52

53Vantage MY Access http://www.vantagelearning.com/myaccess/

54

55

Scoring scales used by My Access & Criterion (1)

My Access

Focus & Meaning The extent to which the response demonstrates understanding of the text and the purpose of the task, and makes connections between them through a controlling or central idea.

Content & Development The extent to which ideas are elaborated with specific, accurate, and relevant details (facts, examples, reasons, anecdotes, prior knowledge).

Organization The extent to which the response establishes purposeful structure, direction, and unity, including transitional elements.

Criterion

Organization & Development

Introductory Material Thesis Statement Main Ideas Supporting Ideas Conclusion Transitional Words and

Phrases

56

Scoring scales used by My Access & Criterion (2)

My AccessLanguage & Style The extent to

which the response demonstrates effective and varied sentences and word choice appropriate to the intended audience

CriterionStyle Repetition of Words Inappropriate Words or

Phrases Sentences Beginning with

Coordinating Conjunctions Too Many Short Sentences Passive Voice Number of words: (number) Number of sentences:

(number) Average number of words

per sentence: (number)

57

Scoring scales used by My Access & Criterion (3)

My Access

Mechanics & Conventions The extent to which

the response demonstrates control of conventions, including paragraphing, grammar, usage, punctuation, and spelling.

CriterionGrammar (problems to be detected) Fragments Run-on sentences Garbled Sentences Subject-Verb Agreement

Errors Verb-Form Errors Ill-formed verbs Pronoun Errors Possessive Errors Wrong or Missing Words Proof read this!

58

Mechanics Spelling Capitalize Proper Nouns Missing Initial Capitalized

Letter in a Sentence Missing Question Mark Missing Final Punctuation Missing Apostrophe Missing Comma Hyper Error Fused Words Compound Words Duplicates

59

Usage (problems to be

detected) Wrong Article Missing or Extra Article Confused Words Wrong Form of Word Faulty Comparisons Preposition Error Nonstandard Verb or

Word Form

60

Feedback Wording My Access

Overall 5: On a scale of one to six, your response to this assignment was rated a 5. Your response was evaluated on the basis of how well it communicates its message considering important areas of writing including focus and meaning, content and development, organization, language use and style, and conventions and mechanics.

Criterion Score of 5: Skillful

Performance Tells a clear story that is well-

developed and supported with pertinent details in much of the response.

Well organized with story elements that are connected across most of the response; may have occasional lapses in transitions.

Exhibits some variety in sentence structure and uses good word choice; occasionally, words may be used inaccurately.

Errors in grammar, spelling, and punctuation do not interfere with understanding.

61

STUDY 1: AWE AS THE SOLE MEDIATION

62

Research Questions How helpful is the opportunity to use the

AWE program in a self-access mode? How helpful is the AWE feedback system

in developing writing skills in a self-access learning mode?

What major instructional support may be necessary to help learners develop writing proficiency with the AWE program?

63

The Context

In the spring semester of 2005, for promotional purpose, the Taiwanese representative of Criterion offered the author’s affiliated department 265 free accounts which allowed students unlimited use of the AWE for a period of twelve months.

64

Presented as paradoxes, the assertions below are derived from the learners’ self-report in interviews and comments in face-to-face meetings.

65

Q1: How is the opportunity to use the AWE program helpful?

The learner liked the opportunity to practice writing at their own pace, but very few of them actually took advantage of it.

66

Table 4.1. Frequency of useTopics Due Date Number of People

who wrote this topic

Number of submissions

Required Goals Feb 7 164 (64.82%) 375 470 required essays (89.69%)

Technology Feb 13 84 (33.20%) 167

Special Object Feb 20 89 (35.18%) 206

Teaching Styles Mar 7 63 (24.90%) 118

Guest Speaker Mar 21 44 (17.39%) 98

Make a Change Apr 4 26 (10.28%) 53

Self-selected topics-- 54 essays (10.30%) 31 (12.25%)

Total: Number of people in the program: 253People who wrote at least one topic: 179 (70.75%). People who finished all six required topics: 13 (5.13%)

524 essays in total. 1147 submissionsRevision: 2.19 times per topic

67

0%

10%

20%

30%

40%

50%

60%

70%

7-Feb 13-Feb 20-Feb 7-Mar 21-Mar 6-Apr

Perc

enta

ge

68

Q2. How is the AWE feedback system helpful in developing writing skills?

• The feedback system is adequate in areas which a definite comment is possible and which are included in the criteria, but it is inadequate in other areas which are more complex or illusive.

• Those who made the most of the program chose to trust the feedback and used metacognitive skills extensively, although they were fully aware of the program’s insufficiencies.

69

Q3 What kind of instructional solutions may be necessary to help learners develop writing proficiency with the AWE program?

• Learners liked to develop their writing in private, but they also wanted a sense of group.

70

DESIGNING A COURSE WITH SCT

71

Three Teaching Approaches to EFL Writing

1. The text-oriented approach to writing “focuses on the products of writing by examining texts in various ways, either through their formal surface elements or their discourse structures”

2. The writer-oriented approach focuses on the writer and describes writing in terms of the processes used to create texts, including the views that writing as personal expression, writing as a cognitive process, and writing as a situated act.

3. The reader-oriented approach emphasizes the role that readers play in writing, including such views as writing as social interaction, writing as social construction, and writing as power and ideology.

AWE

72

The SCT Touch in the Course

• lies in its emphasis of writers as unique contributors and creators of world knowledge, who can best be supported by a learning community that encourages constant dialogic interaction through which participants and the instructor engage collaboratively in the inquiry into EFL writing and reflectively in the actual practice of writing.

73

The design of the course follows Gabrielatos’s writing skills development cycle, including --

awareness-raising, support, practice, and feedback as the key activities

74

STUDY 2: MEDIATED LEARNING WITH AND AROUND AWE

75

The Context A part-time English writing course which featured using My Acces

s as a tool for self-regulated learning. Nine people enrolled in the writing course They were teachers, government officials, students, college profe

ssors, or bank clerks who were interested in advancing their English writing skills.

All of them had college degrees, with five (5) of them having an advanced graduate degree, including one PhD in architecture and one MFA in fine art.

Their writing proficiency levels, based on their self-report on the first day of the class, ranged from low intermediate to high intermediate.

As this is a smaller class than the previous year and students have higher education levels, it was expected that these students would be able to use more metacogintive strategies and that it would be more likely for the learners to have intense interaction as a class.

76

Research Questions How do two learners regulate their learning through the mediation system formed by the AWE program, the teacher, and fellow students?

1. How do they regulate their learning with the mediation of the AWE program?

2. How do they regulate their learning with the help of the teacher and peers in the class?

3. What meaningfulness do they derive from their experiences in this writing course?

77

Key Informants: Anne & Grace Two reasons to choose these two learners as the

cases focused in this study -- (1) they were among the most motivated in the

class. They were comfortable about articulating their thoughts in class and in face-to-face conferences, which allowed me many opportunities throughout the course to understand how they regulated their learning in this context.

(2) they happily accepted my invitation the moment I asked. The rapport existing between each of them and me was expected to lubricate the data collection process. Below is the two learners’ background information.

78

Anne

In her mid fifties. Married with two children.

Job: In the nursing profession. She had two years’ experience studying for a master’s degree in the U.S. about fifteen years ago.

Goal: Passing GEPT Focus: Text-based writing techniques

79

Grace In her early forties, married with a teen

daughter. Job: a computer programmer in a well-

established bank for fifteen years. Goal: Purely for interest Focus: Developing content. Comparatively speaking, she writes better

than Anne and many other members in this class.

80

Data Sources Interviews, class field notes, learner’s

written works, and records of conference with the teacher.

Data taken directly from the AWE system: number of prompts used, number of scored essays, number of pending essays, scores for the first try and the best try, average scores, and the date when the participant input the system for the last time.

81

Data Analysis

Construct profiles for the two learners using and triangulating various data sets

Categorization based on critical incidents

Interpretation Member checking

82

Critical IncidentsQ1 How do they regulate their learning with the

mediation of the AWE program? Using AWE functions and feedback Analyzing techniques in model essays

Q2 How do they regulate their learning with the help of the teacher and peers in the class?

Adopting external resources to enrich writing

Q3 What meaningfulness do they derive from their experiences in this writing course?

Deriving insights from interacting with the instructor: class activities and conferencing

83

What is learned?

84

Activity Theory

85

The Subject, Object, & Outcome

The learner’s goal for learning how to write played an important role in determining how they interacted with the tool.

86

The Mediation

Second, AWE is not a neutral tool: It serves a specific type of learner, who is perhaps more like Anne than Grace in terms of goals and proficiency levels.

87

The Community

Not everybody learned the same thing with the same

software.

88

Four levels of contradictions in a network of human activity systems

89

Theoretical Implication

1. The concept of mediation in light of the two studies

• The AWE tool

90

Theoretical Implication

2. AWE as mediation in the intrapsychological categories

91

Pedagogical Implications Design of AWE: Supporting thoughtful engagement,

in the process of ‘learning by doing’ and ‘learning by reflection’

John Seely Brown (2000) argues that the point is designing tools to support functions that are most comfortable and natural to human learning in everyday environment. He says, “Our challenge and opportunity, then, is to foster an entrepreneurial spirit toward creating new learning environments-- a spirit that will use the unique capabilities of the Web to leverage the natural ways that humans learn” (p. 13).

92

Pedagogical Implications Design for social interaction It would be necessary to explain to

learners the importance of interaction so that they do not think of reflective conversations as a waste of time.

More research is necessary to investigate learner difference in response to the emphasis of interaction and reflection in language learning environments.

93

The Contributions of QI

94

QI holds potential to allow documentation of the challenges encounte

red in implementing interventions designed to change or reform existing practice

pay attention to cultural and contextual factors acquire the benefit of formative research, attendi

ng to the specific needs and resources of the target population

( Nastasi & Schensul, 2005, pp.186-187)

The schism between research and practice is a major challenge facing applied researchers.

95

QI [is] critical for documenting the adaptations necessary f

or application of interventions to real-life contexts, and for identifying core intervention components which are related to desired outcomes.

can help researchers to describe various manifestations of intended outcomes that may not be reflected in standardized instruments, and to identify unintended positive or negative outcomes for the individual and institution/community.

( Nastasi & Schensul, 2005, pp.186-187).

96

The researcher as a learner

A key contribution of qualitative research is the development of theories and concepts that can aid an understanding of education…

Within a qualitative worldview all knowledge is partial, situated, and contextual.

Our own learning as researchers and practitioners never ends; the more we know the more there is to know (Giangreco & Taylor, 2003, 135-136)

97