Testing Testing Grammar Grammar

Jonathan MagdalenaJonathan Magdalena

What What are we trying to are we trying to MEASUREMEASURE??

““English English grammargrammar is chiefly a system of is chiefly a system of syntaxsyntax, that decides , that decides

the the orderorder and and patternspatterns in which in which

words are arrangedwords are arranged into sentences.into sentences.”” ((Close, 1982)Close, 1982)

WhyWhy should we test should we test grammargrammar??

LinguisticLinguistic or communicative or communicative competence?competence?

Structures Structures or functions?or functions?

UsageUsage or use? or use?

PrescriptivePrescriptive or descriptive? or descriptive?

HowHow should we test should we test grammargrammar??

Separated Separated from skills?from skills?

Discrete Discrete point point or integrative items?or integrative items?

Focus on Focus on grammargrammar or meaning? or meaning?

What are the most common What are the most common test formats test formats for for testing grammar?testing grammar?

RECOGNITIONRECOGNITION Multiple choice items (Liu, 121)Multiple choice items (Liu, 121) Error-recognition items (Liu, 129)Error-recognition items (Liu, 129) True/False (Baxter, 35) True/False (Baxter, 35) Pairing and matching itemsPairing and matching items Cloze/Modified Cloze (Hughes, 144)Cloze/Modified Cloze (Hughes, 144)

What are the most common What are the most common test formats test formats for for testing grammar?testing grammar?

ProductionProduction Completion items (Hughes, 143) Completion items (Hughes, 143) Transformation items (Liu, 131)Transformation items (Liu, 131) Paraphrase (Hughes, 143)Paraphrase (Hughes, 143) RearrangementRearrangement EditingEditing Combination and addition itemsCombination and addition items Items involving the changing of wordsItems involving the changing of words

WhenWhen should we go for should we go for grammargrammar recognitionrecognition??

When…When… More material needs to be covered. More material needs to be covered. You want to test different levels of You want to test different levels of

learning. learning. You have little time for scoring. You have little time for scoring. You are not interested in evaluating how You are not interested in evaluating how

well a test taker can formulate a correct well a test taker can formulate a correct answer. answer.

You have a large number of test takers. You have a large number of test takers.

WhenWhen should we go for should we go for grammargrammar productionproduction??

When…When… You want to evaluate a person’s ability You want to evaluate a person’s ability

to formulate a correct answer. to formulate a correct answer. You have more time to score the itemsYou have more time to score the items You want to test a persons ability to You want to test a persons ability to

apply concepts and information to a new apply concepts and information to a new situation. situation.

You have a clear idea of the aspects and You have a clear idea of the aspects and concepts that should be tested. concepts that should be tested.

True/FalseTrue/False

Is this sentences true or false?Is this sentences true or false?““He works from Monday to Friday”He works from Monday to Friday”

Does not demonstrate broader knowledge Does not demonstrate broader knowledge Difficult to construct in higher levels  Difficult to construct in higher levels  Encourages guessing due to 50/50 chance Encourages guessing due to 50/50 chance Difficult to test attitudes toward learning Difficult to test attitudes toward learning

Multiple ChoiceMultiple Choice

John usually ______from Monday to FridayJohn usually ______from Monday to Fridaya. works b. worked c. has worked d. is working a. works b. worked c. has worked d. is working

They can be tricky or too picky   They can be tricky or too picky   Difficult to test attitudes towards learning Difficult to test attitudes towards learning Knowledge is limited to options provided Knowledge is limited to options provided Difficult to construct at higher levels Difficult to construct at higher levels Encourages guessing (25% chance) Encourages guessing (25% chance) More than one option may be possibleMore than one option may be possible All options must be grammatically possibleAll options must be grammatically possible

Gap-FillingGap-FillingJohn usually ______(work) from Monday to FridayJohn usually ______(work) from Monday to Friday

They must be part of a broader context   They must be part of a broader context   More than one option may be possibleMore than one option may be possible First gap may be offered as a sampleFirst gap may be offered as a sample Difficult to test attitudes towards learning Difficult to test attitudes towards learning Knowledge is limited to options provided Knowledge is limited to options provided Limitations result in negative backwashLimitations result in negative backwash Focus should be on the aspect assessed.Focus should be on the aspect assessed.

Cloze/C-testCloze/C-testJohn ______ works ______Monday ______FridayJohn ______ works ______Monday ______Friday

Tests student’s understanding of the whole Tests student’s understanding of the whole language   language  

More than one option may be possibleMore than one option may be possible First letter may be offered as a sampleFirst letter may be offered as a sample Difficult to test attitudes towards learning Difficult to test attitudes towards learning Some words may be impossible to guess Some words may be impossible to guess Can test student’s IQ rather than language Can test student’s IQ rather than language

proficiencyproficiency Difficult for some students (bias)Difficult for some students (bias)

Sentence BuildingSentence BuildingFriday/usually/to/John/Monday/workFriday/usually/to/John/Monday/work

Tests recognition and/or production  Tests recognition and/or production  More than one option may be possibleMore than one option may be possible Capital letters and punctuation may interfere Capital letters and punctuation may interfere

in the production of the answerin the production of the answer Difficult to test attitudes towards learningDifficult to test attitudes towards learning Some context may be added Some context may be added Can test student’s IQ rather than language Can test student’s IQ rather than language

proficiencyproficiency Difficult for some students (bias)Difficult for some students (bias)

Sentence TransformationSentence TransformationJohn is a doctor. He ______________ (work)John is a doctor. He ______________ (work)

More than one option may be possibleMore than one option may be possible Clues will limit the answersClues will limit the answers Difficult to test attitudes towards learningDifficult to test attitudes towards learning Some context may be added Some context may be added Instructions are essentialInstructions are essential Few aspects might be assessed (passive, Few aspects might be assessed (passive,

reported, comparatives, conditionals…)reported, comparatives, conditionals…)

Correction/EditingCorrection/EditingJohn usually working from Monday to FridayJohn usually working from Monday to Friday

Tests recognition of mistakesTests recognition of mistakes Can also test production through Can also test production through

correctionscorrections More than one correction may be possibleMore than one correction may be possible Mistakes must be grammatically possible Mistakes must be grammatically possible Good backwash Good backwash Context is essentialContext is essential

Five characteristics to measure Five characteristics to measure communicativecommunicative grammargrammar

1.1. The test must provide more context than The test must provide more context than only a single sentence.only a single sentence.

2.2. The test taker should understand what The test taker should understand what the communicative purpose of the task is.the communicative purpose of the task is.

3.3. He or she should also know who the He or she should also know who the intended audience is.intended audience is.

4.4. He or she must have to focus on meaning He or she must have to focus on meaning and not form to answer correctly.and not form to answer correctly.

5.5. Recognize is not sufficient. The test taker Recognize is not sufficient. The test taker must be able must be able ““to produce grammatical to produce grammatical responses.responses.””

(R. Dickins)

Therefore…Therefore… Have the test taker say or write Have the test taker say or write

something of discourse length in something of discourse length in order to perform some order to perform some communicative task for a known communicative task for a known audience.audience.

What is said or written must make What is said or written must make sense.sense.

Split SentencesSplit Sentences Write out some sentences, and then Write out some sentences, and then

cut each sentence in half. Place the cut each sentence in half. Place the two halves in two separate columns two halves in two separate columns and ask students to find the matching and ask students to find the matching half. half.

StudentsStudents’’ abilities in combing abilities in combing sentencessentences

Grammar knowledge to all forms of the Grammar knowledge to all forms of the sentencessentences

ExamplesExamples If you eat thatIf you eat that If you touch the dogIf you touch the dog If you steal my If you steal my

boyfriendboyfriend If you go out nowIf you go out now If you donIf you don’’t leavet leave If you donIf you don’’t book a t book a

ticketticket

youyou’’ll be sickll be sick itit’’ll bite youll bite you II’’ll never speak ll never speak

to you againto you again youyou’’ll get ll get

soakedsoaked II’’ll call the policell call the police youyou’’ll be lucky ll be lucky

to get a seatto get a seat

Sentences from picturesSentences from pictures Hand out a sheet of pictures. Ask Hand out a sheet of pictures. Ask

students to come out sentences from students to come out sentences from the pictures or ask them to tell a the pictures or ask them to tell a story.story.

Enough grammar knowledge to make Enough grammar knowledge to make sentences by students. sentences by students.

AccuracyAccuracy

Memory TestMemory Test Give limited time for students to see Give limited time for students to see

the picture. Ask them to describe the the picture. Ask them to describe the picture without seeing it. picture without seeing it.

Testing studentsTesting students’’ ability of making ability of making sentences.(enough grammar sentences.(enough grammar knowledge)knowledge)

AccuracyAccuracy The transformation of studentsThe transformation of students’’

sentencessentences

Picture DictationPicture Dictation A student try to describe the picture A student try to describe the picture

to other students who havento other students who haven’’t seen t seen it.it.

Testing if a student can make Testing if a student can make sentences which can help him/her sentences which can help him/her communicate well.communicate well.

AccuracyAccuracy

Strip StoryStrip Story Give students a text from a strip Give students a text from a strip

story. Ask them to find the order and story. Ask them to find the order and resolve the problem from the story.resolve the problem from the story.

See if students can find the time See if students can find the time order from the tenses of the order from the tenses of the sentences.sentences.

AccuracyAccuracy

Miming an actionMiming an action Ask students to role play in a limited Ask students to role play in a limited

situation.situation. Through acting, it can test studentsThrough acting, it can test students ’’

ability of thinking of lines. (which ability of thinking of lines. (which may contains different tenses.)may contains different tenses.)

AccuracyAccuracy

Growing StoriesGrowing Stories Story-building activitiesStory-building activities StudentsStudents’’ ability of making sentences ability of making sentences

with different tenseswith different tenses

QuestionnairesQuestionnaires Turn what have taught in class into a Turn what have taught in class into a

questionnaire. Get students to questionnaire. Get students to survey each other. (does not contain survey each other. (does not contain fully-written-out questions.)fully-written-out questions.)

It can see if studentsIt can see if students’’ totally totally understand what have learned in understand what have learned in class. Also it can see if studentsclass. Also it can see if students ’’ can can use it well.use it well.