Upload
efl-learning
View
549
Download
2
Embed Size (px)
Citation preview
Introduction to Introduction to AssessmentAssessment
EFL LearnersEFL Learners
What is assessment?What is assessment?
Not the same as testing!Not the same as testing! An ongoing process to ensure that the An ongoing process to ensure that the
course/class objectives and goals are met. course/class objectives and goals are met. A process, not a product.A process, not a product. A test is a form of assessment. (Brown, 2004, A test is a form of assessment. (Brown, 2004,
p. 5)p. 5)
Informal and Formal AssessmentInformal and Formal Assessment
Informal assessment can take a number of Informal assessment can take a number of forms:forms: unplanned comments, verbal feedback to students, unplanned comments, verbal feedback to students,
observing students perform a task or work in small observing students perform a task or work in small groups, and so on.groups, and so on.
Formal assessment are exercises or procedures Formal assessment are exercises or procedures which are:which are: systematicsystematic give students and teachers an give students and teachers an appraisalappraisal of students’ of students’
achievement such as tests.achievement such as tests.
Traditional AssessmentTraditional Assessment
Multiple-choiceMultiple-choice True-falseTrue-false MatchingMatching Norm-referenced and criterion referenced testsNorm-referenced and criterion referenced tests
Norm and Criterion-referenced testsNorm and Criterion-referenced tests
Norm-referenced testNorm-referenced test standardized tests (college board, TOEFL, GRE)standardized tests (college board, TOEFL, GRE) Place test-takers on a mathematical continuum in Place test-takers on a mathematical continuum in
rank orderrank order Criterion-referenced testsCriterion-referenced tests
give test-takers feedback on specific objectives give test-takers feedback on specific objectives (“criterea”)(“criterea”)
test objectives of a course test objectives of a course known as “instructional value”known as “instructional value”
Authentic AssessmentAuthentic Assessment
Authentic assessment Authentic assessment reflects student learning, achievement, motivation, reflects student learning, achievement, motivation,
and attitudes on instructionally relevant classroom and attitudes on instructionally relevant classroom activities (O’Malley & Valdez, 1996).activities (O’Malley & Valdez, 1996).
Examples: Examples: performance assessmentperformance assessment portfoliosportfolios self-assessmentself-assessment
Purposes for AssessmentPurposes for Assessment
Diagnose students strengths and needsDiagnose students strengths and needs Provide feedback on student learningProvide feedback on student learning Provide a basis for instructional placementProvide a basis for instructional placement Inform and guide instructionInform and guide instruction Communicate learning expectationsCommunicate learning expectations Motivate and focus students’ attention and Motivate and focus students’ attention and
efforteffort Provide practice applying knowledge and skillsProvide practice applying knowledge and skills
Purposes continuedPurposes continued
Provide a basis for evaluation for the purpose Provide a basis for evaluation for the purpose of:of: GradingGrading Promotion/graduationPromotion/graduation Program admission/selectionProgram admission/selection AccountabilityAccountability Gauge program effectivenessGauge program effectiveness
Assessment InstrumentsAssessment InstrumentsPre-assessment Pre-assessment (diagnostic)(diagnostic)
Formative (ongoing)Formative (ongoing) Summative (final)Summative (final)
PretestsPretests QuizzesQuizzes Teacher-made testTeacher-made test
ObservationsObservations DiscussionsDiscussions PortfoliosPortfolios
Journals/logsJournals/logs AssignmentsAssignments ProjectsProjects
DiscussionsDiscussions ProjectsProjects Standardized testsStandardized tests
QuestionnairesQuestionnaires ObservationsObservations
InterviewsInterviews PortfoliosPortfolios
Journal logsJournal logsStandardized testsStandardized tests
DiscussionDiscussion
How would you document a student How would you document a student performance during a discussion?performance during a discussion?
Which types of assessments noted in the chart Which types of assessments noted in the chart could be considered authentic assessment?could be considered authentic assessment?
Principles of Language AssessmentPrinciples of Language Assessment
PracticalityPracticality ReliabilityReliability ValidityValidity AuthenticityAuthenticity WashbackWashback
PracticalityPracticality
An effective test is practicalAn effective test is practical Is not excessively expensiveIs not excessively expensive Stays within appropriate time constraintsStays within appropriate time constraints Is relatively easy to administerIs relatively easy to administer Has a scoring/evaluation procedure that is specific Has a scoring/evaluation procedure that is specific
and time-efficientand time-efficient
ReliabilityReliability
A reliable test is consistent and dependable. If A reliable test is consistent and dependable. If you give the same test to the same students in you give the same test to the same students in two different occasions, the test should yield two different occasions, the test should yield similar results.similar results. Student-related reliabilityStudent-related reliability Rater reliabilityRater reliability Test administration reliabilityTest administration reliability Test reliabilityTest reliability
Student Related ReliabilityStudent Related Reliability
The most common issue in student related The most common issue in student related reliability is caused by temporary illness, fatigue, reliability is caused by temporary illness, fatigue, a bad day, anxiety, and other physical and a bad day, anxiety, and other physical and psychological factors which may make an psychological factors which may make an “observed” score deviate from a “true” score. “observed” score deviate from a “true” score.
Rater ReliabilityRater Reliability
Human error, subjectivity, and bias may enter Human error, subjectivity, and bias may enter into the scoring process.into the scoring process.
Inter-rater reliability occurs when two or more Inter-rater reliability occurs when two or more scorers yield inconsistent scores of the same test, scorers yield inconsistent scores of the same test, possibly for lack of attention to scoring criteria, possibly for lack of attention to scoring criteria, inexperience, inattention, or even preconceived inexperience, inattention, or even preconceived bias toward a particular “good” and “bad” bias toward a particular “good” and “bad” student. student.
Test Administration ReliabilityTest Administration Reliability
Test administration reliability deals with the Test administration reliability deals with the conditions in which the test is administered.conditions in which the test is administered. Street noise outside the building Street noise outside the building bad equipment bad equipment room temperature room temperature the conditions of chairs and tables, photocopying the conditions of chairs and tables, photocopying
variationvariation
Test Reliability Test Reliability
The test is too longThe test is too long Poorly written or ambiguous test itemsPoorly written or ambiguous test items
ValidityValidity
A test is valid if it actually assess the objectives A test is valid if it actually assess the objectives and what has been taught.and what has been taught. Content validityContent validity Criterion validity (tests objectives)Criterion validity (tests objectives) Construct validityConstruct validity Consequential validity Consequential validity Face validityFace validity
Content ValidityContent Validity
A test is valid if the teacher can clearly define the A test is valid if the teacher can clearly define the achievement that he or she is measuringachievement that he or she is measuring
A test of tennis competency that asks someone A test of tennis competency that asks someone to run a 100-yard dash lacks content validityto run a 100-yard dash lacks content validity
If a teacher uses the communicative approach to If a teacher uses the communicative approach to teach speaking and then uses the audiolingual teach speaking and then uses the audiolingual method to design test items, it is going to lack method to design test items, it is going to lack content validitycontent validity
Criterion-related ValidityCriterion-related Validity
The extent to which the objectives of the test have been The extent to which the objectives of the test have been measured or assessed. For instance, if you are assessing measured or assessed. For instance, if you are assessing reading skills such as scanning and skimming reading skills such as scanning and skimming information, how are the exercises designed to test information, how are the exercises designed to test these objectives? these objectives?
In other words, the test is valid if the objectives taught In other words, the test is valid if the objectives taught are the objectives tested and the items are actually are the objectives tested and the items are actually testing this objectives.testing this objectives.
Construct Validity Construct Validity
A construct is an explanation or theory that A construct is an explanation or theory that attempts to explain observed phenomenaattempts to explain observed phenomena
If you are testing vocabulary and the lexical If you are testing vocabulary and the lexical objective is to use the lexical items for objective is to use the lexical items for communication, writing the definitions of the communication, writing the definitions of the test will not match with the construct of test will not match with the construct of communicative language usecommunicative language use
Consequential Validity Consequential Validity
Accuracy in measuring intended criteriaAccuracy in measuring intended criteria Its impact on the preparation of test-takersIts impact on the preparation of test-takers Its effect on the learnerIts effect on the learner Social consequences of a test interpretation (exit Social consequences of a test interpretation (exit
exam for pre-basic students at El Colegio, the exam for pre-basic students at El Colegio, the College Board)College Board)
Face ValidityFace Validity Face validity refers to the degree to which a test looks Face validity refers to the degree to which a test looks
right, and appears to measure the knowledge or ability right, and appears to measure the knowledge or ability it claims to measureit claims to measure A well-constructed, expected format with familiar tasksA well-constructed, expected format with familiar tasks A test that is clearly doable within the allotted time limitA test that is clearly doable within the allotted time limit Directions are crystal clearDirections are crystal clear Tasks that relate to the course (content validity)Tasks that relate to the course (content validity) A difficulty level that presents a reasonable challengeA difficulty level that presents a reasonable challenge
AuthenticityAuthenticity
The language in the test is as natural as possibleThe language in the test is as natural as possible Items are contextualized rather than isolatedItems are contextualized rather than isolated Topics are relevant and meaningful for learnersTopics are relevant and meaningful for learners Some thematic organization to items is providedSome thematic organization to items is provided Tasks represent, or closely approximate, real-Tasks represent, or closely approximate, real-
world tasksworld tasks
WashbackWashback
Washback refers to the effects the tests have on Washback refers to the effects the tests have on instruction in terms of how students prepare for instruction in terms of how students prepare for the test “Cram” courses and “teaching to the the test “Cram” courses and “teaching to the test” are examples of such washbacktest” are examples of such washback
In some cases the student may learn when In some cases the student may learn when working on a test or assessmentworking on a test or assessment
Washback can be positive or negativeWashback can be positive or negative
Alternative Assessment OptionsAlternative Assessment Options
Self and peer-assessmentsSelf and peer-assessments Oral production-student self-checklist, peer checklist, Oral production-student self-checklist, peer checklist,
offering and receiving holistic rating of an oral presentation offering and receiving holistic rating of an oral presentation Listening comprehension- listening to TV or radio broadcasts Listening comprehension- listening to TV or radio broadcasts
and checking comprehension with a partnerand checking comprehension with a partner Writing-revising work on your own, peer-editingWriting-revising work on your own, peer-editing Reading- reading textbook passages followed by self-check Reading- reading textbook passages followed by self-check
comprehension questions, self-assessment of reading habitscomprehension questions, self-assessment of reading habits
(page 416, Brown, 2001)(page 416, Brown, 2001)
Authentic AssessmentAuthentic Assessment
Performance assessment- any form of Performance assessment- any form of assessment in which the student constructs a assessment in which the student constructs a response orally or in writing. It requires the response orally or in writing. It requires the learner to accomplish a complex and significant learner to accomplish a complex and significant task, while bringing to bear prior knowledge, task, while bringing to bear prior knowledge, recent learning, and relevant skills to solve recent learning, and relevant skills to solve realistic or authentic problems (O’Malley & realistic or authentic problems (O’Malley & Valdez, 1996; Herman, et. al., 1992). Valdez, 1996; Herman, et. al., 1992).
Examples of Authentic AssessmentExamples of Authentic Assessment
Portfolio assessmentPortfolio assessment Student self-assessmentStudent self-assessment Peer assessmentPeer assessment Student-teacher conferencesStudent-teacher conferences Oral interviewsOral interviews Writing samplesWriting samples Projects or exhibitionsProjects or exhibitions Experiments or demonstrationsExperiments or demonstrations
Characteristics of performance Characteristics of performance assessmentassessment
Constructed responseConstructed response Higher-order thinkingHigher-order thinking AuthenticityAuthenticity IntegrativeIntegrative Process and productProcess and product Depth versus breadthDepth versus breadth
Journals Journals
Specify to students the purpose of the journalSpecify to students the purpose of the journal Give clear directions to students on how to get started Give clear directions to students on how to get started
(prompts for instance “I was very happy when…)(prompts for instance “I was very happy when…) Give guidelines on length of each entryGive guidelines on length of each entry Be clear yourself on the principal purpose of the journalBe clear yourself on the principal purpose of the journal Help students to process your feedback, and show Help students to process your feedback, and show
them how to respond to your responsesthem how to respond to your responses
ConferencesConferences
Commonly used when teaching writingCommonly used when teaching writing One-on-one interaction between teacher and One-on-one interaction between teacher and
studentstudent Conferences are formative assessment as Conferences are formative assessment as
opposed to offering a final grade or a summative opposed to offering a final grade or a summative assessment. In other words, they are meant to assessment. In other words, they are meant to provide guidance and feedback.provide guidance and feedback.
PortfoliosPortfolios
Commonly used with the communicative language Commonly used with the communicative language teaching approach (CLT)teaching approach (CLT)
It is a collection of students’ work that demonstrates to It is a collection of students’ work that demonstrates to students and others the efforts, progress and students and others the efforts, progress and achievements in a given area. You can have a reading achievements in a given area. You can have a reading portfolio or a writing portfolio, for instanceportfolio or a writing portfolio, for instance
You can also have a reflective or assessment portfolio You can also have a reflective or assessment portfolio as opposed to collecting every piece of evidence for as opposed to collecting every piece of evidence for each objective achieved in the courseeach objective achieved in the course
Portfolio GuidelinesPortfolio Guidelines
Specify the purpose of the portfolioSpecify the purpose of the portfolio Give clear directions to students on how to get startedGive clear directions to students on how to get started Give guidelines of acceptable materials or artifactsGive guidelines of acceptable materials or artifacts Collect portfolios on a pre-announced dates and return Collect portfolios on a pre-announced dates and return
promptlypromptly Help students to process your feedbackHelp students to process your feedback Establish a rubric to evaluate the portfolio and discuss Establish a rubric to evaluate the portfolio and discuss
it with your studentsit with your students
Cooperative Test ConstructionCooperative Test Construction
Cooperative test construction involves the Cooperative test construction involves the students contribution to the design of test items. students contribution to the design of test items. It is based on the concept of collaborative and It is based on the concept of collaborative and cooperative learning in which students are cooperative learning in which students are involved in the processinvolved in the process
(Brown, 2001, p. 420)(Brown, 2001, p. 420)