Pedrajita, J

Embed Size (px)

Citation preview

  • 8/9/2019 Pedrajita, J.

    1/48

    Item Bias Detection ModelsItem Bias Detection Models

    for Test Reliabilityfor Test Reliabilityand Validityand Validity

    JOSE Q. PEDRAJITA, PhDJOSE Q. PEDRAJITA, PhDCollege of EducationCollege of Education

    University of the PhilippinesUniversity of the Philippines

  • 8/9/2019 Pedrajita, J.

    2/48

    TABLE OF CONTENTSTABLE OF CONTENTS

    Background of the StudyBackground of the Study ObjectivesObjectives

    Methodology and HypothesesMethodology and Hypotheses

    ResultsResults ConclusionsConclusions

    RecommendationsRecommendations

    AcknowledgementAcknowledgement

  • 8/9/2019 Pedrajita, J.

    3/48

    BACKGROUND OF THE STUDYBACKGROUND OF THE STUDY

    One way to investigate potential bias at the item level is throughOne way to investigate potential bias at the item level is through differentialdifferentialitem functioningitem functioning (DIF) analysis.(DIF) analysis.

    What is Differential Item Functioning (DIF) Analysis?What is Differential Item Functioning (DIF) Analysis?

    -- is a means of statistically identifying unexpected differences inis a means of statistically identifying unexpected differences in

    performance across matched groups of examinees. Itperformance across matched groups of examinees. Itcompares the performance of matched majority (orcompares the performance of matched majority (orreference) and minority (or focal) group examinees.reference) and minority (or focal) group examinees.

    What is Differential Item Functioning (DIF)?What is Differential Item Functioning (DIF)?

    -- Refers to the differing probabilities of success on an item ofRefers to the differing probabilities of success on an item ofexaminees of the same ability but belonging to differentexaminees of the same ability but belonging to differentgroups; that is, when examinees from different groups have agroups; that is, when examinees from different groups have adifferent probability or likelihood of answering an itemdifferent probability or likelihood of answering an itemcorrectly after controlling for overall test performance.correctly after controlling for overall test performance.

  • 8/9/2019 Pedrajita, J.

    4/48

    BACKGROUND OF THE STUDYBACKGROUND OF THE STUDY

    DIF is said to be present in a test:DIF is said to be present in a test: When despite controls for overall test performance,When despite controls for overall test performance,

    examinees from different groups have a differentexaminees from different groups have a differentprobability of answering an item correctly or whenprobability of answering an item correctly or whenexaminees from two subpopulations with the same traitexaminees from two subpopulations with the same trait

    level have different expected scores on the same itemlevel have different expected scores on the same item((CamilliCamilli andand ShepardShepard, 1994;, 1994; KamataKamata and Vaughn, 2004).and Vaughn, 2004).

    When individuals having the same ability, but fromWhen individuals having the same ability, but from

    different groups, do not have the same probability ofdifferent groups, do not have the same probability ofgetting the item right (getting the item right (HambletonHambleton,, SwaminathanSwaminathan, and, andRogers, 1991).Rogers, 1991).

  • 8/9/2019 Pedrajita, J.

    5/48

    BACKGROUND OF THE STUDYBACKGROUND OF THE STUDY

    An item that exhibits DIF may or may notbe biased foror againstAn item thatexhibitsDIF mayormaynotbe biased fororagainstanygroup(anygroup(KanjeeKanjee, 2007), 2007)

    DIF maybe attributed toitem biasbutmayalsoreflectperformanceDIF maybe attributed toitembiasbutmayalsoreflectperformancedifferencesthatthe testisdesigned tomeasure (differencesthatthe testisdesigned tomeasure (CamilliCamilli andandShepardShepard, 1994)., 1994).

    WhatisItemBias?WhatisItemBias?

    refers to invalidity or systematic error in how a test itemrefers to invalidity or systematic error in how a test itemmeasures a construct for the members of a particular groupmeasures a construct for the members of a particular group((CamilliCamilli andand ShepardShepard, 1994). When a test item unfairly favors, 1994). When a test item unfairly favors

    one group of examinees over another, the item is biased.one group of examinees over another, the item is biased.

  • 8/9/2019 Pedrajita, J.

    6/48

    OBJECTIVESOBJECTIVES

    This study isThis study is

    designed to detect potentially biased test items anddesigned to detect potentially biased test items and

    tests the effect of potentially biased itemstests the effect of potentially biased itemselimination on theelimination on the

    contentcontent validityvalidity,,

    concurrentconcurrent validityvalidity,, andand

    internal consistencyinternal consistency reliabilityreliability

    of an Achievement Test in Chemistry.of an Achievement Test in Chemistry.

  • 8/9/2019 Pedrajita, J.

    7/48

    METHODOLOGYMETHODOLOGY

    DIF ModelsDIF Models

    ChiChi--Square (XSquare (X22)) This approach to the identification of test item bias examinesThis approach to the identification of test item bias examines

    the likelihood or probability of test takers from differentthe likelihood or probability of test takers from differentgroups with the same ability levels correctly responding to angroups with the same ability levels correctly responding to anitem. An item is considered unbiased when all persons at aitem. An item is considered unbiased when all persons at a

    given ability level have an equal probability of correctlygiven ability level have an equal probability of correctlyanswering an item regardless of their group membership.answering an item regardless of their group membership.With this strategy, the proportion of responses within abilityWith this strategy, the proportion of responses within abilitycategories for two groups diverse in some criterion iscategories for two groups diverse in some criterion isexamined.examined.

    Hypothesis:Hypothesis:

    There is no significant difference in proportions attaining aThere is no significant difference in proportions attaining acorrect response across total score categories on the testcorrect response across total score categories on the testitems between the public and private, the male and female,items between the public and private, the male and female,

    and the low and high ability examineesand the low and high ability examinees ..

  • 8/9/2019 Pedrajita, J.

    8/48

    METHODOLOGYMETHODOLOGYDIF ModelsDIF Models

    DistractorDistractor Response Analysis (DRA)Response Analysis (DRA) The essential strategy of this technique is to examine theThe essential strategy of this technique is to examine the

    incorrect alternatives to a test item for differences in patternsincorrect alternatives to a test item for differences in patternsof response among different subgroups of a population. Theof response among different subgroups of a population. Thefunction of distracter is to determine the significance of thefunction of distracter is to determine the significance of the

    differences among two or more groups response frequenciesdifferences among two or more groups response frequenciesin the discrete categories of question distracters.in the discrete categories of question distracters.

    Hypothesis:Hypothesis:

    There is no significant difference in proportions selectingThere is no significant difference in proportions selecting

    distracters on the test items between the public and private,distracters on the test items between the public and private,the male and female, and the low and high abilitythe male and female, and the low and high abilityexamineesexaminees..

  • 8/9/2019 Pedrajita, J.

    9/48

    METHODOLOGYMETHODOLOGY

    DIF ModelsDIF ModelsLogistic Regression (LR)Logistic Regression (LR)

    A kind of regression analysis often used when the dependentA kind of regression analysis often used when the dependentvariable is dichotomous and scored 0 or 1. It can also bevariable is dichotomous and scored 0 or 1. It can also beused when the dependent variable has more than twoused when the dependent variable has more than two

    categories. It is usually used for predicting whethercategories. It is usually used for predicting whethersomething will happen or notsomething will happen or not -- anything that can beanything that can beexpressed as event/nonexpressed as event/non--event. Independent variables mayevent. Independent variables maybe categorical or continuous.be categorical or continuous.

    Hypothesis:Hypothesis: The population value is zero for either the differenceThe population value is zero for either the difference

    between the proportions correct or the logs odds ratio on thebetween the proportions correct or the logs odds ratio on thetest items between the public and private, the male andtest items between the public and private, the male andfemale, and the low and high ability examineesfemale, and the low and high ability examinees ..

  • 8/9/2019 Pedrajita, J.

    10/48

    METHODOLOGYMETHODOLOGY

    DIF ModelsDIF Models

    MantelMantel--HaenszelHaenszel Statistic (MH)Statistic (MH) The MH is a nonThe MH is a non--parametric contingency table procedureparametric contingency table procedure

    commonly used to perform statistical test for uniform DIF. It iscommonly used to perform statistical test for uniform DIF. It is

    also used to estimate a log odds ratio (also used to estimate a log odds ratio (MHMH) that yields a measure) that yields a measureof effect size for evaluating the amount of DIF that is present. Thisof effect size for evaluating the amount of DIF that is present. Thisratio value is rescaled asratio value is rescaled as DD == -- 2.352.35MHMH to produce the deltato produce the delta--MHMH(D(D--MH). A positive DMH). A positive D--MH indicates DIF in favor of the focalMH indicates DIF in favor of the focalgroups, and a negative value signifies DIF in favor of thegroups, and a negative value signifies DIF in favor of thereference groups. The degrees of DIF in test items are labeled A,reference groups. The degrees of DIF in test items are labeled A,B, and C (ETS item category) to indicateB, and C (ETS item category) to indicate negligiblenegligible,, moderatemoderate,,

    andand largelarge amounts of DIF (amounts of DIF (GierlGierl, 1999)., 1999).

    Hypothesis:Hypothesis:

    There isnosignificantrelationship betweengroupmembershipandtestThere isnosignificantrelationship betweengroupmembershipandtestperformance onthe testitemsbetweenthe publicandprivate,the maleperformance onthe testitemsbetweenthe publicandprivate,the male

    andfemale,andthe lowandhighabilityexamineesandfemale,andthe lowandhighabilityexaminees ..

  • 8/9/2019 Pedrajita, J.

    11/48

    METHODOLOGYMETHODOLOGY

    Content Validity AnalysisContent Validity Analysis

    The content validity of the original test and its versionThe content validity of the original test and its version

    was estimated based on the remaining items afterwas estimated based on the remaining items aftereliminating the biased items identified by each of the DIFeliminating the biased items identified by each of the DIFmethods.methods.

    The degree to which the items comprise an adequateThe degree to which the items comprise an adequate

    sample were based on a sixsample were based on a six--point scale, namely : Adequatepoint scale, namely : Adequate(86(86--100%), Moderately Adequate (71100%), Moderately Adequate (71--85%), Slightly85%), Slightly

    Adequate (56Adequate (56--70%), Slightly Inadequate (4170%), Slightly Inadequate (41--55%),55%),Moderately Inadequate (26Moderately Inadequate (26--40%), and Inadequate (25%40%), and Inadequate (25%and below).and below).

  • 8/9/2019 Pedrajita, J.

    12/48

    METHODOLOGYMETHODOLOGY

    Concurrent Validity AnalysisConcurrent Validity Analysis

    Concurrent validity evidence was secured by examining theConcurrent validity evidence was secured by examining the

    relationship between the examinees test score and their grade pointrelationship between the examinees test score and their grade pointaverage in Science III.average in Science III.

    The Pearson r was used to assessed the concurrent validity of theThe Pearson r was used to assessed the concurrent validity of theinstrument (original and test version) by correlating the examinees scoresinstrument (original and test version) by correlating the examinees scores

    to their GPA in Science III.to their GPA in Science III.

    Hypothesis:Hypothesis:

    There isnosignificantrelationshipbetweenthe examineestestThere isnosignificantrelationshipbetweenthe examineestest

    scoresandtheirgradesinScience III.scoresandtheirgradesinScience III.

    InternalConsistency Reliability AnalysisInternalConsistency Reliability Analysis

    The internal consistency reliability was determined byThe internal consistency reliability was determined bycalculating the KRcalculating the KR--20 Formula coefficient for the20 Formula coefficient for the

    original and the test version.original and the test version.

  • 8/9/2019 Pedrajita, J.

    13/48

    METHODOLOGYMETHODOLOGYStatistical Criteria for Identifying Potentially Biased Test ItemsStatistical Criteria for Identifying Potentially Biased Test Items

    --------------------------------------------------------------------------------------------------------------------------------------------------------

    DIF Methods Focusof AnalysisDIF Methods Focusof Analysis Measure of BiasMeasure of Bias

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    ChiSquareChiSquare Difference inproportionattaininga Significance ofchisquareDifference inproportionattaininga Significance ofchisquare

    correctresponse acrosstotalscorecorrectresponse acrosstotalscore

    categoriescategories

    Distracter Response Difference inproportionsselecting Significance ofchisquareDistracter Response Difference inproportionsselecting Significance ofchisquare

    Analysis distractersAnalysis distracters

    Logistic Regression Odds orlikelihoodofgettingan Significance ofchisquareLogistic Regression Odds orlikelihoodofgettingan Significance ofchisquare

    itemrightitemright

    MantelMantel--HaenszelHaenszel Performstatisticaltestfor Significance ofchisquarePerformstatisticaltestfor Significance ofchisquare

    Statistic evaluatingthe amountofDIF andlarge DIFeffectStatistic evaluatingthe amountofDIF andlarge DIFeffect

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

  • 8/9/2019 Pedrajita, J.

    14/48

    METHODOLOGYMETHODOLOGY

    Test VersionsTest Versions

    Chi SquareChi SquareDistracter Response AnalysisDistracter Response AnalysisLogistic RegressionLogistic RegressionMantelMantel--HaenszelHaenszel StatisticStatistic

    ValidityValidity

    ContentContentConcurrentConcurrent

    DIF ModelsDIF ModelsChi SquareChi SquareDistracter Response AnalysisDistracter Response AnalysisLogistic RegressionLogistic RegressionMantelMantel--HaenszelHaenszel StatisticStatistic

    Matched GroupingMatched Grouping Test ReliabilityTest ReliabilityClass TypeClass Type

    publicpublicprivateprivate

    GenderGendermalemale Chemistry AchievementChemistry Achievementfemalefemale TestTest

    English AbilityEnglish Abilitylowlow

    highhigh METHODOLOGICAL FLOWCHART OF THE STUDYMETHODOLOGICAL FLOWCHART OF THE STUDY

  • 8/9/2019 Pedrajita, J.

    15/48

    METHODOLOGYMETHODOLOGY

    The SampleThe SamplePublic School ExamineesPublic School Examinees

    MaleMale FemaleFemale TotalTotal

    AmorsoloAmorsolo LowLow 22 00 22

    HighHigh 1818 19 19 3737

    TotalTotal 2020 1919 39 39

    KasilagKasilag LowLow 15 15 1414 2929

    HighHigh 44 33 77

    TotalTotal 1919 1717 3636

    MalangMalang LowLow 77 1010 1717

    HighHigh 44 44 88TotalTotal 1111 1414 2525

    Aggregate TotalAggregate Total 50 50 10050 50 100

    the public junior high school examinees was taken from the top, middle, andthe public junior high school examinees was taken from the top, middle, andlower class sections.lower class sections.

  • 8/9/2019 Pedrajita, J.

    16/48

    METHODOLOGYMETHODOLOGY

    The SampleThe Sample

    Private School ExamineesPrivate School Examinees

    MaleMale FemaleFemale TotalTotalPeachPeach LowLow 00 00 00

    HighHigh 1919 2020 3939TotalTotal 1919 2020 39 39

    PurplePurple LowLow 1818 1010 2828HighHigh 11 77 88

    TotalTotal 1919 1717 3636

    FuschiaFuschia LowLow 1010 99 1919HighHigh 22 44 66

    TotalTotal 1212 1313 2525

    Aggregate TotalAggregate Total 50 50 10050 50 100

    the private junior high school examinees was taken the from top, middle, andthe private junior high school examinees was taken the from top, middle, andlower class sectionslower class sections

  • 8/9/2019 Pedrajita, J.

    17/48

    METHODOLOGYMETHODOLOGY

    Research InstrumentResearch InstrumentThe preparation of the researcherThe preparation of the researcher--constructed Chemistryconstructed Chemistry

    Achievement Test involved the following steps:Achievement Test involved the following steps:

    1. Development of a Table of Specifications1. Development of a Table of Specifications

    2. Consultation with adviser/experts2. Consultation with adviser/experts

    3. Generation of an item pool3. Generation of an item pool

    4. Review of the initial item pool by experts4. Review of the initial item pool by experts5. Field5. Field--testingtesting

    6. Item Analysis and test revision6. Item Analysis and test revision

    Data Collection ProcedureData Collection Procedure1) Administration of the test to the original intact classes1) Administration of the test to the original intact classes2) Checking and scoring the test2) Checking and scoring the test

  • 8/9/2019 Pedrajita, J.

    18/48

    METHODOLOGYMETHODOLOGY

    Data Analysis ProcedureData Analysis Procedure

    The analysis of data involved:The analysis of data involved:(a) assignment of examinees to the three comparison groups(a) assignment of examinees to the three comparison groups

    matched by section and total score;matched by section and total score;

    (b) organizing data for every item into a three(b) organizing data for every item into a three--way contingencyway contingencytable;table;

    (c) encoding data in the Statistical Analysis System (SAS)(c) encoding data in the Statistical Analysis System (SAS)computer program;computer program;

    (d) analysis for detecting and testing for differential item(d) analysis for detecting and testing for differential itemfunctioning/item bias for each comparison group,functioning/item bias for each comparison group,

    (e) eliminating the correct responses on the biased items(e) eliminating the correct responses on the biased itemsidentified in the public/private matched group by each of theidentified in the public/private matched group by each of the

    DIF methods, respectively;DIF methods, respectively;(f) retaining the unbiased items, herein referred to as the test(f) retaining the unbiased items, herein referred to as the test

    version for each of the DIF methods, respectively;version for each of the DIF methods, respectively;

    (g) assessment of the test versions content validity, concurrent(g) assessment of the test versions content validity, concurrentvalidity, and internal consistency reliability.validity, and internal consistency reliability.

  • 8/9/2019 Pedrajita, J.

    19/48

    RESULTSRESULTSClass Type BiasClass Type Bias

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Chi SquareChi Square Dis tracter LogisticDistracter Logistic MantelMantel--HaenszelHaenszel

    Response Analysis RegressionResponse Analysis Regression StatisticStatistic

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Potentially Biased Items 1 1 1Potentially Biased Items 1 1 1 11

    Against the Private School 3 3 3 Against the Private School 3 3 3 33

    ExamineesExaminees 5 5 55 5 5

    9 9 9 99 9 9 9

    10 1010 1019 19 1919 19 19 1919

    2121 21 2121 21

    26 2626 26

    3030 30 30 3030 30 30

    3131

    33 33 3333 33 33 3333

    3535

    434347 47 4747 47 47 4747

    5050

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Sub TotalSub Total 9 129 12 1111 1010

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

  • 8/9/2019 Pedrajita, J.

    20/48

    RESULTSRESULTS

    Class Type BiasClass Type Bias

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Chi SquareChi Square Dis tracter LogisticDistracter Logistic MantelMantel--HaenszelHaenszel

    Response Analysis RegressionResponse Analysis Regression StatisticStatistic

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Potentially Biased ItemsPotentially Biased Items 2 22 2

    Against the Public School Against the Public School 8 8 88 8 8

    ExamineesExaminees 13 13 13 1313 13 13 13

    1414 141416 16 1616 16 16 1616

    22 22 2222 22 22 2222

    3232 3232

    3636 3636 3636

    37 37 3737 37 37

    4040 4040

    4141 4141 4141

    4646

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    SubtotalSubtotal 4 6 11 124 6 11 12

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    TotalItems IdentifiedTotalItems Identified 13 1813 18 2222 2222

  • 8/9/2019 Pedrajita, J.

    21/48

    Sample Test ItemsSample Test Items

    Item3Item3

    Whentwoatomsinamolecule are heldtogether bythe transferof an electron fromone atomtotheWhentwoatomsinamolecule are heldtogether bythe transferof an electron fromone atomtotheother,the chemical bond betweenthemis knownasother,the chemical bond betweenthemis knownas

    a.atomic bond.a.atomic bond. c.ionic bond.c.ionic bond.

    b.covalent bond.b.covalent bond. d.metallic bond.d.metallic bond.

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    ChiChi--Square AnalysisSquare Analysis

    Item Ability Level Computed XItem Ability Level Computed X22

    Tabular XTabular X22

    Sig. Prob.Sig. Prob. Biased Against Biased Against

    33 1515 1919 0.65610.6561 6.25 0.01256.25 0.0125 0.41790.4179

    2020 2424 2.68672.6867 0.10120.1012

    2525 2929 0.00000.0000 1.00001.0000

    3030 4141 6.32346.3234 0.0119* Private0.0119* Private

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Distracter Response AnalysisDistracter Response Analysis

    ItemItem Distracters Computed XDistracters Computed X22 ProbabilityProbability Biased Against Biased Against

    33 AA 0.16030.1603 0.68890.6889

    BB 13.905113.9051 0.0002*0.0002* PrivatePrivate

    DD 1.39691.3969 0.23730.2373

  • 8/9/2019 Pedrajita, J.

    22/48

    Item3Item3

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    33 SI 3SI 3 6.67 6.67 0.08320.0832

    CTCT 11 8.768.76 0.0031* Private0.0031* Private

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Distracter Response AnalysisDistracter Response Analysis

    ItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased AgainstRatioRatio CategoryCategory

    33 8.5611 0.0034* 2.4140 0.888.5611 0.0034* 2.4140 0.88 -- 2.072.07 C*C* PrivatePrivate

  • 8/9/2019 Pedrajita, J.

    23/48

    Item 19Item 19

    Whatisthe molarmassof FeWhatisthe molarmassof Fe22OO33 ? (Fe = 56 O = 16)? (Fe = 56 O = 16)

    a. 48a. 48 c. 112c. 112b.72b.72 d. 160d. 160

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    ChiChi--Square AnalysisSquare Analysis

    Item Ability Level Computed XItem Ability Level Computed X22 Tabular XTabular X22 Sig. Prob.Sig. Prob. Biased Against Biased Against

    1919 1515 2828 6.42066.4206 5.026 0.025 0.0113* Private5.026 0.025 0.0113* Private

    2929 4141 0.05030.0503 0.82250.8225

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

  • 8/9/2019 Pedrajita, J.

    24/48

    Item 19Item 19

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    1919 SISI 11 24.0524.05 < 0.0001< 0.0001

    CTCT 11 5.345.34 0.0208*0.0208* PrivatePrivate

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    MantelMantel--HaenszelHaenszel AnalysisAnalysis

    ItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased Against

    RatioRatio CategoryCategory

    1919 5.3047 5.3047 0.0213* 2.1095 0.750.0213* 2.1095 0.75 -- 1.751.75 C*C* PrivatePrivate

  • 8/9/2019 Pedrajita, J.

    25/48

    Item22Item22

    A medicine dropperiscalibrated bycounting the numberof dropsthatdeliver 1 ml.IfA medicine dropperiscalibrated bycounting the numberof dropsthatdeliver 1 ml.If20drops equal 1 ml,the volume of adropis20drops equal 1 ml,the volume of adropis

    a.0.005mla.0.005ml c.0.25mlc.0.25ml

    b.0.05mlb.0.05ml d.0.5mld.0.5ml

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    ChiChi--Square AnalysisSquare Analysis

    Item Ability Level Computed XItem Ability Level Computed X22 Tabular XTabular X22 Sig. Prob.Sig. Prob. Biased AgainstBiased Against

    2222 1515 2323 8.08878.0887 5.73 0.0166 0.0045* Public5.73 0.0166 0.0045* Public

    2424 3030 12.747412.7474 0.0004* Public0.0004* Public

    3131 4141 1.12161.1216 0.28960.2896

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Distracter Response AnalysisDistracter Response Analysis

    Item Distracters Computed XItem Distracters Computed X22 ProbabilityProbability Biased Against Biased Against

    2222 A 7.0913 A 7.0913 0.0077*0.0077* PublicPublic

    CC 6.96166.9616 0.0083*0.0083* PublicPublic

    DD 15.885615.8856 < 0.0001*< 0.0001* PublicPublic

  • 8/9/2019 Pedrajita, J.

    26/48

    Item22Item22

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    2222 SISI 22 12.73 0.0017 12.73 0.0017

    CTCT 11 24.84 < 0.0001* Public24.84 < 0.0001* Public

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------MantelMantel--HaenszelHaenszel AnalysisAnalysis

    ItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased Against

    RatioRatio CategoryCategory

    2222 23.388223.3882

  • 8/9/2019 Pedrajita, J.

    27/48

    Item 13Item 13

    A certain gashasa volume of 1.25A certain gashasa volume of 1.25 lili whenpressure is.8 atm. Whatisthe newwhenpressure is.8 atm. Whatisthe new

    pressure whenthe gasis compressedtoa volume of 1pressure whenthe gasis compressedtoa volume of 1 lili ??a.0.001 atma.0.001 atm c. 1.0atmc. 1.0atm

    b.0.1 atmb.0.1 atm d. 10.0atmd. 10.0atm

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    ChiChi--Square AnalysisSquare Analysis

    Item Ability Level Computed XItem Ability Level Computed X22 Tabular XTabular X22 Sig. Prob. Biased AgainstSig. Prob. Biased Against

    1313 1515 22 1.893022 1.8930 5.735.73 0.0166 0.16890.0166 0.1689

    2323 30 6.814630 6.8146 0.0090* Public0.0090* Public

    3131 41 2.332641 2.3326 0.1267 0.1267

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Distracter Response AnalysisDistracter Response Analysis

    ItemItem Distracters Computed XDistracters Computed X

    22

    ProbabilityProbability Biased Against Biased Against

    1313 AA 2.03952.0395 0.15330.1533

    BB 6.90096.9009 0.0086*0.0086* PublicPublic

    DD 15.363815.3638 < 0.0001*< 0.0001* PublicPublic

  • 8/9/2019 Pedrajita, J.

    28/48

    Item 13Item 13

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    1313 SISI 22 0.930.93 0.62920.6292

    CTCT 11 14.0514.05 0.0002*0.0002* PublicPublic

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    MantelMantel--HaenszelHaenszel AnalysisAnalysis

    ItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased AgainstRatioRatio CategoryCategory

    1313 13.561513.5615 0.0002* 0.30630.0002* 0.3063 -- 1.181.18 2.782.78 C*C* PublicPublic

  • 8/9/2019 Pedrajita, J.

    29/48

    RESULTSRESULTS

    Gender BiasGender Bias------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Chi SquareChi Square Dis tracter LogisticDistracter Logistic MantelMantel--HaenszelHaenszel

    Response Analysis RegressionResponse Analysis Regression StatisticStatistic

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Biased Items Against 17Biased Items Against 17 17 17 17 17

    the Female Examineesthe Female Examinees 2727 27 27 27 27

    34 3434 34

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Biased Items AgainstBiased Items Against 11 11

    the Male Examineesthe Male Examinees 33 33

    4242 4242 4242

    4747 47 47

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Total Number ofTotal Number of

    Items IdentifiedItems Identified 11 22 7 77 7

  • 8/9/2019 Pedrajita, J.

    30/48

    Sample Test ItemsSample Test ItemsItem 17Item 17

    Choose the correct electronicconfigurationof the elementChoose the correct electronicconfigurationof the element4747Na.Na.

    a. 1sa. 1s22 2s2s22 2p2p66 3s3s22 3p3p66 4s4s22 3d3d1010 4p4p66 5s5s22 4d4d99

    b. 1sb. 1s22 2s2s22 2p2p66 3s3s22 3p3p66 4s4s22 3d3d1010 4p4p66 5s5s33 4d4d99

    c. 1sc. 1s22 2s2s22 2p2p66 3s3s22 3p3p66 4s4s22 3d3d1010 4p4p77 5s5s22 4d4d88

    d. 1sd. 1s22 2s2s22 2p2p66 3s3s22 3p3p66 4s4s22 3d3d99 4p4p55 5s5s33 4d4d1010

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    ChiChi--Square AnalysisSquare Analysis

    Item Ability Level Computed XItem Ability Level Computed X22 Tabular XTabular X22 Sig. Prob. Biased AgainstSig. Prob. Biased Against

    1717 1515 4141 5.34925.3492 3.84 .05 0.0207* Female3.84 .05 0.0207* Female

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    1717 SISI 00 0.000.00

    SexSex 11 6.586.58 0.0103*0.0103* FemaleFemale

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    MantelMantel--HaenszelHaenszel AnalysisAnalysis

    ItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased Against

    Ratio CategoryRatio Category

    1717 6.3342 0.0118* 3.11646.3342 0.0118* 3.1164 1.141.14 -- 2.67 C*2.67 C* FemaleFemale

  • 8/9/2019 Pedrajita, J.

    31/48

    Item27Item27

    Whichof the following illustratesthe compressibilityof gases?Whichof the following illustratesthe compressibilityof gases?

    a. filling uparubbertire withaira. filling uparubbertire withair

    b.increase in volume of doughwhen bakedb.increase in volume of doughwhen bakedc.rising ofc.rising of casserolescasseroles coverwhenwater beginsto boilcoverwhenwater beginsto boil

    d.smelling the aromaof brewedcoffeed.smelling the aromaof brewedcoffee

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Distracter Response AnalysisDistracter Response AnalysisItemItem Distracters Computed X Distracters Computed X22 ProbabilityProbability Biased Against Biased Against

    2727 BB 2.93062.9306 0.08690.0869

    CC 6.20036.2003 0.0128*0.0128* FemaleFemaleDD 3.79893.7989 0.05130.0513

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Logistic Regression AnalysisLogistic Regression AnalysisItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    2727 SI 3SI 3 10.1410.14 0.01740.0174

    SexSex 11 9.57 9.57 0.0020*0.0020* FemaleFemale

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    MantelMantel--HaenszelHaenszel AnalysisAnalysisItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased Against

    Ratio CategoryRatio Category

    2727 9.25159.2515 0.0024*0.0024* 2.53062.5306 0.930.93 -- 2.18 C*2.18 C* FemaleFemale

  • 8/9/2019 Pedrajita, J.

    32/48

    Item 1Item 1

    A neighborchoose to burnhis garbage afterthe failure of garbage truckstocollectA neighborchoose to burnhis garbage afterthe failure of garbage truckstocollectthem. Aftera fewminutes,smoke fromthe burning garbage startedto enterthethem. Aftera fewminutes,smoke fromthe burning garbage startedto enterthehouse. Whatpropertyof gasesisillustrated by thissituation?house. Whatpropertyof gasesisillustrated by thissituation?

    a.compressibilitya.compressibility c. elasticityc. elasticity

    b.b.diffusibilitydiffusibility d. expansibilityd. expansibility

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    11 SISI 33 24.1924.19 < 0.0001< 0.0001

    SexSex 11 4.924.92 0.0265*0.0265* MaleMale

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    MantelMantel--HaenszelHaenszel AnalysisAnalysisItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased Against

    Ratio CategoryRatio Category

    11 4.79224.7922 0.0286* 0.50660.0286* 0.5066 -- 0.68 1.6 C*0.68 1.6 C* MaleMale

  • 8/9/2019 Pedrajita, J.

    33/48

    Item3Item3

    Whentwoatomsinamolecule are heldtogether bythe transferof an electron fromWhentwoatomsinamolecule are heldtogether bythe transferof an electron fromone atomtothe other,the chemical bond betweenthemis knownasone atomtothe other,the chemical bond betweenthemis knownas

    a.atomic bond.a.atomic bond. c.ionic bond.c.ionic bond.

    b.covalent bond.b.covalent bond. d.d.metalliicmetalliic bond.bond.

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    33 SI 3SI 3 9.589.58 0.02250.0225

    Sex 1Sex 1 4.594.59 0.0321*0.0321* MaleMale

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    MantelMantel--HaenszelHaenszel AnalysisAnalysis

    ItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased Against

    Ratio CategoryRatio Category

    33 4.48254.4825 0.0342* 0.52400.0342* 0.5240 -- 0.650.65 1.521.52 C*C* MaleMale

  • 8/9/2019 Pedrajita, J.

    34/48

    RESULTSRESULTS

    Ability BiasAbility BiasChi SquareChi Square Dist racter LogisticDistracter Logistic MantelMantel--HaenszelHaenszel

    Response Analysis RegressionResponse Analysis Regression StatisticStatistic

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Biased Items AgainstBiased Items Against 2929 2929

    the High Abilitythe High Ability 3838 3838

    Examinees 45Examinees 45 4545

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Biased Items AgainstBiased Items Against 22 22 22

    the Low Abilitythe Low Ability 33 33 33 33ExamineesExaminees 66 66 66 66

    77

    88 88 88 88

    1313 1313 1313 1313

    1515

    17171919 1919 1919 1919

    2222 2222 22223030 3030

    3636 3636

    4848 4848 4848

    5050 5050 5050

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Total Number ofTotal Number of

    Items IdentifiedItems Identified 77 1111 13 1413 14

  • 8/9/2019 Pedrajita, J.

    35/48

    Sample Test ItemsSample Test ItemsItem29Item29

    Howmany valence electronsdoesHowmany valence electronsdoes ClCl atomhave ?atomhave ?

    a.2a.2 c. 17c. 17

    b.7b.7 d. 18d. 18

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    2929 SISI 22 8.238.23 0.01630.0163

    AbilityAbility 11 4.984.98 0.0256*0.0256* High AbilityHigh Ability

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    MantelMantel--HaenszelHaenszel AnalysisAnalysis

    ItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased Against

    RatioRatio CategoryCategory

    2929 4.8128 0.0282* 2.0515 0.724.8128 0.0282* 2.0515 0.72 -- 1.691.69 C*C* High AbilityHigh Ability

  • 8/9/2019 Pedrajita, J.

    36/48

    Item36Item36

    Waterisaddedtoashproduced fromthe burning of wood. The solutionchangesredWaterisaddedtoashproduced fromthe burning of wood. The solutionchangesredlitmuspaperto blue. Howwouldyouclassifythe solution?litmuspaperto blue. Howwouldyouclassifythe solution?

    a.acida.acid c.metalc.metal

    b. baseb. base d.nond.non--metalmetal

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Logistic Regression AnalysisLogistic Regression Analysis

    ItemItem LR StatisticsLR Statistics Biased Against Biased Against

    SourceSource dfdf Chi SquareChi Square ProbProb

    3636 SI 3SI 3 0.47 0.47 0.92640.9264

    AbilityAbility 11 4.154.15 0.0416* Low Ability0.0416* Low Ability

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    MantelMantel--HaenszelHaenszel AnalysisAnalysis

    ItemItem MHX MHX22 ProbProb OddsOdds MHMH DD--MH ETS Item Biased AgainstMH ETS Item Biased Against

    RatioRatio CategoryCategory

    3636 4.05234.0523 0.0441* 0.53010.0441* 0.5301 -- 0.630.63 1.491.49 C*C* Low AbilityLow Ability

  • 8/9/2019 Pedrajita, J.

    37/48

    RESULTSRESULTS

    Content Validityof the Test VersionContent Validityof the Test Version

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    TestTest VersionVersion NoNo.. ofof ItemsItems Percent Percent DescriptionDescription

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    OriginalTestOriginalTest 5050 100100 AdequateAdequate

    Chi SquareChi Square 37 37 7474 ModeratelyadequateModeratelyadequate

    DistracterDistracter 3232 6464 SlightlyadequateSlightlyadequate

    Logistic Regression 28Logistic Regression 28 5656 SlightlyadequateSlightlyadequate

    MantelMantel--HaenszelHaenszel 2828 5656 SlightlySlightly adequateadequate

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------The degree to which the items comprise an adequate sample were based on a sixThe degree to which the items comprise an adequate sample were based on a six--point scale, namely :point scale, namely :Adequate (86Adequate (86--100%), ModeratelyAdequate (71100%), ModeratelyAdequate (71--85%), SlightlyAdequate (5685%), SlightlyAdequate (56--70%), Slightly Inadequate70%), Slightly Inadequate

    (41(41--55%), Moderately Inadequate (2655%), Moderately Inadequate (26--40%), and Inadequate (25% andbelow).40%), and Inadequate (25% andbelow).

  • 8/9/2019 Pedrajita, J.

    38/48

    RESULTSRESULTS

    Concurrent Validityof the Test VersionsConcurrent Validityof the Test Versions

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------Test VersionTest Version Validity CoefficientValidity Coefficient DescriptionDescription

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    OriginalOriginalTestTest 00..585585**** moderatemoderate relationshiprelationship

    MantelMantel--HaenszelHaenszel 00..507507**** moderatemoderate relationshiprelationship

    ChiChi SquareSquare 00..504504**** moderatemoderate relationshiprelationship

    LogisticLogistic RegressionRegression 0 0..499499**** moderatemoderate relationshiprelationship

    DistracterDistracter 00..462462**** moderatemoderate relationshiprelationship

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------****pp

  • 8/9/2019 Pedrajita, J.

    39/48

    RESULTSRESULTSInternal Consistency Reliabilityof the Test VersionInternal Consistency Reliabilityof the Test Version

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Test Version No.of ItemsTest Version No.of Items Reliability CoefficientReliability Coefficient

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Original TestOriginal Test 5050 0.710.71

    Chi SquareChi Square 3737 0.640.64

    DistracterDistracter 3232 0.620.62

    Logistic RegressionLogistic Regression 2828 0.570.57

    MantelMantel--HaenszelHaenszel 2 828 00..5757

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    The internal consistency reliability was determined by calculating the KRThe internal consistency reliability was determined by calculating the KR--20 Formula20 Formulacoefficient for the original and the test version.coefficient for the original and the test version.

  • 8/9/2019 Pedrajita, J.

    40/48

    The presence of potential class type bias, gender bias,The presence of potential class type bias, gender bias,

    and ability bias in the Chemistry Achievement Test canand ability bias in the Chemistry Achievement Test can

    be attributed to:be attributed to:

    1.1. Discrepancies in the curriculum of the public andDiscrepancies in the curriculum of the public andprivate school investigated;private school investigated;

    2.2. Unfamiliarity with the content of the items whichUnfamiliarity with the content of the items whichcaused the examinees to be attracted to the incorrectcaused the examinees to be attracted to the incorrect

    options;options;3.3. Disparities of the matched examinees exposure to theDisparities of the matched examinees exposure to theinformation, concepts, vocabularies, or skills reflectedinformation, concepts, vocabularies, or skills reflectedin the content of the biased items;in the content of the biased items;

    4.4. Items which may reflect information and/or skills thatItems which may reflect information and/or skills thatwas not experienced by the matched examinees;was not experienced by the matched examinees;

    5.5. Ambiguities in the item stem, keyed response, orAmbiguities in the item stem, keyed response, ordistracters; anddistracters; and

    6.6. Overly difficult reading level or inability of the matchedOverly difficult reading level or inability of the matchedexaminees to comprehend or understand the conceptsexaminees to comprehend or understand the conceptsreflected on the biased items.reflected on the biased items.

  • 8/9/2019 Pedrajita, J.

    41/48

    CONCLUSIONSCONCLUSIONS

    The results of the differential item functioning analysis showedThe results of the differential item functioning analysis showed

    that there were potentially biased test items between the publicthat there were potentially biased test items between the publicand private, the male and female, and the low and high abilityand private, the male and female, and the low and high abilityexaminees. Hence, potentialexaminees. Hence, potential class type biasclass type bias,, gender biasgender bias, and, andEnglish ability biasEnglish ability bias were present in the Chemistry Achievementwere present in the Chemistry AchievementTest. Overall, it appears that students from public schoolsTest. Overall, it appears that students from public schoolsperformed better than those from private schools; male andperformed better than those from private schools; male and

    female examinees performed fairly; and low ability examineesfemale examinees performed fairly; and low ability examineesperformed miserably than the high English ability examinees.performed miserably than the high English ability examinees.

    There were agreement and disagreement among the DIFThere were agreement and disagreement among the DIFmethods in the identity and number of potentially biased itemsmethods in the identity and number of potentially biased itemsidentified. There were items which were identified identically (a)identified. There were items which were identified identically (a)by the four models, b) by any three of the four models, c) byby the four models, b) by any three of the four models, c) byany two of the four models, and d) by a single model.any two of the four models, and d) by a single model.

  • 8/9/2019 Pedrajita, J.

    42/48

    CONCLUSIONSCONCLUSIONSThe Logistic Regression and the MantelThe Logistic Regression and the Mantel--HaenszelHaenszel StatisticStatisticyielded very similar results with respect to uniform DIF. Theyielded very similar results with respect to uniform DIF. Thetwo procedures result in similar number and identity oftwo procedures result in similar number and identity ofitems identified. Hence, there is a high degree ofitems identified. Hence, there is a high degree ofcorrespondence between these two procedures.correspondence between these two procedures.

    Elimination of biased items in a test tend to decrease itsElimination of biased items in a test tend to decrease its

    content validity, concurrent validity, and internalcontent validity, concurrent validity, and internalconsistency reliability, as it diminishes the length or numberconsistency reliability, as it diminishes the length or numberof items of the test.of items of the test.

    The use of statistical methods in identifying biased testThe use of statistical methods in identifying biased test

    items is a relatively better kind of item analysis because byitems is a relatively better kind of item analysis because bysubjecting test items to DIF approaches, test items whichsubjecting test items to DIF approaches, test items whichwere unfairly difficult and widely discriminating for awere unfairly difficult and widely discriminating for aparticular group of examinees are determined. Byparticular group of examinees are determined. Byeliminating, replacing, or revising these biased items aeliminating, replacing, or revising these biased items avalid, reliable, and fairer test would be made.valid, reliable, and fairer test would be made.

  • 8/9/2019 Pedrajita, J.

    43/48

    RECOMMENDATIONSRECOMMENDATIONS

    1. Test experts and developers should use contingency table1. Test experts and developers should use contingency table(CT) methods, particularly the LR and MH methods, in item bias(CT) methods, particularly the LR and MH methods, in item biasdetection. The two procedures result in similar number of itemsdetection. The two procedures result in similar number of items(and similar items) being identified. The two methods are viable in(and similar items) being identified. The two methods are viable inthe detection of DIF and are widely implemented in both testthe detection of DIF and are widely implemented in both testconstruction and research settings.construction and research settings.

    2. Educational evaluation practitioners should engage in item2. Educational evaluation practitioners should engage in itembias detection and use Logistic Regression or Mantelbias detection and use Logistic Regression or Mantel--HaenszelHaenszelStatistic for bias correction, which means that identified biasedStatistic for bias correction, which means that identified biaseditems should be revised or replaced. Then, reitems should be revised or replaced. Then, re--administer the testadminister the testand subject it anew to item bias detection in order to further refineand subject it anew to item bias detection in order to further refine

    and purify the required item content of a test. This process couldand purify the required item content of a test. This process couldmake differentially functioning items between groups of interest bemake differentially functioning items between groups of interest bemore valid, reliable, and fair. Bias correction may maintain ormore valid, reliable, and fair. Bias correction may maintain orimprove the measurement qualities of a test such as its contentimprove the measurement qualities of a test such as its contentvalidity, concurrent validity, and internal consistency reliability.validity, concurrent validity, and internal consistency reliability.

  • 8/9/2019 Pedrajita, J.

    44/48

    RECOMMENDATIONSRECOMMENDATIONS

    3.3. PotentiallyPotentially biased items should either be revised or replacedbiased items should either be revised or replacedsince its elimination and nonsince its elimination and non--replacement lessen the number of items inreplacement lessen the number of items ina test. The lesser the number of items, the smaller was the contenta test. The lesser the number of items, the smaller was the contentvalidity, concurrent validity, and internal consistency.validity, concurrent validity, and internal consistency.

    4. In this study, matching was done by conditioning simultaneously4. In this study, matching was done by conditioning simultaneouslyon test score, and a categorical variable, namely,on test score, and a categorical variable, namely, total scoretotal score andand classclasstypetype for the public/private comparison group,for the public/private comparison group, total scoretotal score andand sexsex forforthe male/female comparison group, andthe male/female comparison group, and total scoretotal score andand English abilityEnglish abilityfor the low/high ability comparison group. In connection with the abovefor the low/high ability comparison group. In connection with the above--mentioned conditioning, it is also recommended that a study bementioned conditioning, it is also recommended that a study be

    conducted using Logistic Regression or Mantelconducted using Logistic Regression or Mantel--HaenszelHaenszel Statistic byStatistic byincorporating more than two or multiple ability estimate into a DIF/itemincorporating more than two or multiple ability estimate into a DIF/itembias analysis. That is, matching should be conditioned simultaneously onbias analysis. That is, matching should be conditioned simultaneously ontotal score, a categorical variable, and additional educationaltotal score, a categorical variable, and additional educationalbackground variables like age, verbal ability, mathematical ability, socialbackground variables like age, verbal ability, mathematical ability, social

    class, educational attainment, type of community, and the like..class, educational attainment, type of community, and the like..

  • 8/9/2019 Pedrajita, J.

    45/48

    RECOMMENDATIONSRECOMMENDATIONS

    5.5. Future studies shouldFuture studies should focus onfocus on other psychometric issuesother psychometric issues

    not addressed in this study. These include matters related tonot addressed in this study. These include matters related tocomparative study ofcomparative study ofItem Response TheoryItem Response Theory (IRT) and(IRT) andContingency TableContingency Table (CT) methods on any relevant psychometric(CT) methods on any relevant psychometricissue, such as of test equating, and item banking.issue, such as of test equating, and item banking.

    6. Educational institutions, educational evaluators, and test6. Educational institutions, educational evaluators, and testexperts and developers should give increasingattention toexperts and developers should give increasingattention toequity of test scores for various subpopulations of examinees,equity of test scores for various subpopulations of examinees,be it regular or students with learningdisabilities. Test equitybe it regular or students with learningdisabilities. Test equitycan be achieved by ensuring that a test measures onlycan be achieved by ensuring that a test measures only

    constructconstruct--relevant differences between subpopulations ofrelevant differences between subpopulations ofexaminees. To achieve test equity amongsubpopulations ofexaminees. To achieve test equity amongsubpopulations ofexaminees, bias testingmust be conducted especially for veryexaminees, bias testingmust be conducted especially for veryimportant tests like entrance examination and professionalimportant tests like entrance examination and professionallicensure examination.licensure examination.

  • 8/9/2019 Pedrajita, J.

    46/48

    RECOMMENDATIONSRECOMMENDATIONS

    7.7. One of the objectives of this paper is on detectingOne of the objectives of this paper is on detecting

    DIF/biased items. However, it is also recommended that furtherDIF/biased items. However, it is also recommended that furtherstudies be conducted to go beyond detecting biased items andstudies be conducted to go beyond detecting biased items andobtain additional information about DIF/biased items. Someobtain additional information about DIF/biased items. Someitems may show larger magnitude of DIF, while some othersitems may show larger magnitude of DIF, while some othersshow relatively small magnitude of DIF. In such a situation, it isshow relatively small magnitude of DIF. In such a situation, it is

    of interest to investigate sources of such variation.of interest to investigate sources of such variation.

  • 8/9/2019 Pedrajita, J.

    47/48

    REFERENCESREFERENCESCamilliCamilli, G. and, G. and ShepardShepard, L. (1994)., L. (1994). Methodsfor Identifying Biased TestItemsMethodsfor Identifying Biased TestItems. Volume 4, Sage. Volume 4, Sage

    Publications, Inc., California.Publications, Inc., California.Ferguson, G. A. andFerguson, G. A. and TakaneTakane, Y. (1989)., Y. (1989). Statistical Analysis in Psychology and EducationStatistical Analysis in Psychology and Education. Sixth. SixthEdition, New York : McGrawEdition, New York : McGraw--Hill, Inc.Hill, Inc.FraenkelFraenkel, J.R. and, J.R. and WallenWallen, N.E. (1993)., N.E. (1993). How ToDesign and Evaluate Research in EducationHow ToDesign and Evaluate Research in Education..Second Edition,Second Edition, MsGrawMsGraw--Hill Book, Inc.Hill Book, Inc.GierlGierl, M. J. (1999)., M. J. (1999). Differential Item Functioningon the Alberta Education Social Studies 30Differential Item Functioningon the Alberta Education Social Studies 30Diploma ExaminationDiploma Examination. Canadian Social Studies, Vol. 33, No. 2. Canadian Social Studies, Vol. 33, No. 2..KamataKamata, A. and Vaughn B. (2004)., A. and Vaughn B. (2004). An Introduction toDifferential Item Functioning Analysis.An Introduction toDifferential Item Functioning Analysis.LearningDisabilities: A Contemporary Journal 2(2), 49LearningDisabilities: A Contemporary Journal 2(2), 49 69.69.KanjeeKanjee, A. (2007)., A. (2007). Using Logistic Regression toDetect BiasWhen Multiple Groups Are Tested.Using Logistic Regression toDetect BiasWhen Multiple Groups Are Tested.South African Journal ofPsychology, 37, 47South African Journal ofPsychology, 37, 47 61.61.MazorMazor, K. E.,, K. E., KanjeeKanjee, A., and, A., and ClauserClauser, B. E. (1995)., B. E. (1995). Using Logistic Regression and the MantelUsing Logistic Regression and the Mantel--HaenszelHaenszel With Multiple AbilityEstimatestoDetectDifferential Item Functioning.With Multiple AbilityEstimatestoDetectDifferential Item Functioning. Journal ofJournal ofEducational Measurement, 32, 131Educational Measurement, 32, 131 144.144.OsterlindOsterlind, Steven J. (1983)., Steven J. (1983). TestItem BiasTestItem Bias. Sage Publications, Inc., California.. Sage Publications, Inc., California.Reynolds, C. R., Livingston, R. B., andReynolds, C. R., Livingston, R. B., and WillsonWillson, V. (2006). Measurement and Assessment in, V. (2006). Measurement and Assessment inEducation. Pearson Education, Inc.Education. Pearson Education, Inc.

    Rogers, H. J. &Rogers, H. J. & SwaminathanSwaminathan, H. (1993)., H. (1993). A Comparison ofLogistic Regression and MantelA Comparison ofLogistic Regression and Mantel--HaenszelHaenszel Proceduresfor DetectingDifferential Item FunctioningProceduresfor DetectingDifferential Item Functioning. Applied Psychological. Applied PsychologicalMeasurement, 17, 105Measurement, 17, 105 116.116.SwaminathanSwaminathan, H. and Rogers, H. J. (1990)., H. and Rogers, H. J. (1990). DetectingDifferential Item Functioning Using LogisticDetectingDifferential Item Functioning Using LogisticRegression Procedures.Regression Procedures. Journal ofEducational Measurement, 27, 361Journal ofEducational Measurement, 27, 361 370.370.

  • 8/9/2019 Pedrajita, J.

    48/48

    ACKNOWLEDGEMENTACKNOWLEDGEMENTThe presenter expresses his sincere appreciationThe presenter expresses his sincere appreciation

    and gratitude to :and gratitude to :

    Conference OrganizersConference Organizers

    Dr. CarloDr. Carlo MagnoMagno Chair, NCEME Scientific CommitteeChair, NCEME Scientific Committee

    Dr.Dr. LinaLina MiclatMiclat Secretary, PEMEASecretary, PEMEA

    Thank you!Thank you!