Magno, C

8/9/2019 Magno, C

1/22

Carlo Magno, PhD.De La Salle University, Manila

8/9/2019 Magno, C

2/22

Development of Psychometrics

What is psychometrics? Field of study concerned with the theory and

technique of educational and psychologicalmeasurement, which includes the measurementof knowledge, abilities, attitudes, and personality

traits. Two major research tasks: (i) the construction of instruments and proceduresfor measurement; and

(ii) the development and renement of theoreticalapproaches to measurement.

Michell, J. (1997). Quantitative science and the denition of measurement inpsychology. British Journal of Psychology, 88, 355-383.Thurstone, L.L. (1929). The measurement of psychological value. In T.V. Smith &W.K. Wright (Eds.), Essays in Philosophy by Seventeen Doctors of Philosophy of theUniversity of Chicago. Chicago: Open Court.

8/9/2019 Magno, C

3/22


Quality of our work adheres to the standardsset by the Joint Committee on Standards forEducation Evaluation:

Propriety Utility Feasibility Accuracy

Joint Committee on Standards for Educational Evaluation. (2003; 1998; 1994).

8/9/2019 Magno, C

4/22

Topical Areas in Psychometrics

CTT: Estimation of errors and true score Establishing validity and reliability Item Response Theory (IRT)

One, two, three parameter models Cognitive diagnosis (DINA Model) Polychotomous Response theory Testlet Response theory

Multidimensional IRT Differential Item Functioning (DIF) Equating Tests

8/9/2019 Magno, C

5/22

Regulated Psychometrics

Philippine Psychology Act of 2009 Functions of a psychometrician: (1) administering and scoring of objectivepersonality test, structured personality test,pen and paper intelligence, achievement andinterest tests (excluding projectives)

(2) Interpret results (3) Conducting preparatory intake interview

8/9/2019 Magno, C

6/22


Who is qualied to be a psychometrician? Bachelors degree in psychology Good moral character Passed a test on the following areas:

Theories of personality Abnormal psychology

Industrial psychology Psychological assessment

8/9/2019 Magno, C

7/22


Given the functions of a psychometricianaccording to the Philippine Psychology Act:

Forfeits the need for advance training because the

job function is primarily administering and scoringtests! Construction of tests and renement of

methodologies are neglected! The functions regulated do not capture the real

functions of a psychometrician. Low status given to psychometricians because

they are not as qualied as the psychologists.

8/9/2019 Magno, C

8/22


in PublicationsPublication

PublicPresentation

Write reports

Our work isreviewed by a

panel of experts

Our work is evaluatedwhether we are

contributing to theory

8/9/2019 Magno, C

9/22

Levels of Publication

ISI abstracted Journals

Abstracted journal

International Journal

Local journal

Conference proceeding

In house publication

8/9/2019 Magno, C

10/22

Publications that accommodate studies npsychometrics (in the Philippines)

Educational Measurement and Evaluation Review(International)

The Assessment Handbook (International) APSA Journal of SBA Research (local)

Philippine Journal of Educational Measurement(local) Learning Edge (local) Philippine Journal of Psychology (local) Philippine Journal of Counseling Psychology

(local) Guidance Journal (local) Asia-Pacific Education Researcher (International,

ISI)

8/9/2019 Magno, C

11/22

Publication Counts of Studies onPsychometrics

Survey on Philippine Publication inPsychology from 1986-1996 (Bernardo, 2002)

Area N

Social psychology 63 (41.7%)

Child Psychology 19 (12.6%)Family psychology 15 (9.9%)

Health psychology 13 (8.6%)

Clinical psychology 11 (7.3%)

Counseling psychology 8 (5.3%)

Industrial and Organizational psychology 8 (5.3%)

General psychology (history of psychology) 5 (3.3%)

Methodology 5 (3.3%)

Educational psychology 4 (2.6%)

Cognitive psychology 4 (2.6%)

Psychological Measurement 3 (2.0%)

Cross-cultural psychology 1 (0.7%)

8/9/2019 Magno, C

12/22

Publication Counts of

Studies on Psychometrics Type of ResearchType of Research N

Descriptive 78 (51.7%)

Theoretical analysis 19 (12.6%)Review of literature 18 (11.9%)

experiment 16 (10.6%)

Theory development/modelbuilding

12 (7.9%)

Program development 10 (6.6%)Statistical analysis 4 (2.6%)

Test development 3 (2.0%)

Historical analysis 2 (1.3%)

8/9/2019 Magno, C

13/22

2010 Study of Bernardo

Area of study 1994-1998

1999-2003

2004-2008

Total

Social psychology and personality 3 13 22 38

Clinical and counseling psychology 6 16 13 35

Developmental psychology 4 6 7 17

Industrial and organizational psychology 1 1 13 15

Educational psychology 2 3 5 10

Psychometrics or psychological measurement 1 3 3 7

General (history, trends, current issues) 2 3 2 7

Cognitive psychology 1 1 0 2

Biological psychology 0 0 1 1

Sport psychology 0 0 1 1

Total 20 46 67 133

Studies published in the Philippine Journal of Psychology

8/9/2019 Magno, C

14/22

2010 study of Bernardo

Research approach 1994-1998

1999-2003

2004-2008

Total

Non-experimental quantitative (descriptive) 4 16 31 51

Qualitative 4 12 15 31

Literature review (historical analysis, trends) 8 9 4 21

Non-experimental quantitative (predictive) 0 2 8 10

Program development and evaluation 3 3 3 8

Psychometric analysis 1 3 3 7

Experiment 0 1 4 5

Total 20 46 67 133

Studies published in the Philippine Journal of Psychology

8/9/2019 Magno, C

15/22

Filipino authors in International refereedjournals

Area of study 1994-1998

1999-2003

2004-2008

Total

Social psychology and personality 2 12 17 31

Industrial and organizational psychology 0 7 7 14

Cognitive psychology 3 4 3 10Educational psychology 3 5 2 10

Clinical and counseling psychology 1 5 2 8

Developmental psychology 0 0 2 2

General (history) 0 1 1 2

Psychometrics or psychologicalmeasurement 0 1 0 1

Sport psychology 0 0 1 1

Total 9 35 35 79

8/9/2019 Magno, C

16/22

Filipino authors in International refereedjournals

Institution 1994-1998

1999-2003

2004-2008

Total

Experiment 5 13 7 25

Non-experimental quantitative(predictive)

1 7 13 21

Non-experimental quantitative(descriptive)

0 4 6 10

Qualitative 0 5 4 9

Program development and evaluation 1 2 4 7

Psychometric analysis 2 2 1 5

Literature review (historical analysis,trends)

0 2 0 2

Total 9 35 35 79

Research approaches of international refereed publications

8/9/2019 Magno, C

17/22

Implications

According to Bernardo (2002) Lack of research resources Research as a marginal activity Random and isolated efforts at research Lack of critical mass and peer review

8/9/2019 Magno, C

18/22

Moves of IRT in the Philippines

Introduction of the IRT approach in psychometricscourses in the graduate programs. Dr. Letty Azusano includes a topic on Rasch model in

advance psychometric theory course Dr. Davies introducing IRT analysis in personality testing Dr. Pedrajita and his team developed papers detecting item

Bias Tests developed by the Asian Psychological Services

Assessment (APSA) uses the Rasch model in theselection of good items.

Assessment of College Potential Assessment of Nursing Potential Assessment of Engineering Potential Assessment of Accounting Potential

8/9/2019 Magno, C

19/22


De La Salle-Santiago Zobel (DLSZ) asinitiated by Ms. Ma Socorro Diestaconstructed a curriculum-based exam (items

were calibrated as persona and ability free). DLSZ pioneered on training the basiceducation teachers in analyzing theirquarterly test using the Rasch model

8/9/2019 Magno, C

20/22


Publication Special issue on studies comparing CTT and IRT by 4

Filipino authors in The International Journal ofEducational and Psychological Assessment (ISI, 2009): Magno

Hernandez Morales Tiratira

Pedrajta published a study on detecting item biasusing logistic regression in a 2009 issue.

The Asia-Pacic Education Researcher (ISI) publishedone article where Polychotomous IRT was used for theAsian Values Scale

8/9/2019 Magno, C

21/22


Schools that offer graduate training onMeasurement and Evaluation:

University of the Philippines

De La Salle University Philippine Normal University Mirriam College

8/9/2019 Magno, C

22/22

Recommendations

Funding and scholarship should be given tograduate students who intend to pursue agraduate training on Measurement,

assessment, and evaluation. Training should be provided specically in theconduct of IRT

Encourage graduate students and scholars toconduct and publish studies using IRT.

Documents

Magno, C