Educ Rsch Pp14

McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved.

Educational Research: Fundamentals for the Consumer

Woolfolk / PerryChild and Adolescent Development © 2012 Pearson Education, Inc. All rights reserved.

Sixth Edition

McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved.

The Intelligent Consumer: Putting It All Together

Chapter 14

McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 3

Discussion Topics

The purpose of the text Questions for quantitative studies Questions for qualitative studies Questions for mixed-method studies Reviewing two studies


Text Purpose

Present and explain the fundamental principles of educational research• The quality of published educational research varies

greatly• There is a need to have the knowledge and skills to

critically evaluate educational research Rephrasing the consumer tips from each

chapter into questions to provide guidelines for the evaluation of research articles


Quantitative Criteria

Topics for evaluation questions for quantitative studies (see pages 380 to 382 for a full list of questions)• Research problem• Review of the literature• Research hypotheses• Selection of participants• Instrumentation• Design• Results and analysis• Discussion and conclusions


Questions for Quantitative Studies

Research problem• Is the problem researchable?• Is the problem significant? Will the results

have practical or theoretical importance?• Does the problem indicate the variables and

population studies?



Review of the literature• Does the review of literature seem

comprehensive? • Is the review up-to-date?• Have studies been critically reviewed, flaws

noted, and results summarized?• Does the review explicitly relate previous

studies to the problem?• If appropriate, does the review establish a

basis for research hypotheses?



Research hypotheses• Is the hypothesis stated in declarative form?• Does the hypothesis follow from the

literature?• Does the hypothesis state expected

relationships or differences?• Is the hypothesis testable?• Is the hypothesis clear and concise?



Selection of participants• Are the participants clearly described?• Is the population and sampling clearly

defined?• What is the return rate in a survey study?• Are volunteers used?• Is there an adequate number of participants?



Instrumentation• Is evidence for validity and reliability clearly

stated and adequate? • If an instrument is designed for a study by the

researchers, is there a description of its development?

• Are the procedures for gathering data clearly described?

• Do the scores distort the reality of the findings?• Do response sets or faking influence the

results?



Design - nonexperimental• Do graphic presentations distort the findings?• Are causative conclusions reached from

correlational findings?• Is the correlation affected by restriction in the range

and reliability of the instruments?• If causal-comparative, has the causal condition

already occurred? How comparable are the subjects in the groups being compared?



Design - experimental• Is there direct manipulation of an

independent variable?• Is the design clearly described?• What extraneous variables are not

controlled in the design?• Is each replication of the treatment

independent of the other replications?



Single subject• Is the measurement of the target behavior reliable?• Is the target behavior clearly defined?• Are there enough measures of the behavior to

establish stability?• Are procedures, subjects, and settings described in

detail?• Are there experimenter or observer effects?



Results• Is there an appropriately descriptive statistical

summary?• Is statistical significance confused with practical

significance?• Is statistical significance confused with internal or

external validity?• Are appropriate statistical tests used?• Is there a sufficient number of subjects to give valid

statistical results?• Are data clearly and accurately presented in graphs

and tables?



Discussion and conclusions• Are the results discussed in relation to previous

research, methodology, and the research problem?• Do the conclusions follow from the interpretation of

the results? • Are the conclusions appropriately limited by the

nature of the subjects, treatments, and measures?• Are the limitations of the findings reasonable? • Are the recommendations and implications specific?


Qualitative Criteria

Topics for evaluation questions for qualitative studies (see pages 382 and 383 for a full list of questions)• Introduction and problem• Review of the literature• Methodology• Results and Analysis• Discussion and conclusions


Questions for Qualitative Studies

Introduction and problem• Are the researcher’s background, interests,

and potential biases clear from the outset?• Is the problem significant?• Is there a clear conceptual and theoretical

framework for the problem?• Does the introduction include an overview

of the design of the study?• Is the purpose of the study clearly stated?



Review of the literature• Does the review indicate that the researcher is

knowledgeable about previous work in the area?• Is the review up-to-date?• Does the review establish an adequate background

and theoretical framework for the study?• Is the review well organized?• Is the literature analyzed as well as summarized?



Methodology• Will the participants selected provide a

credible answer to the research question?• Will the researcher’s involvement affect the

findings?• How detailed are the field notes?• Are multiple methods of data collection used?• Is the study long enough?



Discussion and conclusions• Are descriptions clearly separate from

interpretations and researcher’s opinions?• Is the credibility of the findings addressed?• Are the results discussed in relation to previous

research?• Do the conclusions follow from the interpretation of

the results? Are the conclusions consistent with what is known from previous research?

• Are appropriate limitations indicated?• Are appropriate recommendations and implications

indicated?


Mixed-method Criteria

Topics for evaluation questions for qualitative studies (see pages 384 and 385 for a full list of questions)• Introduction and problem• Review of the literature• Selection of participants• Instrumentation• Design• Results• Discussion and conclusions


Questions for Mixed-method Studies

Research problem• Is the problem significant? Will the results

have practical or theoretical importance?• Are there separate questions for each part

of the study?• Is it clear if the study is primarily

quantitative, qualitative, or convergent?



Review of the literature• Does the review of literature seem

comprehensive? • Is the review up-to-date?• Have studies been critically reviewed, flaws

noted, and results summarized?• Does the review establish an adequate

background and theoretical framework for the study?



Selection of participants• Are the participants clearly described?• Is it likely that the participants will provide

credible data?• Are the participants likely to be biased



Instrumentation• Is evidence for validity and reliability clearly

stated and adequate? • Are the procedures for gathering data clearly

described?• Is there likely to be bias from participants,

interviewers, and/or observers?• Are multiple methods of data collection used?



Design• Is the design explanatory, exploratory, or

convergent?• Is the design clearly described?• Is the quantitative part nonexperimental or

experimental? How do you know?• Is it clear how the research gained entry in to the

field and his or her role?• Is the design likely to provide credible information?



Results• Are data clearly and adequately described?• Is statistical significance confused with

practical significance?• Are verbatim transcriptions and quotations

used?• Are appropriate statistical tests used?• Are results clearly separate from researcher

opinion?



Discussion and conclusions• Are the results discussed in relation to

previous research, methodology, and the research problem?

• Do the conclusions follow from the interpretation of the results?

• Are the limitations of the findings indicated?

• Are the recommendations appropriate?


Evaluation of Studies

Evaluation of a qualitative research article• Article 1 – How Elementary School Counselors Can

Meet the Needs of Students with Disabilities by Frye, H.N.

Evaluation of a quantitative, experimental research article• Article 2 – The Effect of Reciprocal Peer Counseling

in the Enhancement of Self-Concept Among Adolescents by Egbochuku, E.O., & Obiunu, J.J.

Documents

Educ Rsch Pp14