ECNU Educational Research Methodology

Embed Size (px)

Text of ECNU Educational Research Methodology

  • ECNU Educational Research Methodology

    Lecture 2 Research design in quantitative research

    2016/4/18 1 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Review of last class

    2016/4/18 Fang Chen ECNU 2

  • ECNU Educational Research Methodology

    QUANTITATIVE RESEARCH

    Quantitative research has its roots in positivism, which Gall, Gall and Borg (2003) have defined as: The epistemological doctrine that physical and social reality is independent of those who observe it, and that observations of this reality, if unbiased, constitute scientific knowledge (p.632).

    2016/4/18 3 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Research Design in quantitative research

    Research Design in quantitative research tends to be structured and prescriptive, much more so than in qualitative research.

    The outcomes are typically expressed as

    numbers and enables the researchers to make valid interpretations through comparisons and partitioning of those numbers.

    2016/4/18 4 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Purpose of design

    What is the purpose of research design? 1. Provide answers to questions

    2. Control variance This, to a large extent, unique to quantitative

    research.

    2016/4/18 5 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Controlling variance

    Controlling variance means creating conditions that allow the researcher to get a clear look at the variable of interest by limiting or eliminating the influence of some variables and explaining the influence of others.

    2016/4/18 6 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Example

    Research statement: A study of the effect of teaching method on the

    performance of high school students enrolled in chemistry

    Method 80%

    Method 55%

    2016/4/18 7 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Controlling Variance

    1. Randomization

    2. Building conditions into the design as independent variables

    3. Holding conditions constant

    4. Statistical adjustments

    2016/4/18 8 Fang Chen ECNU

  • ECNU Educational Research Methodology

    1. Randomization

    Recall that at least some of the variability may be from the differences between my groups.

    If this is the case, we want it to be only because of the teaching method, NOT because of other factors Randomly pick participants from population Randomly place in groups

    2016/4/18 9 Fang Chen ECNU

  • ECNU Educational Research Methodology

    2016/4/18 Fang Chen ECNU 10

  • ECNU Educational Research Methodology

    The randomization essentially equalized the groups with respect to ability level, but it also equalized them with respect to other variables such as motivation, attitude, to name a few.

    Although this process distributes the effect of ability level evenly, it does not allow the researcher to quantify the influence of ability level on the scores of the chemistry test.

    Figure 4.1

    2016/4/18 Fang Chen ECNU 11

  • ECNU Educational Research Methodology

    2. Build in Factors

    If there is something that could affect the outcome, we can include it in the design.

    They also account for variability, therefore reducing what is considered random noise.

    Figure 4.3 Build in the ability factor

    2016/4/18 12 Fang Chen ECNU

  • ECNU Educational Research Methodology

    2016/4/18 Fang Chen ECNU 13

  • ECNU Educational Research Methodology

    Issues

    At times, adding additional factors is not possible

    Need more examinees

    May be too expensive

    May take too much time

    2016/4/18 14 Fang Chen ECNU

  • ECNU Educational Research Methodology

    3. Holding Factors Constant

    We could constrain some factors to be constant.

    Think of student ability Only use students with IQ score between 100-108.

    2016/4/18 15 Fang Chen ECNU

  • ECNU Educational Research Methodology

    One danger is that you will lose external validity. Generalizability of research findings

    Issues

    2016/4/18 Fang Chen ECNU 16

  • ECNU Educational Research Methodology

    4. Statistical control

    Use all the data collected, Statistically control the factors that are

    relevant to the study if measure of those factors exist

    Statistical control would most likely account for more variance than building in student ability as a two-category, independent variable.

    2016/4/18 17 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Example

    2016/4/18 Fang Chen ECNU 18

    MethodoreChemTestSc 10 +=

    IQMethodoreChemTestSc 210 ++=

    TeacherMethodoreChemTestSc 210 ++=

    MethodoreChemTestSc 10 +=

  • ECNU Educational Research Methodology

    Adequate scores on a control variable may not be available.

    Statistical assumptions must be met; if they are not tenable, the statistical procedures should not be used.

    Issues

    2016/4/18 Fang Chen ECNU 19

  • ECNU Educational Research Methodology

    Extending the chemistry example: In stead of using only 90 students, we used 180

    within a school. Students come from different elementary schools

    with various science background. The three teaching method is still the independent

    variable of interest Three teachers teaching 6 classes in total, with 2

    classes per teaching method IQ scores from 8th grade are available and

    controlled

    Using procedures for control in combination

    2016/4/18 Fang Chen ECNU 20

  • ECNU Educational Research Methodology

    Figure 4.7

    2016/4/18 Fang Chen ECNU 21

    1. Randomization

    2. Building conditions into the design as independent variables

    3. Holding conditions constant

    4. Statistical adjustments

  • ECNU Educational Research Methodology

    Characteristics of Good Research

    Freedom From Bias

    Freedom from Confounding

    Control Extraneous Variables

    Statistical Precision

    2016/4/18 22 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Characteristics of Good Research

    Freedom From Bias E.g. higher ability students assigned to one method Students volunteered into a program may have more

    motivation

    Freedom from Confounding If each teacher uses one method, method effectiveness

    may be confounded with teacher effectiveness

    Control Extraneous Variables Statistical Precision 2016/4/18 23 Fang Chen ECNU

  • ECNU Educational Research Methodology

    Characteristics of Good Research

    Freedom From Bias Freedom from Confounding Control Extraneous Variables Randomization Build the factor into design Hold the factor constant Statistical control

    Statistical Precision Statistical precision is increased with larger samples

    and when additional independent variables are built into the research design.

    2016/4/18 24 Fang Chen ECNU

  • ECNU Educational Research Methodology

    When you read any article

    Think about: 1. Independent and dependent variables. 2. Controlling variance. 3. Are there other factors that they might have

    wanted to account for? 4. Does the study have bias or confounding

    variables?

    2016/4/18 25 Fang Chen ECNU

    Lecture 2Review of last classQuantitative ResearchResearch Design in quantitative researchPurpose of designControlling varianceExampleControlling VarianceRandomizationSlide Number 10Figure 4.1Build in FactorsSlide Number 13IssuesHolding Factors ConstantIssuesStatistical controlExampleIssues Using procedures for control in combinationFigure 4.7Characteristics of Good ResearchCharacteristics of Good ResearchCharacteristics of Good ResearchWhen you read any article