Download pdf - ECNU Educational Research Methodologyfaculty.ecnu.edu.cn/picture/article/220/40/7d/8ad5b9c74a... · 2016-09-22 · ECNU Educational Research Methodology Five steps for systematic

ECNU Educational Research Methodology

•Reading assignment •Class PPT •Requirements/Assessments •Email for questions

Getting started

陈芳华东师大英语系 Fang CHEN 2016/9/22 1


A. characteristics of educational research B. The research process

Lecture 1: Introduction

2016/9/22 陈芳华东师大英语系 Fang CHEN 2


The general characteristics of educational research are: 1. Empirical 2. Systematic 3. Valid 4. Reliable 5. Variety of forms

陈芳华东师大英语系 Fang CHEN

A. Characteristics of educational research

2016/9/22 3


All knowledge is derived from sense experience.

But the result of this experience must take some kind of informational form (data) so that knowledge can be generated: testing hypotheses.

e.g. Fingernail

1. Empirical



Five steps for systematic approach to research: 1. Identifying the problem 2. Reviewing information 3. Collecting data 4. Analyzing data 5. Drawing conclusions

This is not a lock-step process.

2. Systematic



In general, for something to be valid, we want it to be based on fact or evidence, that is, “capable of being justified”.

Validity of research deals with the accurate interpretability of the results (internal validity) and the generalizability of the results (external validity).

3. Validity



Internal validity is the extent to which the results of a research study can be interpreted accurately and with confidence.

Internal validity



Good: Figure1.1: Physical education: Possible effects of two different

exercise programs upon the physical performance test scores of 8th grade boys.

Bad: Figure 1.2: The difference effects of three types of materials on

performance in eighth-grade science What are the plausible and competing explanations for

high test scores in School III?

Internal validity: Examples



Figure 1.1



Figure 1.2



External validity is the extent to which research results are generalizable to populations and/or conditions.

External validity



Case1-Good: 125 interviews conducted of parents’ perceptions of the school, random selection, random replacement for parents not available

Case2---?: 712 surveys sent out, 149 completed and returned. Return rate of 21%.

Case3---?: 10 graduate students volunteered for a research on the effect length of visual exposure on the recall of nonsense symbols.

External validity: example



2016/9/22 13 陈芳华东师大英语系 Fang CHEN


Validity is a matter of degree. No perfect validity in a study. Attempts in research design to enhance internal

validity may decrease external validity, and vice versa.

Balance: so that results can be interpreted with reasonable certainty and still have some useful generalizability.



Reliability refers to the consistency of the research and the extent to which studies can be replicated.

Internal reliability: e.g. observer agreement External reliability: e.g. Is the study (methods,

conditions,etc.) replicable? If replicated, will the result be the same?

4. Reliability



Validity vs. reliability



Classification of educational research Basic vs. applied research Quantitative vs. qualitative research General methods of research

5. Classification of educational research



Basic and applied research are differentiated by their goals or purposes.

The primary purpose of the basic research is the

extension of knowledge; • E.G. learning observed in a laboratory setting: presenting

learning materials in a figural or a verbal manner

The purpose of applied research is the solution of an immediate, practical problem.

Basic vs. applied



Misconceptions

1. Basic research is complex and applied research is simple in its methodology.

2. Basic research is performed by abstract, impractical thinkers, whereas applied research is carried out by unsophisticated practitioners.

3. Basic research is precise and exacting but of little or no value in a real situation, whereas applied research is often sloppy and haphazard but of great practical value.

☒ ☒ ☒



Action research is usually conducted by teachers, administrators, or other educational professional for solving a specific problem or for providing information for decision making at the local level. Focuses on the solution of day-to-day problems, Little concern about generalizing to other

educational settings. Evaluation

More about applied research



Krathwohl 1993 shows the main distinction: Qualitative research: research that describes

phenomena in words instead of numbers or measures….

Quantitative research: research that describes phenomena in numbers and measures instead of words.

Qualitative vs. quantitative research



Other differences



Break



Commonly used methods: 1.Experimental research 2.Quasi-experimental research 3.Nonexperimental quantitative research 4.Historical research 5.Ethnographic research

General Methods of Research



Quasi- Variables of interest; Manipulate Survey research: a nonexperimental research variables are studied as they exist in the situation

but not manipulated a) Describing the status quo b) ex post facto research c) causal-comparative studies or correlational



Historical research What happened in the past How did they start, how did they influence later

decisions E.g. a study of federal assistance programs in USA

for secondary education during the period 1945-1960.



Ethnographic research Associated with anthropology. the process of providing scientific descriptions of

educational systems, processes, and phenomena within their specific contexts.

Keywords: observation, description, qualitative, natural, holistic,

no strong theoretical base. e.g. the nature of science instruction in a junior high

school



Key words:



A theory is a generalization or series of generalizations by which we attempt to explain some phenomena in a systematic manner.

？？？Theory is used more common with basic research than with applied research, and with quantitative research more than with qualitative research.

The Role of Theory



It provides a framework for conducting research, and it can be used for synthesizing and explaining (through generalizations) research results.

Theory stimulates research, and conversely,

research stimulates theory development and theory testing.

Role of theory



However, theories are valuable only to the extent that they are useful. A good theory is developed in such a way that

generalizations can be tested. The theory must adequately explain the events of phenomena under study.

A theory should be stated in the simplest form, they should be succinct and precise, avoiding ambiguities and unnecessary complexity.



B. The research process Identifying a Research Problem Reviewing the Literature Research Design and Data Collection Analyses Interpretation and Discussion



The Activities of the Research Process



Important because Helps start the research Defines important variables Begins to define type of study Determines the success of the research

Identification of a Research Problem



Selection of the Research Problem

There are many ways to select a research problem They are not developed in a day. Your own basic interests Given to you

In general, they are developed and refined

through multiple interactions with the integration of several forms of information. So it can take a while. Interactions with others



Characteristics of Question

Should be interesting to you. Should be original The answer should be useful or significant Should be researchable Some considerations are philosophical in nature

and not researchable. Should be feasible



Statement of the Problem

The Statement can have two forms Declarative/Descriptive Form Question Form

Both are fine!

The key is that the statement should be precise, concise and should identify the key factors (variables) of the research study..



Example 2.1

Original: Achievement and teaching techniques Restatement: A study of the effects of three teaching

techniques on science achievements of junior high school students.

Questions: Do three different teaching techniques have differing effects on science achievement scores of junior high students?

Direction? Focus? Key Factors/Framework?



Example 2.2: In real paper…

2016/9/22

variables

39 陈芳华东师大英语系 Fang CHEN


Example 2.3

2016/9/22

variables

40 陈芳华东师大英语系 Fang CHEN


Constants, variables, and operational definition

Constants – Those things that are the same for all individuals in the study. E.g. grade level, all fifth-graders

Variables – Those things that are not the

same for all individuals in the study Scores on the science achievement test.



Variables

Independent variable (IV) Those variables that affect the outcome, or what

we use to explain the difference in the outcome e.g. Teaching techniques

Dependent / response variable (DV) Those variables that are typically measured as an

outcome of the independent variable, that can be affected by the independent variable

e.g., Achievement in the forms of test scores



How does different teaching techniques and materials affect students’ achievement on various subjects?

A comprehensive study: Five schools, students were grouped on the basis

of previous performance as high, average and low, both boys and girls, two kinds of materials, science achievement, reading achievement, spelling achievement, you are also informed of the learning style of the students and the teaching style of the teachers…

What are the variables?

Example 2.4: Figure 2.1



Figure 2.1



Table 2.1 for four scales of measurement Example 4 School? Gender? Previous performance level? Materials? Achievement? Teaching style?

Measured Variables



Operational Definition

An operational definition of a variable describes how or by what means we are going to measure the variable.

Examples of variables that need an operational

definition. Anxiety Previous performance level Achievement High school seniors



Hypotheses

Elaborate on the Research Statement

Conjecture or guess at the solution to the problem.

Takes on some characteristics of a theory

Come from… Research statement, Literature, Data



Characteristics

Should have a reason (e.g., theory)

States a relationship between variables of interest

It should be concise and testable So we should also think about operationalizing our

variables



Example 2.5

Original hypothesis: Bright students have good attitudes toward school.

Revised hypothesis 1: Students aged nine through eleven who score in the

upper 25 percent of their class on the (standardized) IQ test have a higher mean score on the “X-Y-Z Attitude Toward School Inventory” than students who score in the lower 75 percent of the class on the IQ test.

Revised hypothesis 2: A positive relationship exists between the scores on the

(specific) IQ test and the ( specific) attitude inventory for students aged nine through eleven.



Types of Hypotheses

Substantive Hypothesis

Statistical Hypothesis



Sometimes called a “research hypothesis”. It is a tentative statement about what could happen. “As punitive disciplinary methods are increased in

an elementary school, student achievement will decrease.”

May also be in the form of foreshadowed

statements which is usually associated with ethnographic research.

Substantive Hypothesis



Research statement: “An ethnographic study of the school environment of regular

and learning disabled students to determine factors related to potential dropout.”

Foreshadowed statements:

Interaction among regular and learning disabled students during instruction

Role of the teachers in enhancing student learning …

Foreshadowed statements do not specify anticipated

results. Essentially, they identify factors for the researcher to observe as the research gets underway.

Example 2.6



Statistical Hypothesis

Given in quantitative terms.

Could be written in terms of parameters. “The mean reading achievement of the population

of third-grade students taught by Method A equals the mean reading achievement of the population taught by Method B.”



Relevant terminologies

Two kinds of hypotheses: Null hypothesis

• Assumes no effect Alternative hypothesis

• Included all other possibilities • “The reading achievement means of the populations of

third-graders taught by Method A and Method B are not equal.”

Also they can be directional or nondirectional



All research is not done for the purpose of testing hypothesis. “What is the average physical performance level of

young adult males on Text XYZ?”

Exploratory studies…



To Summarize

In determining the research problem Construct research statement Construct all operation definitions Construct all hypotheses

Remember: researchable and feasible



Should be systematic Where is the information found What should be done with the information What is made of the information

Review of the literature



Example: discussion part



Next Time

Reading 1. Research Design In Quantitative Research 2. Experimental research
