Introduction to Quantitative Research Methods

Jen DeWittOctober 2013

Quantitative and qualitative data Quantitative data: derived from

measurement, quantification Qualitative data: derived from study of social

processes in naturalist way Underlying philosophical assumptions (may

be) incompatible But spectrum rather than dichotomy Social researchers increasingly use mixed

methods

Quantitative vs Qualitative research strategies Quantitative:

deductive approach: research assumes the existence of an objective, stable truth

positivistic - “scientific” - can be replicated usually large-scale: many respondents – but little depth? statistical analysis

Qualitative: inductive approach: research subjective - reality

constructed, dynamic relativistic - reflexive - researcher effects usually fewer participants – but greater depth interpretive analysis

Deductive and inductive strategies

Deductive

Theory

Hypothesis

Data collection

Findings

Hypothesis confirmed/rejected

Revision of theory

Inductive Research question

Data collection

Analysis

Theory

Data collection and analysis

Refinement of theory

Research concerns suited to quantitative research methods Measurement: How many? What proportion?

including measurement of attitudes Distribution: average values, dispersion Association: how is y affected by changes in

x? Causation - Why? Warning: an association between 2 variables

does not necessarily indicate a causal relationship

Large-scale research: sampling

To generalise from sample to population, sample needs to be representative of population

Ideal: probability sampling For random sampling the larger the sample, the

more representative Response rate Bias, including from non-response

How do non-respondents differ from respondents?

Ingredients of quantitative research Variables: characteristics that may vary

amongst subjects in sample or population Explanatory/independent variable eg gender Response/dependent variable eg GCSE

subjects chosen Descriptive statistics eg frequencies,

averages, distributions, standard deviation, variance, associations/correlations

Types of variableType of variable will affect nature of analysisContinuous /interval: Scale of magnitude with equal distance between points e.g. age in years; test scores

Ratio: 0 is meaningful and can calculate ratios (e.g. income)

Ordinal (categories can be ordered/ranked, but ‘distance’ between categories not equal e.g. social class/level of education)Nominal/categorical (categories which cannot be rank- ordered e.g. modes of transport to school)

Dichotomous (yes/no; male/female)

Presenting data

Histogram Bar chart Scatterplot Line graph

Presenting data

Histogram: graph of distribution of interval data using bars: no gaps between bars E.g. frequency of Maths exam scores

Histogram

Presenting data

Bar chart: graph of distribution of categorical data using bars: gaps between bars to indicate data not continuous E.g. frequency of subjects taken at GCSE

Presenting data

Scatterplot: shows association of 2 variables, e.g. marks in English coursework and on English exam; extracurricular lessons and free school meals

0.0 3.0 6.0 9.0 12.0 15.0

percentage of pupils entitled to free school meals

0.0

20.0

40.0

60.0

80.0

100.0

pe

rce

nta

ge

of

pu

pil

s w

ho

ha

ve

ev

er

ha

d l

es

so

ns

R Sq Linear = 0.513

Presenting data

Line graph: shows association of 2 variables where points are in sequence, e.g. how average height varies with age

Descriptive statistics 1Average measures Mean

Mode

Median

Descriptive statistics 1Average measures Mean

Arithmetic average Unsuitable when extreme scores distort

distribution Mode

Most frequently occurring value Median

Middle score of values in rank order Most suitable where there are a few extreme

scores

Calculate mean, mode, median10 children, marks out of 10, score:

1 5 6 6 6 7 8 8 9 10

Mode?

Median?

Mean?

Descriptive statistics 2: Frequency distribution, Continuous data For most naturally occurring variables on a

continuous scale, many values lie close to the mean, and progressively fewer values are seen towards the ends of the distribution E.g. height, weight, IQ

Called the normal distribution Important to statistics because of the

geometric/arithmetic properties of the curve Continuous data approximating to the normal

distribution called parametric data

The bell-shaped curve:normal distribution

The bell-shaped curve:normal distribution

Central Limit Theorem Vital to probability calculations Has to do with mathematical properties of

normal distributions and sampling

Area under the curve

More terminology

Variance is a measure of dispersion i.e. spread of values around the mean

Standard deviation is square root of variance Measures of spread/variance are important to statistical

calculations/tests

Properties of the (perfect) normal distribution

Mean, mode and median are same 68% of population scores will fall within 1 standard

deviation of the mean and 95% within 2 [1.96] standard deviations

Area under the curve

Shape of the curve

Wide flat curve – large standard deviation High thin curve – small standard deviation So size of standard deviation tells you about

the spread around the mean Before doing parametric tests you need to

make sure that the data is roughly normally distributed and not too skewed ie mean and median not too far apart

Hypothesis testing

Hypothesis: prediction of likely outcome Statistical tests examine either a difference

between 2/more groups or an association between 2/more variables

null hypothesis states that there is no difference between groups/ no association between variables All statistical tests start from basis null hypothesis

is true until tests demonstrate otherwise

P values p = ‘probability’ (how likely/probable

something is to be true…) commonly used p values are

p ≤ 0.05 -result likely to be observed less than 1/20 if there is no difference in reality (i.e. if null hypothesis true); usually regarded as ‘statistically significant’

p ≤ 0.01 - result likely to be observed less than 1/100 if there is no difference

Statistical tests: overview

Which tests to use depends on whether you are:

Comparing 2 or more groups of data Looking for associations between 2 variables

as well as whether you are using Categorical Ordinal or Continuous data

Comparing groups: Parametric tests (continuous data): Independent t test Assumes normal distribution; continuous data; similar

variance in each of 2 groups; data from 2 groups not paired

Used to calculate if 2 groups differ from each other: 1-tailed test for directional hypothesis e.g. intervention

group will show greater improvement in numeracy scores than control group

2-tailed test for non-directional hypothesis e.g. there will be a difference between AQA and OCR GCSE results but direction unknown

Comparing groups: Non-parametric tests Mann-Whitney U test for continuous or

ordinal scale data, regardless of distribution and OK for small or unequal sample sizes

Chi-square test for number counts in discrete categories (nominal/categorical); minimum response size needed in each cell

Fisher’s exact test: dichotomous dependant variable with mutually exclusive frequency count e.g. present/absent; passed/failed

Comparing 3/more unrelated groups Parametric: ANOVA (continuous dependent

measure, approximate normal distribution, similar variance)

Chi-square for dependent measure with frequency counts in mutually exclusive, unrelated categories and minimum sample size

Tests of association:correlation and linear regression Correlation reflects strength of relationship between a

paired set of measurements Does not imply causation Straight line relationship Tests may be parametric or non-parametric

Linear regression examines (nature of) relationship between 2 variables where Both variables continuous and distribution normal Relationship between variables is causal and straight

line Also need paired data. . .

Correlation Parametric data:

Pearson (coefficient r) if both variables are continuous and data spread across full range

Non-parametric data: Spearman’s rho for ordinal or continuous data with a

skewed distribution All correlation coefficients are between -1 to 1

0 no association + correlation: as scores on one variable increase, so

do those on the other - correlation: as scores on one variable increase,

those on the other decrease

Linear regression

Uses equation for a straight line ie y = a + xb

y dependent variable

x independent variable

a is value of y when x is 0 (where graph crosses y axis)

b is slope of line For more than 2 variables, need multiple or

logistic regression

Examples Researchers suspect boys and girls have a

different attitude to homework. Outline a research method to investigate effect of gender on homework undertaken. What variables would you use? Formulate a hypothesis. What problems might be encountered?

How would you go about finding out whether and to what extent the amount of revision undertaken impacts on exam scores?

Evaluating the methodology Validity: are you measuring what you think you

are? do the measures you are using accurately represent

the construct you want to investigate e.g. how valid are self-report scales for depression? Do IQ tests really measure intelligence?

Reliability: do you consistently get the same measurements in same/comparable conditions?

Generalisability: to what extent will the findings from your sample apply to the population as a whole?

Golden Rules Demonstrate appropriateness of methods

chosen to research question

Discuss any implementation issues

Show awareness of strengths and limitations of method(s) chosen

Suggested texts Bryman A and Cramer D (2009) Quantitative data analysis with

SPSS 14, 15 & 16: a guide for social scientists, London: Routledge Cohen L, Manion, L and Morrison K (2000 5th ed) Research

methods in education London: Routledge Falmer Edwards A and Talbot R (1999 2nd ed) The hard-pressed

researcher, London: Longman Field A (2009 3rd ed) Discovering Statistics using SPSS, London:

Sage Plewis I (1997) Statistics in Education, London: Arnold Scott I and Mazhindu D (2005) Statistics for health care

professionals London: Sage Steinberg W (2008) Statistics Alive! Thousand Oaks: Sage Walker J and Almond P (2010) Interpreting Statistical Findings

Maidenhead: Open University Press Wright B (2002) First Steps in Statistics, London: Sage

jennifer.dewitt@kcl.ac.uk