Evidence, argumentation and rhetoric in a PCT

Trish GreenhalghJill RussellPrimary Care and Population Sciences

With input from Janet McDonnell and Emma Byrne from University of Greenwich

Background

Trish’s dual interest in evidence based medicine and medical humanities (‘narrative based medicine’)

Jill’s interest in social policy Janet’s interest in argumentation Emma’s disillusionment with formal logic

Where we are coming from

We are less interested in the rules of evidence or its commonalities across disciplines than in how evidence is used in practice

We acknowledge an important research tradition that challenges the primacy of ‘evidence’ in the policymaking process

We seek to contribute to a scholarly inquiry about how people reason with evidence in real-world decision making

Policymaking

‘An authoritative exposition of values’ Judgements in the face of uncertainty ‘Wicked problems’

occur in an open system, solutions are not true or false but contingently better or worse

[cf tame problems occur in a closed system, have clear boundaries, and are capable of being resolved given sufficient resources]

3 levels of policymaking

First order: Should we implement programme X, programme Y or programme Z to fix problem P?

Second order: How should we decide which of these programmes to implement for problem P?

Third order: how should we decide whether problem P has priority over competing problem Q?

3 levels of policymaking

“Those who would want to promote evidence based health policy must be prepared to enter debates about meta-policy without the expectation that a higher court of scientific method will relieve the burdens of judgment. A major challenge of making meta-policy is to find a way of approaching it reasonably, and of integrating questions of values with questions of empirical evidence as separate-but-inseparable parts of policy making, not mutually exclusive foes”.

Lin & Gibson 2003Evidence Based Health Policy

Policymaking: an example

How should we deal with the obesity epidemic? Free slimming pills on demand? ‘Healthy eating’ labelling of foods? Free stomach stapling operations? Compulsory stomach stapling operations? Ban hot dog vans from outside schools? A ‘fat tax’ on crisps?

What priority does obesity have over other health problems?

THE RHETORICAL TRADITION

Four distinctive features1. The struggle over ideas2. How we deal with uncertainty3. The nature of rationality4. The centrality of audience

1. The struggle over ideas

“The essence of policymaking in political communities [is] the struggle over ideas. Ideas are at the centre of all political conflict... Each idea is an argument, or more accurately, a collection of arguments in favour of different ways of seeing the world.”

Deborah Stone 1990Policy Paradox

1. The struggle over ideas

“The focus on argumentation allows us to recognise the complex ways analysts not only solve but also formulate problems, the ways their arguments express or resist broader relations of power and belief, and the ways their practical arguments are inescapably normative and descriptive.”

Fisher and Forrester 1993The Argumentative Turn in

Policy Analysis and Planning

2. How we deal with uncertainty

Uncertainty is not just a problem of knowledge: “what do we know?”

It is also about practical, ethical judgements: “what should we do?”

2. How we deal with uncertainty

What can I know? What should I do? What dare I hope?

The science of evidence “what can we know?”

The science/artof politics andpolicymaking“what should

we do?”

2. How we deal with uncertainty

2. How we deal with uncertainty

“Recent philosophical trends…have resisted the scientific identification of rationality with procedure, the conflation of the logical with the reasonable.”

Carolyn Miller 1990In Simons H:

The Rhetorical Turn. Invention and Persuasion in the Conduct of Inquiry

3. The nature of rationality

a) ‘Provably true’ – mathematical logicb) ‘Probably true’ – Bayesian logicc) ‘Plausibly true’ – that which will (or once

did) persuade a reasonable audience

(a) and (b) are part of the scientistic tradition – rationality as a procedure for arriving at a correct answer

(c) is part of the rhetorical tradition – rationality as a process or activity that invites and responds to criticism

3. The nature of rationality

“To recognise that policy analysis has less to do with proof and computation than with the process of argument is to make contact with an old philosophical tradition that defines rationality not in instrumental terms but as the ability to provide acceptable reasons for one’s choices and actions.”

Majone 1989 Evidence, Argument and Persuasion in the Policy Process

3. The nature of rationality

“Hip protectors” (the language of risk management and randomised trials)

versus

“Padded knickers” (the language of human dignity and self determination)

Judith Green 2000Epistemology, Evidence and Experience

4. The centrality of audience

To persuade an audience, we need Logos – the argument itself Pathos – the appeal to emotion (including the

audience’s beliefs, values, knowledge and imagination)

Ethos – the credibility, legitimacy and authority a speaker brings and builds during the course of the argument

Aristotle Rhetoric

4. The centrality of audience

The ‘universal audience’: What are its taken-for-granted

assumptions? What are its common values? What will it accept as good reasons for

belief and action?

Perelman and Olbrechts-TytecaThe New Rhetoric

Research aims

To explore how policymakers introduce and reason with evidence in a local Primary Care Trust (PCT)

To analyse the negotiation of policy from an argumentation perspective

To feed back insights into the policymaking process with a view to improving its effectiveness

Wider context

BARNET

PRIMARY CARE TRUST

Priorities Forum

ProfessionalExecutiveCommittee

PCTBoard

Figure 1: Our case – the Priorities Forum – and its context

National policy drivers

Local priorities and constraints

Questions driving the study

How are policy problems ‘named and framed’? What rhetorical strategies are used to direct

attention to particular problems & outcomes? How is credibility and authority established? How are audiences constructed and arguments

addressed to a particular audience (or not)? What emerge as the key ethical questions

(‘what should we do’)? What is taken for granted as common sense? What are claimed as values and how are they

drawn upon? In the above, where does ‘evidence’ come in?

Higher-order research questions

What can we say about how language is used in the policymaking process to

prioritise and frame particular issues to particular audiences / stakeholders?

construct the ‘evidence base’ around these issues?

produce action around these issues? distribute social goods and privileges? link and co-ordinate social action?

Methodology: Discourse analysis

“Traditional qualitative approaches often assume a social world and then seek to understand the meaning of this world for participants. Discourse analysis, on the other hand, tries to explore how the socially produced ideas and objects that populate the world were created in the first place and how they are maintained and held in place over time.”

Phillips and Hardy 2002Discourse Analysis

The ‘building tasks’ of language

Language confers significance Language builds relationships Language enacts roles Language distributes social goods Language codifies and transmits knowledge Language privileges particular forms of

knowledge Actions happen through language

Gee 1999 An Introduction to Discourse Analysis

Discourse analysis – the method

Close reading of texts Focus on description and interpretation Explores general patterns in data using

analytic concepts from theoretical perspective

May include a narrative perspective Systematic investigation of patterns using

NVivo qualitative data analysis software

Wetherell et al 2001

Discourse as Data: A Guide for Analysis

What are our texts?

Transcripts of group discussions – especially at the Priorities Forum

One to one interviews How do the different players see their role in the

Priorities Forum and view decisions it comes to? Documentary texts (minutes, reports, etc.)

Who constructs these reports; how are problems framed and how is evidence brought in?

E-mail exchanges – especially around ‘urgent’ ethical decisions around funding expensive treatments

Issues discussed at the Priorities Forum

Should we appoint a specialist community nurse to improve epilepsy services?

Should we expand the cardiac rehabilitation service – and if so, to whom?

Should we appoint a consultant to help old people leave hospital (i.e. ease the ‘bed blocking’ problem)?

Should we fund IVF and if so, for whom and how many attempts?

Email exchanges: should we fund…

A 26 year old man to have an expensive brain tumour drug, when the research evidence shows that 20% of people respond well but the rest not at all?

A woman to have her bunion operation re-done in France?

Injectable heroin for an addict ‘intolerant’ of methadone (the usual heroin substitute given to addicts), plus the private GP fees?

‘Specialing’ a self harming 14 year old?

Summary

We are studying how people reason with evidence in real-world decision making

Our focus of inquiry is healthcare policymaking Our theoretical perspective is argumentation

and rhetoric Our methodological approach is discourse

analysis Our unit of analysis is the text, including the

process by which (and the context in which) it is generated

Our intended output is illumination

Thank you for your attention

Trish GreenhalghJill Russell

Where are we now?

Approximately halfway through a 3yr study Good relationships with the PCT and PF Large amounts of field data – mainly texts JR has completed OU course in discourse

analysis Data analysis has begun Plans to feed back findings to the Priorities

Forum in about September 06 We have been asked to help with

developing a north London Priorities Forum

2. How we deal with uncertainty

“.. the abundance of information tends to overshadow the phenomena to which information refers: the discussion about crime easily slips to debating crime rates and spending on police;..; the concern with the performance of hospitals leads to debating readmission rates and other indicators. In short, the more information we have about the world, the more we distance ourselves from what is going on and the less able we become in comprehending its full complexity.”

Haridimos Tsoukas 1996 The Tyranny of Light

2. How we deal with uncertainty

The more information we have, the more we miss the wood for the trees

The more information we have, the longer we spend processing it instead of taking action

The more information we have, the less we trust it

Haridimos Tsoukas 1996The Tyranny of Light