From perception to action: From perception to action: an economic model of brain an economic model of brain

processesprocesses

Isabelle BrocasIsabelle BrocasUSC and CEPRUSC and CEPR

Juan D. CarrilloJuan D. CarrilloUSC and CEPRUSC and CEPR

Theoretical REsearch in Neuroeconomic Decision-making(www.neuroeconomictheory.org)

What is “Neuroeconomic What is “Neuroeconomic Theory”?Theory”?

Use evidence from neuroscience to revisit economic theories of decision-making

Neuroscience evidence includes:

Multiple systems in the brain

Interactions between systems

Physiological constraints

What is “Neuroeconomic What is “Neuroeconomic Theory”?Theory”?

Revisiting theories of decision-making includes:

• Revisit the individual decision-making paradigm: not decision theory but game-theory approach

• Provide “micro-microfoundations” for preferences, i.e. elements traditionally considered as exogenous (discounting, risk-aversion, etc.)

• Provide foundations for processes traditionally taken for granted (learning, information processing, etc.)

• Understand intra-personal conflicts and behavioral “biases”

This paperThis paper

Build a brain-based model of information processing using evidence from

Neurobiology:

i. Neurons carry information from sensory circuitry to decision-making circuitry through a cell firing process

ii. There is stochastic variability in neuronal cell firingiii. Thresholds of neuronal activity trigger actions (economical

information processing technology)iv. Thresholds can be modified (learning, adaptation)

A “simple” problem of information processing!

An illustrationAn illustration

• Today is a dangerous day (state A) or a safe day (state B)• Individual collects some imperfect information before making decision:

he takes a look out of the cave• Individual decides to stay in the cave (action a) or go hunting (action b)• Individual obtains a payoff (catches animal, killed by predator, starves)

sensory system decision system motor system

OVERVIEW

• A two-actions model• More complex environments• Behavior implications• Relation to neuroscience theories• Conclusions

A two-actions modelA two-actions model

(most interesting for neuroeconomics, least interesting for economic theory)

There are three main elements: environment, preferences, information

1. Environment.- Set of states of the world: S={A,B} with element s- Set of possible actions: ={a,b} with element

2. Preferences over outcomes.

Represented by loss/utility function l(|s).Payoff maximized if a in A and if b in B.

The complete description is: L = {l(a|A),l(b|A),l(a|B),l(b|B)}

The sensory systemThe sensory system

3. Information structure. It consists of:

- Prior belief : assessment of likelihood of states without information (memories, encoded values in cells). Pr(A) = p0 and Pr(B) = 1 - p0

- Information from outside world: encoded and translated into neuronal activity c [0,1] which is an indicator of the state from c = 0 (no perceived danger) to c = 1 (highest perceived danger)

There is stochastic variability in neuronal cell firing (cell activity varies, competition between neurons, metabolic costs, noise, circumstances)

fB(c) fA(c)

Pr(c)

0 1/2 1 c

(stochastic) lowcell firing if S=B

(stochastic) highcell firing if S=A

Example: c = proportion of neurons detecting a danger c is high in “dangerous” days, c is

low in “safe” days

(MLRP) 0)(

)( with )()|Pr( and )()|Pr(

cf

cf

dc

dcfBccfAc

A

BBA

(symmetry) )1()( cfcf BA

Perception is correlated with true state

The decision systemThe decision system

A decision is a mapping from (L, p0, c) into

Objective of the brain: conditional on L and p0, determine under which

circumstances c should trigger action a or action b.

Premise of the model: process must be economical and compatible with

existing evidence of neuronal functioning.

• Changes in beliefs and magnitude of payoffs are correlated with activity and choice support for maximization of expected payoff

• Neurons perform statistical inferences closely based on information received support for Bayesian updating

• Neuronal activity: neuronal thresholds (high or low), synaptic connections (weak or strong). Information is interpreted to trigger decision support for decision-threshold, i.e. a mechanism like threshold x such that one action if c < x and another action if c > x

Note. This process is “economical” (some information is filtered out)

• Thresholds are modulated support for (as if) optimization process

(see paper for discussion of neuroscience and neurobiology literature)

action

{a,b} payoff

l (|S)

nature

S {A,B}

prior p0

threshold

x cell firing

c x><

updated p1

Two-alternative task (6)Two-alternative task (6)Summary of timingSummary of timing

Assumption: decision threshold is set optimally

Decision threshold x maximizes expected payoff given Bayesian updating and neuronal constraints (stochastic variability + inability to process all information)

• At stage 2, the posterior is pH1 Pr(A | p0, c > x) or pL

1 Pr(A | p0, c < x)

• Given Bayesian updating, it is immediate that for any x : pH

1 (x) > p0 > pL1 (x)

that is, c > x is evidence of state A and c < x is evidence of state B

• Therefore, optimal threshold x* such that = a if c > x and = b if c < x. It solves:

max V(x) = Pr(c > x )U(a, pH1(x)) + Pr(c < x )U(b, pL

1(x))

• Let l(a|A) = A, l(b|B) = B, l(a|B) = l(b|A) = 0, and = A / B

Optimal thresholdOptimal threshold

Proposition 1. dx*/dp0 < 0 and dx*/d < 0

dx*/dp0 < 0

Tradeoff between likelihood and impact of informationSuppose that prior strongly favors state B (p0 is “small”):

Only strong evidence of A convinces subject to choose = a. Strong evidence means a high threshold must be surpassed. A high threshold means a low prob. (whether true state is A or

B).(there is an analogous result in Theory of Organizations

literature)dx*/d < 0

Tradeoff between likelihood and impact of mistakesSuppose that correct choice in B has higher potential (B is large):

Only strong evidence of A convinces subject to choose = a.… same logic as before

Property of the optimal Property of the optimal thresholdthreshold

Let pJ(c) = Pr(A|c). The optimal threshold x* is such that:

U(a, pJ(x*)) = U(b, pJ(x*))

Therefore: U(a, pJ(c)) > U(b, pJ(c)) for all c > x*

U(a, pJ(c)) < U(b, pJ(c)) for all c < x*

For the purpose of choice, it is equivalent to observe exact c

or only whether c x*

Threshold is economical and fully efficient when there are only

two possible actions (not robust to more complex situations)

><

More complex environmentsMore complex environments

(robustness analysis)

DefinitionsDefinitions

Note: A decision threshold discriminates between two actions.

We make a distinction between

• Cognitive processes: discriminates between all relevant actions (involves several thresholds if more than two actions)

• Affective processes : neglects some relevant actions (involves less thresholds than necessary)

Continuum of actionsContinuum of actionsLoss function l(s-1) if S=A and l(s) if S=BExample: A = danger, B = no danger

Choice = hunt at distance 1-s from the cave

a. l(z) linear or convex. Optimal solution is corner: * = 0 or * =1 identical analysis as before

b. l(z) quadratic. Departures are increasingly costly. There is one optimal action *(p1 ) for each posterior.

Cognitive process: requires a continuum of thresholds. Affective process setting one threshold: x affects posteriors

(pH1, pL

1). Optimal x* minimizes expected error.

Proposition 2. Under regularity conditions: (i) x* has same qualitative properties: dx*/dp0 < 0 and dx*/d < 0

(ii) There is a utility loss of not observing the exact cell firing c.

Dynamic information Dynamic information acquisitionacquisition

Individual takes a second look before selecting an action: Neuronal activation threshold x

Brain learns if x is surpassed or not. Beliefs updated to p1

Neuronal activation threshold yBrain learns if y is surpassed or not. Beliefs updated to p2

Proposition 3. Under regularity conditions, beliefs are again more likely to be reinforced. For linear and symmetric densities functions:

y*(p) < x*(p) <1/2 if p > 1/21/2 < x*(p) < y*(p) if p < 1/2

Intuition. At stage 1, information acquisition is more important than

knowing if p1 is

greater or smaller than 1/2. Thus, weaker 1st period modulation Snowball effect: threshold modulation exacerbated in dynamic

settings There is a utility loss of not observing the exact cell firing c.

Continuum of statesContinuum of states

State S [0,1] with Pr(S) = pS

2 actions, {a,b}Payoff is l(1 - S) if s=a and l(- S) if s=bExample: S = intensity of danger

a = stay , b = hunt(assume continuous version of MLRP)

Proposition 4. Same qualitative properties as before: if weight on dangerous (high) states increases, threshold decreases; undesirable outcomes are likely to be avoided.

Conclusions regarding threshold modulation are robust to:

• Dynamic choices• Increased action space• Increased state space

However, there is a utility loss in using this economical mechanism

Behavioral implicationsBehavioral implications

Existing beliefs are more likely to be endorsed and less likely to be refuted than opposite beliefs.

If state A becomes more likely (p0 increases), threshold x* decreases, so more likely to be surpassed.

The sequence in which signals are received affects beliefs and actions

Rational Stubbornness:- Why people develop habits that are difficult to change- Why individual are less likely to change their mind with age

Implication 1. Belief anchoring and first impressions

Individuals with different priors who are exposed to the same evidence and are subject to the same cell firing may update beliefs in opposite directions

Suppose pi0 > pj

0. By proposition 1, xi*< xj*. If c [xi*, xj*], then pi

1 > pi0 and pj

1 < pj0.

The individual with stronger conviction that predators are present will interpret a mixed signal as evidence of danger whereas the other will interpret the same signal as evidence of no danger.

Implication 2. Polarization of opinions

Individuals with identical priors, exposed to the same evidence, and subject to the same cell firing may end up with different posteriors due exclusively to differences in their marginal preferences over outcomes

Suppose iA / i

B > jA / j

B. By proposition 1, xi*< xj*. If c [xi*, xj*], then (i*=a, j*=b) and pi

1 > p0 > pj1

The individual with higher utility of catching a prey may go hunting while the other stays in the cave. In that case, the former will also believe that there is less danger than the latter.

Implication 3. Preferences shape beliefs

• An optimal decision threshold mechanism generates payoff dependent posterior beliefs.

• The individual is best represented as one entity with utility function

∑ Ps(πA, πB) πsl(γ|s)

Implication 4. Probability functions

• The number of relevant actions determine the number of thresholds necessary to achieve efficient decision making

• Affective and cognitive channels:- 3 states: S {A,B,C}, and 3 actions: {a,b,c} - Payoff 1 if a when A or if b when B or if c when CAffective process: sets one threshold xCognitive process: sets two thresholds x1 and x2

The affective channel optimally discriminates between the two actions that are most likely to occur and ignores the third one. The loss is therefore smallest when one state is highly unlikely

Implication 5. Elimination

Relation to neuroscience Relation to neuroscience theoriestheories

a. “Pre-existing somatic states influence the threshold of neuronal cell firing [...] Pre-existing positive states reinforce positive states, but they may impede negative ones” (Bechara-Damasio, GEB 2005)

Emotions regulate neuronal activity by affecting thresholds in a precise way: existing beliefs are likely to be supported

b. “These somatic states are indeed beneficial, because they consciously or non-consciously bias the decision in an advantageous manner”

This threshold modulation improves decision-makingSMH provides no explanation why such a “bias” is “advantageous”

Our paper formalizes and proves their claim: if emotions are responsible

for threshold modulation in that particular way, then it is true that

emotions help decision-making.

The Somatic Marker Hypothesis (Damasio ’94)

• The function of the PFC is to override automatic responses to The function of the PFC is to override automatic responses to shape behaviorshape behavior on the basis of plans or intentions (e.g. take on the basis of plans or intentions (e.g. take best action given preferences and information)best action given preferences and information)

• The flow of neural activity is The flow of neural activity is guided along pathwaysguided along pathways that that establish establish correct mappingscorrect mappings between inputs, internal states between inputs, internal states and eventual actions (e.g. represented by decision threshold)and eventual actions (e.g. represented by decision threshold)

• Some Some features are retainedfeatures are retained, others are neglected (e.g. , others are neglected (e.g. threshold acts as a filter of information). threshold acts as a filter of information).

Cognitive control (Miller and Cohen ’01)

ConclusionsConclusions

• A theory of the brain as a (constrained) optimal processor of information (needless to say, it is an “as if” approach).

• The theory relates the ability to make correct choices to primitives. It determines why and when this economical process implies an inefficiency.

• The theory has behavioral implications:- The existence of a confirmatory bias and belief anchoring

mechanism- The possibility to generate polarization of opinions- The active role of preferences on formation of beliefs (agree to

disagree)- The relationship to non expected utility theory- The fact that options in large choice sets are eliminated

• Implications of the theory are consistent with findings in neuroscience. Some testable predictions can help design new experiments.