Vidna kognicija II

Danko Nikolić

Teme• Neurofiziološki kodovi prijenosa i obrade

informacija u vidnom sustavu • Dva kôda za percepciju svjetline • Problem povezivanja dijelova vidne scene u

cjelinu (tzv. binding problem) • Uloga pažnje u pohranjivanju informacija u radno

pamćenje • Uloga radnog pamćenja za formiranje dugoročnog

vidnog pamćenja • Mehanizmi sinestezijskih asocijacija

Visual cognition

• Memory for visual objects:

The modal model of memory

Sens

ory

mem

ory

(ico

nic

mem

ory)

Shor

t-te

rm m

emor

y (w

orki

ng m

emor

y)

Lon

g-te

rm m

emor

y

200-300 ms Several seconds or while rehearsing

Up to life long

Objects can have different familiarity...

Familiar Novel

… and different complexity.

ComplexSimple

Long-term memory

Short-term memory (Working memory)

Luck & Vogel, 1997

Wheeler & Treisman, 2002

Detection of visual objects requires attentionFeature binding theory

Pop-out

Lack of pop-out Pop-out

Stimulus size

Automatic processing

Searc

h t

ime

Focused attention

No po

p-ou

t

Two different processes:

Automatic Attentive

Short presentation

X T R P X

Short presentation

X T R P X

Short presentation

X T R P X

Luck & Vogel, 1997

Set size

50

Pe

r c e

nt

Co

rre

ct

2 4 6

75 SizeGap

ColourConjunction

100

Orientation

Conclusion: the capacity of visual WM is about four objects, while every object can consist of multiple features.

- WM similar to attention.

Study Test

Wheeler & Treisman, 2002

Study Test

Conclusions:

- Feature dimensions are independent.

- Only four features per feature dimension.

- Attention binds features within WM.

- Proof: With distractors memory for conjunctions impaired.

Performance drops for conjunctions but not for features.

G.A. Alvarez and P. Cavanagh The Capacity of Visual Short-Term Memory Is Set Both by Visual Information Load and by Number of Objects, Psychological Science, 2004

Henrik Olsson* and Leo Poom Visual memory needs categories, PNAS 2005

What about LTM?

• Hypotheses:– The same attentional mechanisms that bind objects for

perceptual purpose also store the binding information into LTM.

– WM plays central role in the formation of LTM.

– Therefore, formation of LTM for visual objects is limited in the capacity; more complex objects are stored by a serial process.

• More hypotheses:– The formation of LTM is limited by the capacity of

attention to bind features.

– The capacity of attention is equivalent to the capacity of WM to store bindings.

– Thus, changes in the capacity of attention, change the capacity of WM.

WMAttention

LTM

Stimuli

The paradigm

Strategy

Target

Distractor

Test arraySample array

Two memorization alternatives

WM ‘slots’ Chunking

Visual object

Manipulation of pop-out

• Why is lack of pop-out needed?– Subjects might chunk with pop-out

but not without pop-out.

– In this case, formation of LTM is not formed by the same attentional mechanisms that operate during visual search (lack of pop-out).

– Alternatively, there is no difference between pop-out and no pop-out in the ability to form LTM.

Experiment 1: The capacity of visual WM

• Short presentation time (1000 ms).• Four perceptual conditions.• Adaptive change in the array size.

– (correct response > increase by 1 element).

• 150 trials.• Starting from small array size of 7 elements.• 7 subjects.

Array growth in exp. 1

4.1 target locations

1.4 target locations

Probability to give a correct response: Pc Number of correctly stored elements: N Total number of elements: S The probability of giving the correct response by guessing: Pg

Probability of giving the correct response:

Pc = N /S + Pg (1-N /S).

Pg = 0.5

Related to model of Pashler (1988)

The expected change in array size in a single trial

E{ΔS} = E{Increase} + E{Decrease},

which leads to

E{ΔS} = 2 Pc – 1.

It follows that: E{ΔS} = N /S.

Conclusions from exp. 1

• WM capacity is narrowly limited.

• Without distractors WM capacity is not so limited.

• Thus, the reason is the presence of distractors.

• This is supported by the further decrease in the capacity without lack of pop-out.

• The capacity of WM depends on the binding capacity of visual attention:– With pop-out, about four objects.

– Without pop-out, fewer objects.

Shape II

Non pop-out Location

Shape

If bindings are created in both conditions, there will be no qualitative difference difference between pop-out and no pop-out conditions in the formation of LTM.

Pop-out Location

Shape

Experiment 2: Chunking

• unlimited presentation time.– dependent variable.

• 2 sessions, 50 trials each.• Fixed array sizes (10, 15, 20 and 25 elements).• 2 perceptual conditions (pop-out; no pop-out).• 6 subjects.

Results exp. 2

Encoding times, exp. 2

• Sequential (serial) encoding in both conditions.

• Formation of objects (chunks) is a capacity limited process.

• Similarly dependent on pop-out as the capacity of WM.

P

NP

NP

P

S

S

C

C

9.24.1

1.4

NP

P

C

C

8.21345

3781

P

NP

S

S

WM capacity predicts chunking speed

SP = 1345 ms/elem.

SNP = 3781 ms/elem.

Conclusions exp. 2

• The speed of object formation (chunking) depends on the capacity of WM.

• Thus, the contents of WM are integrated in parallel – one chunking step.

• The remaining elements are integrated by repeating the chunking steps.

Experiment 3: LTM

• Same as exp. 2 + unexpected LTM test.

• 10 chunking followed by 10 trials of LTM test.

• 2 conditions: Small and larger array (WM vs. chunk).

• 8 subjects (4 in each condition).

Figure

4

a b

Ac

cuar

cy

Sm all array Large array

0.5

0.6

0.7

0.8

0.9

1

W M LTM

Sm all arrayLarge array

Results exp. 3

Figure

4

a bA

ccu

arcy

Small array Large array

0.5

0.6

0.7

0.8

0.9

1

W M LTM

Sm all arrayLarge array

Conclusions

• When storing bindings, the capacity of WM depends on the binding capacity of visual attention (magic number 4).

• Subjects exceed the capacity of WM by storing visual objects (chunks) in LTM.

• No qualitative difference between pop-out and non pop-out conditions in the formation of LTM (always a sequential process; no change in strategy).

• The only difference is in the speed of the sequential process.• The differences in the speed can be explained by the capacity of visual

WM for the same stimuli.

• The working component of WM is visual attention.• WM and attention jointly store the binding information into LTM, enabling

thus storage of visual objects.

Figure 2. The procedure used in Experiment 1. Participants detected the target items and memorized the shapes surrounding them. The presentation time that was needed to achieve high WM performance was determined by the participants themselves. After an interval of 8 s participants had to judge whether the test shape matched one of the target shapes. ITI: Inter-trial interval.

Figure 3. Results from Experiment 1. A. Mean response accuracy at test as a function of WM load and attentional demand. B. Mean presentation times as a function of WM load and attentional demand (PO: pop-out; NPO: non pop-out). Vertical bars: the standard error of the mean.

Figure 4. The procedure used in Experiment 2. Participants detected and counted the target items. After pressing the response button a question mark appeared prompting the participants to enter the number of the counted targets. ITI: Inter-trial interval.

Figure 5. Results from Experiment 2. A. Mean response accuracy at test as a function of WM load and attentional demand. B. Mean counting times as a function of WM load and attentional demand (PO: pop-out; NPO: non pop-out). Vertical bars: the standard error of the mean.

Experiment 3: Information about the upcoming number of targets.

Figure 6. Results from Experiment 3 compared to the results from Experiment 1. A. Mean response accuracy at test as a function of WM load and attentional demand. B. Mean presentation times as a function of WM load and attentional demand (PO: pop-out; NPO: non pop-out). C. Differences in the presentation times between pop-out and non pop-out conditions across WM load conditions. Vertical bars: the standard error of the mean.

Figure 6. C. Differences in the presentation times between pop-out and non pop-out conditions across WM load conditions. Vertical bars: the standard error of the mean.

Figure 7. A. Empirically obtained offset in the presentation times produced by lack of pop-out in Experiment 3 and theoretically predicted offset based on search times from Experiment 2, computed for five different memory loads. B, Offset in the presentation times produced by lack of pop-out that is not explained by the visual search and that is expressed as a function of the number of target items. Dashed line: linear fit (see text).

Figure 8. The procedure used in Experiment 4. Participants detected the target items and memorized their locations only. After an interval of 8 s participants judged whether the location of the missing item in the test array matched one of the target locations. ITI: Inter-trial interval.

Figure 9. Results from Experiment 4. A. Mean response accuracy at test as a function of WM load and attentional demand. B. Mean presentation times as a function of WM load and attentional demand (PO: pop-out; NPO: non pop-out). Vertical bars: the standard error of the mean.

Experiment 5: same as experiment 4 but with knowing the upcoming number of targets (as in experiment 3).

Figure 10. Results from Experiment 5 compared to the results from Experiment 3. A. Mean presentation times as a function of WM load and attentional demand (PO: pop-out; NPO: non pop-out). B. Differences in the presentation times between pop-out and non pop-out conditions across WM load conditions. Vertical bars: the standard error of the mean. C. Offset in the presentation times produced by lack of pop-out that is not explained by the visual search and that is expressed as a function of the number of target items. Dashed lines: linear fit.

Figure 10. C. Offset in the presentation times produced by lack of pop-out that is not explained by the visual search and that is expressed as a function of the number of target items. Dashed lines: linear fit.

Conclusions

• WM and attention interfere and perhaps use the same resources.

• Memory for locations prevents interference.

Funkcionalna magnetska rezonanca

BOLD signal

Attentional Demand Influences Strategies for Encoding into Visual Working MemoryJutta S. Mayer1, Robert A. Bittner1, David E. J. Linden1, 2 and Danko Nikolić3, 4

(under review)

Information maintenance

0

3000

6000

9000

12000

15000

18000

1 2 3 4 5

Number of Targets

Pre

sent

atio

n T

ime

(ms)

NPO

PO

Kraj kognitivnog dijela

Mreže maloga svijeta

(small-world networks)

Small world networks

Stanley Milgram (1967)

Watts and Strogatz (1998)

A small world of neuronal synchrony

A small world network

• A small-world network partially shares properties with random networks, which have short average path lengths (i.e., any pair of nodes is likely to be connected through a small number of other nodes)

• … and partially also with regular networks, which are organized into clusters (i.e. a high level of local interconnectivity).

• A small-world property: the same average path lengths as random networks, λ ≈ 1, but empirical network should have a larger clustering coefficient than a random network, γ > 1.

Applying Ising model to estimate neuronal interactions

Bottom-up

Common input

Input is not shared

Lateral interactionsTangential connections

Top-downLower visual area Higher visual area

Feedback connections

Applying Ising model to estimate neuronal interactions

A small world of neuronal synchrony

Clustering coefficient