The Visual-Spatial System: Cognition & PerceptionCognition & Perception

D R . J A M E S L . M O H L E R (马健思 博士 )

C O M P U T E R G R A P H I C S T E C H N O L O G Y

P U R D U E U N I V E R S I T Y

The Visual-Spatial Systemp y

Visual Perception

Cognitive processes that receive input from “eyes” and interpret/construct what we are seeing; the

meaning making activity; visual intelligence

Physiology (Physical Mechanisms)

Spatial Cognition (Brain O l )

g g y; g

Mechanisms)

Eyes (as a mechanism)Retina (color & value input)

Only)

Spatial AbilityMental VisualizationRetina (color & value input)

Optic Nerve (“hardwire”)Brain (input receipt)

Mental VisualizationMental RelationsMental Orientation

Examples of Visual-Spatial Thinkingp p g

Vi l ti l thi ki ti iVisual-spatial thinking sometimes requires:Visual recognition or identification

P tt t hi ( D D)Pattern matching (2D or 3D)

Thinking and manipulating information (2D or 3D)

Mental representationsMental representations

Rotations or transformations

Orienting (or reorienting) yourselfOrienting (or reorienting) yourself

Spatial CognitionSpatial Cognition

Why is Spatial Ability Importanty p y p

S ti l bilit h b h t b iti l kill i Spatial ability has been shown to be a critical skill in many different fields:

A t/d i h i i iti /l tArt/design – when we imagine composition/layouts

Music – when we “hear” in our mind

Engineering when we create drawingsEngineering – when we create drawings

Chemistry – understanding chemical compositions

Biology – visualizing structuresBiology visualizing structures

Science – envisioning processes

Geology – mineral compositionsGeology mineral compositions

Medicine – understanding anatomical systems

And the list goes on…g

Spatial Ability is Important!p y p

S ti l f ti i i iti l t l t Spatial functioning is critical to almost every professional field

S h d t d th t it i i t t Some have advocated that it is as important as mathematics/logic and language

While seen as important there is still much we don’t While seen as important, there is still much we don t understand (and much that researchers don’t agree upon)upon)

However, researchers have generally agreed on the three primary factors that comprise spatial abilitythree primary factors that comprise spatial ability

Primary Spatial Ability FactorsPrimary Spatial Ability Factors

S P A T I A L O R I E N T A T I O N

S P A T I A L R E L A T I O N S

V I S U A L I Z A T I O N

Spatial Orientation (Lohman, 1979)p ( , 979)

Abilit t i i h ti l ill Ability to imagine how a stimulus array will appear from another perspective; there is often a left-right discriminationdiscrimination

Ability to comprehend the nature of the arrangement of elements within a visual stimulus pattern

i il i h h i ’ b dprimarily with respect to the examinee’s body as a frame of reference

Spatial Orientation Examplesp p

“W fi di ” di“Wayfinding”, map reading

Driving a car

Piloting an aircraft

Brochure design (folding/facing pages)

Spatial Relations (Lohman, 1979)p ( , 979)

Abilit t l h bl (t i ll t l Ability to solve such problems (typically mental rotations) quickly, by whatever means.

Speed in manipulating relatively simple visual patterns by whatever means (rotation, transformation, or otherwise).

Spatial Relations Examplesp p

I bj t i i d Sk t h l f Image an object in your mind. Sketch an example of what the top of the object would look like.

Which of the following two objects match the stimulus object?

Spatial Visualization (Lohman, 1979)p ( , 979)

Abilit t l l ti l fi l t tAbility to solve complex spatial-figural content

Ability in manipulating visual patterns, as indicated by level of difficulty and complexity in visual stimulus material that can be handled successful, without regard to the speed of task solution.

Minor Spatial Ability FactorsMinor Spatial Ability Factors

C L O S U R E

O C O SF L E X I B I L I T Y O F C L O S U R E

D Y N A M I C S P A T I A L A B I L I T Y

Minor Factors (Lohman, 1979)( , 979)

Cl S dClosure SpeedSpeed in apprehending and identifying a visual pattern without knowing in advance what the pattern is when the without knowing in advance what the pattern is, when the pattern is disguised or obscured in some way.

Flexibility of ClosureFlexibility of ClosureSpeed in finding, apprehending, and identifying a visual pattern, knowing in advance what is to be apprehended, when the pattern is disguised or obscured in some way.

Dynamic spatial abilitythe ability to deal with moving elements and relative motion (arrival and intercept)

i f i l biliHistory of Spatial Ability ResearchResearch

P I O N E E R I N G E R A ( 1 9 0 0 - 1 9 4 1 )

D E F I N I N G E R A ( 1 9 4 1 - 1 9 6 5 )

P A P E R A N D P E N C I L E R A ( 1 9 6 5 - 1 9 8 9 )

T E C H N O L O G Y E R A ( 1 9 8 9 - )

Pioneering Erag

Spatial ability and its role in intelligenceSpatial ability and its role in intelligenceThorndike (1921)

Ab t t h i l d i l i t lliAbstract, mechanical and social intelligence

Kelly (1928)d i h i d fCrossroads in the mind of man

El Kloussy (1935)Found evidence for the existence of a factor “K”

Thurstone (1938)Defined a “space” factor that represented the ability to operate mentally on spatial or visual images.

Defining Erag

Th t ( )Thurstone (1950)S1, recognize an object if viewed from different orientations or angles (rotations/relations)angles (rotations/relations)

S2, recognize the parts of an object if they were moving or are displaced from their original position (visualization)p g p ( )

S3, which is the ability to use one’s body orientation to relate to questions regarding spatial orientation (orientation)

Descriptive terms added later

Paper and Pencil Erap

Focus on subfactorsFocus on subfactorsGenderEnvironmentalEnvironmental

BiologicalCulturalSocialEducational

AgeAgeHemisphere specializationSpeed and efficiency

Technology Eragy

F th f t t h l d it Focus on the use of computer technology and its impact on spatial ability

lExamples:2D and 3D environments

i iAnimation

VR technologies

C t Computer games

Future?

R f i i t lliRefocusing on intelligenceA return to spatial ability’s impact and contribution to intelligenceintelligence

Greater use of technologyUse of emerging technologies (VR real time media et cetera)Use of emerging technologies (VR, real-time media, et. cetera)

How do these new technologies help or hinder spatial ability trainingtraining

Types of SA ResearchTypes of SA Research

Types of SA Researchyp

P h t iPsychometric

Differential

Developmental

Information Processingg

BreakBreak

What is Visual Perception?What is Visual Perception?

I T I S W H A T ( A N D H O W ) W E C O N S T R U C T …

Do we really construct thi ?everything we see?

Is This Possible?

Possible?

Possible?

Difficult to focus?

Visual Intelligenceg

O i l t f ti t f l th t Our visual system functions upon a set of rules that are independent of gender, culture, race, or any other variable that differentiates human beingsother variable that differentiates human beings.

These rules apply to form, color and motion

Constructing FormConstructing Form

What you see is what you constructy y

We know this image is flat but most people perceive We know this image is flat, but most people perceive 3D…why?

And…Wh i b B h i b A?Where is cube B when you are seeing cube A?Where is cube A when you are seeing cube B?

Necker and Kopfermannp

F d t l bl f i d th I t th Fundamental problem of seeing depth. Image at the eye is 2D; therefore it has countless interpretations in 3Din 3D.

You construct visual worlds from ambiguous images i f t i l lin conformance to visual rules.

Symmetryy y

S t ft l d t D t ti Symmetry often leads one to see a 2D representation but the rule of generic views can override this.

bj ?Do you see a 2D or 3D object?

The answer reveals which is dominant for you.

Proximityy

P i it l l i i t t ti f Proximity can play a role in our interpretation of imagery

Rule 4: interpret elements nearby in an image as Rule 4: interpret elements nearby in an image as nearby in 3D

Constructing ShapeConstructing Shape

Contrasting Values?g

K i ’ bj ti t i l th t t i l Kanizsa’s subjective triangles; the center triangle appears “whiter” than the surrounding area.

Contrasting Values?g

Eh t i ’ di k th t di k “ hit ” Ehrenstein’s disks; the center disk appears “whiter” or “blacker” than the surrounding area.

Cuspsp

Ab t h i l ( ) t Abrupt changes in slope (cusps) cause us to see subjective figures

Nonaccidental Relations

O th l ft d bl k t i hit On the left, do you see black stripes on a white background?

O h i h d hi i bl k On the right, do you see white stripes on a black background?

Nonaccidental Relations

If three or more curves intersect at a common point in an image, interpret them as intersecting at a common point in space.

Constructing ColorConstructing Color

Color Context

The context of a color greatly impacts your perception of The context of a color greatly impacts your perception of itNote that color 1 and 2 are the same huesNote that color 1 and 2 are the same hues

Adjacent color impacts perceptionj p p p

Simultaneous Contrast

Y i i t t t l b d Your vision system constructs values based upon surrounding values (it does the same with hue).

i d i h i h di k l k d kViewed in context, the right gray disk looks darker…

But is it?

Simultaneous Contrast

A th b th l ?Are the gray bars the same value?

Are you sure?

Corrugated Mondriang

A th th ? (Y b bl Are the gray squares the same gray? (You probably will not say what you see…)

Corrugated Mondriang

A th th ? (Y b bl Are the gray squares the same gray? (You probably will not say what you see…)

Construction is NOT in Isolation

S f l d t t t d i Surface colors and grays are not constructed in isolation.

l d d i h fColors and grays are constructed in the context of a mutually consistent coordinated construction of

f h f l li h d surface shapes, surface colors, light sources and transparent filters in as simple a manner as

iblpossible.

Transparent Filtersp

T d Ad l Th lTed Adelson – The argyle

Diamond on left looks lighter; example of a “filter”

Separate the diamonds reduces the effect

Why do we use these rules?y

W iti i t i kl We are cognitive misers – we must process quickly and we want to do so efficiently

k f diff b hFour markers of different subprocesses that synergize to construct the most specific and

i i l h i iblcognitively “cheapest” image possible.Light source

Luminance (comparative to surroundings)

Filters

I hImage shape

Constructing MotionConstructing Motion

Rules of Motion

Y t t ti j t lik d thi You construct motion, just like you do everything else you see

A / / illi d i i i dAt 200/50/200 milliseconds – motion is perceived

The delay speed affects what you construct

Rules of Motion

This delay is called the interstimulus interval (ISI) and the ISI controls what you construct.

Sl > D t Slower => Dot movesFaster => Two dots flashSpeed required depends on spacing and sizep q p p g

Rules of Motion

If l th d f th fl hi d t till If we slow the speed of the flashing dots, you still see movement (200/400/200)

Rules of Motion

If i th d f th fl hi d t If we increase the speed of the flashing dots, you see two flashing dots (10/0/10)

Rules of Motion

If th d i t d i dd d ( h If the speed is correct and a curve is added (each one flashing in sequence), you perceive the movement of the dot as an arced path (130/130/130/130)the dot as an arced path (130/130/130/130).

Rules of Motion

Si il l th dditi f b t ti Similarly, the addition of an obstruction causes you to see the dot around the obstruction (130/130/130/130)(130/130/130/130)

Ternus Configurationg

With thi fi ti i l t With this configuration, your visual system causes the center dot to remain stationary, while the outer dot is seen as moving around it (130/130/130/130)dot is seen as moving around it (130/130/130/130).

Ternus Groupp

With thi fi ti b th d t (th i d With this configuration, both dots (the inner and outer) appear to be rotating on elliptical paths.

Von Schiller Motion

Y i l t k t bj ti d t j Your visual system makes two subjective dots jump back and forth together, sometimes side to side and sometimes up and downsometimes up and down.

Von Schiller Cont.’d

Y i l t k t bj ti d t j Your visual system makes two subjective dots jump back and forth together, sometimes side to side and sometimes up and down Notice that all the pairs sometimes up and down. Notice that all the pairs switch together.

Rotary Crossy

Y i i t ill l t t tYour vision system will also create rotary movement.

Here you can see the cross go either clockwise, or l k icounterclockwise

Rotary Crossy

Wh dd t f t th i ti When we add an extra frame to the animation, you see the cross with inertia and only clockwise

i i f i li i d Your vision system focuses on simplicity and consistency.

Rotational Paths

B d V th h d th t ISI ff t h Brown and Voth showed that ISIs affect how you perceive paths.

h h S i l i h bj When the ISI is slower, you perceive the object moving from corner to corner.

Rotational Paths Cont.’d

Wh th ISI i f t i th bj t When the ISI is faster, you perceive the object moving in a circular pattern.

Rotational Paths Cont.’d

Gi th t thi t i ti Given these two things, we can create an animation of something that is not there.

h d i hi i i ?What do you see in this animation?

Rotational Paths Cont.’d

Gi th t thi t i ti Given these two things, we can create an animation of something that is not there.

h d i hi i i ?What do you see in this animation?

And in this one?

Rigid Motiong

If ibl d if th l it i t t i If possible, and if other rules permit, interpret image motions as projections of rigid motions in three dimensionsdimensions.

Rigid Motiong

Wh t i th i th l ft?What is the image on the left?

What is the image in the middle?

What is the image on the right?

The Visual-Spatial System: Cognition & PerceptionCognition & Perception

D R . J A M E S L . M O H L E R (马健思 博士 )

C O M P U T E R G R A P H I C S T E C H N O L O G Y

P U R D U E U N I V E R S I T Y