Change Blindness Images

Li-Qian Ma1, Kun Xu1, Tien-Tsin Wong2, Bi-Ye Jiang1, Shi-Min Hu1

1Tsinghua University2The Chinese University of Hong Kong

Spot-the-difference Game

Spot-the-difference Game

Motivation

• These image pairs are mainly generated by artists manually

• The degree of recognition difficulty is controlled by artists empirically

Goal

• Given an image, automatically generate a counterpart of the image

With a controlled degree of “difficulty”

Psychological background

• Change blindness–Widely studied in psychology

• is caused by failure to store visual information in our short-term memory

–Factors influencing• visual attention (saliency),• object presentation

–Mostly qualitative

The Metric

• We define a metric to measure the blindness of an image pair

• There is a single change between the image pair• The change region and the operator are known in advance• The change is limited to the following operators:

– Insertion/Deletion– Replacement– Relocation– Scaling– Rotation– Color-shift

The Metric

: the amount of changes

Amount of Change

𝐷=ω𝑐𝐷𝑐+𝜔𝑡𝐷𝑡+𝜔𝑠𝐷𝑠

Color Difference

Texture Difference

Spatial Difference

Saliency

• Visual attention is highly context-dependent• No existing saliency model attempts to

explicitly quantify background complexity

Context-Dependent Saliency

• Modulate saliency via spatially varying complexity

𝑆 ( 𝐼𝑘 )=𝑆0 ( 𝐼𝑘 )⋅𝐶 (𝐼𝑘)

Existing saliency model

Spatially varying complexity

Context-dependent saliency

Color Similarity

• Color similarity :

𝑒𝑖𝑗=exp (−𝐷𝑐

2 ( 𝐼 𝑖 , 𝐼 𝑗 )𝜎𝑒

2 )

𝐼 𝑖 𝐼 𝑗 𝐼 𝑖 𝐼 𝑗

Small color similarity Large color similarity

Spatial varying Complexity

• Weighted sum of color similarities between all region pairs around𝐶 ( 𝐼𝑘 )=∑

𝑖 , 𝑗𝜔 𝑖𝑗𝑒𝑖𝑗/∑

𝑖 , 𝑗𝜔𝑖𝑗

Spatial varying Complexity

𝑤𝑖𝑗=|𝐼𝑖||𝐼 𝑗|exp (− (𝑐𝑖−𝑐 𝑗 )2

𝜎𝑤2 )exp (− (𝑐 𝑖−𝑐𝑘 )2+(𝑐 𝑗−𝑐𝑘 )2

𝜎𝑤2 )

𝐼 𝑖 𝐼 𝑗

𝐶 ( 𝐼𝑘 )=∑𝑖 , 𝑗𝜔 𝑖𝑗𝑒𝑖𝑗/∑

𝑖 , 𝑗𝜔𝑖𝑗

𝐼𝑘

Context-Dependent Saliency

Input images Global contrast saliency

Spatial varying complexity

Context-dependent saliency

Context-Dependent Saliency

Input image Global contrast saliency Learning-based saliency Image signature

Itti model AIM saliency Judd model Context-Dependent Saliency

Synthesis

• Optional user manually refinement

Original Image

Desired Difficulty = 0.5

Synthesis

Original Image Changed Counterpart

Desired Difficulty = 0.5

1. Randomly pick a region and a change operator 2. Search in the parameter space of the change operator

Move

Measured Difficulty B =1 0.70.5

More Results

More Results

More Results

Original Image Changed Counterpart

Desired Difficulty = 0.2 Desired Difficulty = 0.5 Desired Difficulty = 0.8

More Results

User Study

• Generate 100 image pairs• 30 subjects• Pearson’s correlation: 0.74

User Study

Model Global contrast

Learning based

Image signature

Ittimodel

Correlation 0.44 0.38 0.34 0.42

Model Judd model

AIMmodel

Context-Dependent

Correlation 0.43 0.42 0.74

Conclusion

• Computational model for change blindness• Context-dependent saliency model• Change blindness image synthesis with

desired degree of blindness

Future Works

• Add high-level image features into the metric

• Improve the predictability using more sophisticated forms

• Improve the accuracy of the metric considering just-noticeable difference(JND)

Acknowledgement

• Anonymous TVCG reviewers

Thank you for your attention.