Scene Completion Using Millions of Photographs • James Hays & Alexei A. Efros • Carnegie Mellon University • ACM Transactions on Graphics (SIGGRAPH 2007) • http://graphics.cs.cmu.edu/projects/scene-completion/
Graph Cut
合併後的圖⽚片 圖⽚片接合處 圖⽚片1 圖⽚片2
Vivek Kwatra, Arno Schodl, Irfan Essa, Greg Turk and Aaron Bobick. Graphcut Textures: Image and Video Synthesis Using Graph Cuts.
Graph Cut cost(s, t, A,B) = |A(s)�B(s)|+ |A(t)�B(t)|
A(s)
A(t)
B(t)
B(s)
A(s)
A(t)
B(t)
B(s)
cut
overlap
Graph Cut
cost =X
p
Cd(p, L(p)) +X
p,q
Ci(p, q, L(p), L(q))
L(p) =
(patch
exist
hole
exist patch
(Cd(p, patch) = (k ⇥Dist(p, hole))
3
Cd(p, exist) = 1 if p in hole
Graph Cut
cost(s, t, A,B) = |A(s)�B(s)|+ |A(t)�B(t)|原本的 graph-cut:融合失敗
轉換到 gradient domain:融合成功
圖⽚片1 圖⽚片2
cost(s, t, A,B) = |rA(s, t)�rB(s, t)|
Poisson Blending
• v : Gradient of a region in an image • g : Selected region of source • f* : Known pixel values in domain S • f : Unknown pixel values in domain Ω • Ω : Region g placed on domain S • ∂Ω : Boundaries
min
ZZ
⌦|rf � v|2 with f |@⌦ = f⇤|@⌦
Poisson Blending : 1D Example
min((f2 � f1 � 1)2 + (f3 � f2 � (�1))2 + (f4 � f3 � 2)2
+ (f5 � f4 � (�1))2 + (f6 � f5 � (�1))2)
min(2f22 � 2f2f3 � 16f2 + 2f2
3 + 6f3 � 2f3f4 + 2f24 � 2f4f5 � 6f4 + 2f2
5 � 2f5 + 59)
with f1 = 6, f6 = 1
min
ZZ
⌦|rf � v|2 with f |@⌦ = f⇤|@⌦
Poisson Blending : 1D Example dQ
df2= 4f2 � 2f3 � 16
dQ
df3= �2f2 + 4f3 � 2f4 + 6
dQ
df3= �2f3 + 4f4 � 2f5 � 6
dQ
df3= �2f4 + 4f5 � 2
8>>><
>>>:
f2 = 6
f3 = 4
f4 = 5
f5 = 3
2
664
4 �2 0 0�2 �4 �2 00 �2 4 �20 0 �2 4
3
775
2
664
f2f3f4f5
3
775 =
2
664
16�662
3
775
source target
近期相關研究 • Deep Learning
• Supervised Semantics-Preserving Deep Hashing for Large-Scale Image Search
http://arxiv.org/pdf/1507.00101.pdf
全景圖產⽣生器
Automatic Generation of An Infinite Panorama
http://cybertron.cg.tu-berlin.de/pdci09/infinite_panorama/ http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/f07/proj_final/www/lisachan/
透視圖產⽣生器
http://parosky.net/projects/undresser.html http://parosky.net/assets/data/anoron2013w.pdf
画像補完技術による衣服の除去
延伸閱讀 • Scene Completion
• James Hays & Alexei A. Efros. Scene Completion Using Millions of Photographs.
• GIST Descriptor • Aude Oliva & Antonio Torralba. Modeling the Shape of the Scene: A
Holistic Representation of the Spatial Envelope. • Aude Oliva & Antonio Torralba. Building the gist of a scene: the role of
global image features in recognition. • Graph Cut
• Vivek Kwatra, Arno Schodl, Irfan Essa, Greg Turk and Aaron Bobick. Graphcut Textures: Image and Video Synthesis Using Graph Cuts.
• Poisson Blending • Patrick Perez, Michel Gangnet and Andrew Blake. Poisson Image
Editing. • http://eric-yuan.me/poisson-blending
圖⽚片來源 • James Hays & Alexei A. Efros. Scene Completion Using Millions of Photographs.
• Aude Oliva & Antonio Torralba. Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope.
• Aude Oliva & Antonio Torralba. Building the gist of a scene: the role of global image features in recognition.