of 41 /41
Camouflaging an Object from Many Viewpoints Andrew Owens 1 , Connelly Barnes 2 , Alex Flint 3 , Hanumant Singh 4 , and William Freeman 1 1 MIT CSAIL, 2 University of Virginia / Adobe, 3 Flyby Media, 4 Woods Hole Oceanographic Inst Presenter: @miyabiarts 関関 CV 関関関 CVPR 関関関関 26, July 2014

Camouflaging an object from many viewpoints

Embed Size (px)

DESCRIPTION

関東CV勉強会

Text of Camouflaging an object from many viewpoints

Page 1: Camouflaging an object from many viewpoints

Camouflaging an Object from Many Viewpoints

Andrew Owens1, Connelly Barnes2, Alex Flint3, Hanumant Singh4, and William Freeman1

1MIT CSAIL, 2University of Virginia / Adobe, 3Flyby Media, 4Woods Hole Oceanographic Inst

Presenter: @miyabiarts

関東 CV 勉強会 CVPR 論文紹介 ( 26, July 2014 )

Page 2: Camouflaging an object from many viewpoints

Camouflage

Disruptive ColorationMasquerade

Optical CamouflageBackground matching

Page 3: Camouflaging an object from many viewpoints

Motivation• 単一の視点に対して隠れることは比較的容易• 複数の視点は困難

Page 4: Camouflaging an object from many viewpoints

Goal• 複数視点から 3D オブジェクトをカモフラージュ

Page 5: Camouflaging an object from many viewpoints

Application: Public Art

http://joshuacallaghan.com/Tianmu.htm

Page 6: Camouflaging an object from many viewpoints

Significance of camouflage research• カモフラージュとは物体検出の逆問題

• 物体検出:境界や注目領域に基づき検出• カモフラージュ:境界や注目領域を隠す

• 検出困難な物体とは何か?• より高精度な物体検出への知見を得る

Page 7: Camouflaging an object from many viewpoints

CV Dazzle

http://cvdazzle.com/

Page 8: Camouflaging an object from many viewpoints

Goal• 複数視点から 3D オブジェクトをカモフラージュ

Page 9: Camouflaging an object from many viewpoints

Contribution• 3D オブジェクトのためのカモフラージュアルゴリズムの提案• 37 シーンへの適用と比較実験• 知覚実験によるカモフラージュ手法の比較

Page 10: Camouflaging an object from many viewpoints

Algorithm• Background matching

• Two Stage Algorithm• Stage1: Capture images of object• Stage2: Camouflage object

Page 11: Camouflaging an object from many viewpoints

Stage 1: Capture Images of Object• 複数視点( 10-25 )から撮影

• カモフラージュ対象の物体(背景・遮蔽なし)• 背景

Page 12: Camouflaging an object from many viewpoints

Stage 1: Capture Images of Object• 撮影した物体を SfM で 3D モデル化

• テクスチャは除去• 背景に対するカメラ位置・姿勢を推定• Target: シンプルなものに限定

• 直方体• 影やライティングの影響を受けない

Page 13: Camouflaging an object from many viewpoints

Stage2: Camouflage Object• 複数視点の画像からテクスチャを生成

Page 14: Camouflaging an object from many viewpoints

Methods• Naïve model

• mean color• Projection from viewpoint

• Random/Greedy• MRF (Proposed)

• Interior/Boundary MRF

Page 15: Camouflaging an object from many viewpoints

Naïve model: mean color• 視点ごとに背景をテクスチャ上に射影• テクセルの値を平均化

Viewpoint 1 Viewpoint 2

Page 16: Camouflaging an object from many viewpoints

Naïve model: mean color• 全体的にぼける• 目立った領域となりやすい

Viewpoint 1 Viewpoint 2

Page 17: Camouflaging an object from many viewpoints

Projection from viewpoint• 視点ごとに見えている面に射影• 全ての面が埋まるまで繰り返す

Viewpoint 1 Viewpoint 2

Page 18: Camouflaging an object from many viewpoints

Random/Greedy• Random

• ランダムに視点の順番を選択• Greedy

Page 19: Camouflaging an object from many viewpoints

Random• ある視点からはうまく隠される• 別の視点からはおかしなものとなる

Viewpoint 1 Viewpoint 2

Page 20: Camouflaging an object from many viewpoints

Random• テクスチャの伸縮・歪みが大きい• 不連続な境界が発生

Viewpoint 1 Viewpoint 2

Page 21: Camouflaging an object from many viewpoints

Random/Greedy• Random

• ランダムに視点の順番を選択• Greedy

• 面に対して正対する角度以下の視点を採用• 本論文では 70° 以下

Page 22: Camouflaging an object from many viewpoints

Greedy• テクスチャの歪みが比較的小さい

Viewpoint 1 Viewpoint 2

Page 23: Camouflaging an object from many viewpoints

Greedy• 不連続な境界が発生

Viewpoint 1 Viewpoint 2

Page 24: Camouflaging an object from many viewpoints

MRF (Proposed)• エネルギー最小化によりテクスチャを生成

• 遮蔽・視点の安定性・内部の連続性をエネルギー化• テクスチャ座標上にグリッドを定義

Page 25: Camouflaging an object from many viewpoints

MRF Formulation

Data Smoothing

Page 26: Camouflaging an object from many viewpoints

MRF (Proposed)• Interior MRF

• 内部の連続性を重視• Boundary MRF

• 面ごとの連続性を重視• データコストは共通

Page 27: Camouflaging an object from many viewpoints

Data cost term

Data

Occlusion Viewpoint Stability

Page 28: Camouflaging an object from many viewpoints

Data cost term

Occlusion Viewpoint Stability

Page 29: Camouflaging an object from many viewpoints

Interior MRF• Smoothing cost term

Page 30: Camouflaging an object from many viewpoints

Interior MRF• 内部のテクスチャが連続• 全ての視点から平均的によく隠れる

Viewpoint 1 Viewpoint 2

Page 31: Camouflaging an object from many viewpoints

Boundary MRF• 面ごとに同じラベルを割り当て

• ある視点から見たときの完成度を重視• Projection に近い結果となる

Page 32: Camouflaging an object from many viewpoints

Boundary MRF• 面内部のテクスチャの伸縮・歪みが少ない• 面の境界で不連続なテクスチャが発生

Viewpoint 1 Viewpoint 2

Page 33: Camouflaging an object from many viewpoints

Psychophysical study design• Amazon Mechanical Turk

• ディスプレイにカモフラージュされた物体を表示• 物体が存在するかどうか?• 画像内のどこにいるか?

• Work• http://camo-exp.appspot.com/game

Page 34: Camouflaging an object from many viewpoints

Evaluation metrics• Confusion rate

• 存在するかどうかの判断時間 [s]

• Time to find• 画像内の場所を見つけ出すまでの時間 [s]

Page 35: Camouflaging an object from many viewpoints

Experiment• 37 シーン(多視点撮影)を対象• カモフラージュする物体は直方体

Page 36: Camouflaging an object from many viewpoints

Results

Page 37: Camouflaging an object from many viewpoints

Results• MRF が優位に性能が高い

Page 38: Camouflaging an object from many viewpoints

Greedy vs Boundary MRF

Page 39: Camouflaging an object from many viewpoints

Interior MRF vs Boundary MRF

Page 40: Camouflaging an object from many viewpoints

Results from multi viewpoints• Boundary MRF

https://www.youtube.com/watch?v=NNlE_hzqdss

Page 41: Camouflaging an object from many viewpoints

Conclusion• 複数視点から 3D オブジェクトをカモフラージュする手法を提案• MRF の結果は良好• 色々なカモフラージュ方法を試してみたい