ボリュームデータ生成 (1)...R.Hartley and A.Zisserman, “Multiple View Geometry”, Cambridge University press, 2001. V ýG/GeGmFî B㌹GJG GZGoG :#ÕFÝFå ... エピポーラ幾何

ボリュームデータ生成 (1)

情報メディア工学特論２００４年１１月２日


ゝ

CT MRI


々々

々

３次元形状復元 (1)

R.Hartley and A.Zisserman, “Multiple View Geometry”, Cambridge University press, 2001.

㌹

３次元形状復元 (2)

X

x x’


実時間ボクセル生成

Capturing Displaying

Reconstruction

Real-time

ボクセル生成処理手順

っ

初期ボクセルモデル生成 (1)


っ

C1

C2C3

F(C1)

F(C2)

F(C3)

O

O

!

O = F Ci( )

i=1

n

I Ci: F(Ci): Ci っ


Camera A Camera B

Silhouette image

Object

Initial volume

Initial volume

Initial planes


Back-projection

ScalingTranslation

ゝ

!

x

y

f

"

#

$ $ $

%

&

' ' '

=

p11

p12

p13

p14

p21

p22

p23

p24

p31

p32

p33

p34

"

#

$ $ $

%

&

' ' '

X

Y

Z

1

"

#

$ $ $ $

%

&

' ' ' '

(

x

y

f

"

#

$ $ $

%

&

' ' '

=

p11

p12

p14

p21

p22

p24

p31

p32

p34

"

#

$ $ $

%

&

' ' '

X

Y

1

"

#

$ $ $

%

&

' ' '

ボクセル可視判定 (1)

Surfacevoxel

ObjectSurface Voxel List (SVL)


AB

IDdistance

っ

Camera

Depth buffer


farnear

ボクセル色割当 (1)

object

Light

Surfacevoxels

C1

C2

C3

V

!

Vcolor

= color(C1,C

2,C

3,L,C

i)


Light

Eye

Specular Reflection

Diffuse Reflection

Object Surface

i

r

dnI = R cos(i) + W(i) cos (r)

(Diffuse Reflection) (Specular Reflection)


object

Light

Surfacevoxels

C1

C2

C3

Cmean=C1 + C2 + C3

3

Surface Voxel List (SVL)

並列処理1: Image Capturing (ICP) Stage

2: Image Correcting (ICR) Stage

3: Base Silhouette Image Generation (BSG) Stage

4: Slice Image Generation (SIG) Stage

5: Slice Image Distribution (SID) Stage

6: Voxel Model Generation (VMG) Stage

7: Depth-Buffer Generation (IBG) Stage

8: Depth-Buffer Distribution (IBD) Stage

9: Colored Voxel Generation (CVG) Stage

10: Colored Voxel Distribution (CVD) Stage

システム構成

Control node

Processing nodes

Camera

3 m

2 m

2 m

Image capturing system

• PC cluster:– 6 PCs (5 processing nodes and a control node)– CPU: Intel Xeon 3.06GHz, RAM: 2.0GB– NIC: Gigabit Ethernet

• Camera:– 5 cameras– IEEE1394 digital camera (Sony DFW-VL500)

復元結果 (1)

Frame #16 Frame #19

Frame #29 Frame #56

復元結果 (2)

(a) 96x96x96 (10mm; 13,530 voxels) (b) 107x107x107 (9mm; 18,130 voxels) (c) 120x120x1120 (8mm; 22,570 voxels)

(d) 137x137x137 (7mm; 33,355 voxels) (e) 160x160x160 (6mm; 50,220 voxels) (f) 192x192x192 (5mm; 78,970 voxels)

平均スループット

!

!"#

$

$"#

%

%"#

&

'()*+,-.,/()*01(2

345(*647*0-89:;<

9:; %"=& %">& $"?& $"# $"$> !"@>

?=A& $!BA& $%!A& $&BA& $=!A& $?%A&

平均処理時間 (1)

813.0480.8404.9280.9120.0133.2CVD Stage96.256.537.325.623.517.8CVG Stage

806.5526.3444.4320.5230.9211.9IBD Stage806.5531.9362.3261.1162.1117.9IBG Stage885.0588.2483.1311.5236.4223.2VMG Stage909.1632.9420.2187.2125.9159.2SID Stage628.9293.3107.035.237.630.4SIG Stage20.920.016.017.013.712.5BSG Stage32.725.525.020.226.727.8ICR Stage0.480.440.450.360.420.38ICP Stage

19231603137312031073933

[msec]

平均処理時間 (2)

!"!!#!!$!!%!!&!!'!!(!!)!!*!!"!!!

+,-+,./010+10+2341+/1+/2,31,32

05678

9:;8<=;>8?@

*$A$

"!(A$

"#!A$

"$(A$

"'!A$

"*#A$

形状復元デモビデオ

Single View Geometry

投影

Reconstruction

Real world

Virtual world

Camera

Camera model

Projection

ピンホールカメラモデル

Zf

Zf

Image plane

Viewpoint (Focal point)

Z: Optical axisf : Focal length

(a)

(b)

透視投影

!

" x = fXc

Zc

, y = fYc

Zc

• 画像座標上の点: p (x,y)• カメラ座標上の点: Pc (Xc,Yc,Zc)

!

x

y

1

"

#

$ $ $

%

&

' ' '

=1

Zc

f 0 0 0

0 f 0 0

0 0 1 0

"

#

$ $ $

%

&

' ' '

Xc

Yc

Zc

1

"

#

$ $ $ $

%

&

' ' ' '

( ˜ p =1

Zc

f 0 0 0

0 f 0 0

0 0 1 0

"

#

$ $ $

%

&

' ' '

˜ P c

y

x

Oc

Yc Xc

Zc

Pc (Xc,Yc,Zc)p (x,y)

Zc = f

0

座標系

• 座標系– ピクセル座標系: p’ (u,v)– 画像座標系: p (x,y)– カメラ座標系: Pc (Xc,Yc,Zc)– 世界座標系: Pw (Xw,Yw,Zw)

u

v

yx

Zw

Xc

Yc

Oc

Ow

Yw

XwWorld coordinate

Image coordinate

Camera coordinate

Pixel coordinate

p’

p

Pc , Pw

カメラパラメータ

Pixel coordinate Image coordinate Camera coordinate World coordinate

Intrinsic parameters Extrinsic parameters

• 内部パラメータ– 焦点距離: f– 画像主点: (u0,v0)– ピクセルサイズ: (ku,kv)– レンズ歪み係数: k1, k2

• 外部パラメータ– 平行移動ベクトル: T– 回転行列: R

内部パラメータ (1)

Pixel coordinate

p’ (u,v)

uo

vo

u

v

!

"

Pixel size: (ku,kv)

Image coordinate

p (x,y)

y

x

From Pixel coordinate to Image coordinate:

!

u

v

1

"

#

$ $ $

%

&

' ' '

=

ku (ku cot) u0

0 kv sin) v0

0 0 1

"

#

$ $ $

%

&

' ' '

x

y

1

"

#

$ $ $

%

&

' ' '

* ˜ p '=

ku (ku cot) u0

0 kv sin) v0

0 0 1

"

#

$ $ $

%

&

' ' '

˜ p

内部パラメータ (2)Lens distortion


!

xd

yd

"

# $

%

& ' = 1+ k

1( r2 + k

2( r4( )

x

y

"

# $ %

& '

r = x2 + y

2

From Ideal (distortion-free) to Real (distorted):

distorted distortion-free

(xd,yd) (x,y)

外部パラメータ (1)

Camera coordinate

World coordinate

!

Pc

=RPw

+ T

From World coordinate to Camera coordinate:

!

R :

T :

Rotation matrix (3x3 matrix)

Translation vector (3D vector)

!

T

!

R

!

" ˜ P c

=R T

0 0 0 1

#

$ %

&

' ( ) ˜ P

w

外部パラメータ (2)

Camera coordinate

World coordinate

!

R =R"R#R$ =

1 0 0

0 cos" %sin"

0 sin" cos"

&

'

( ( (

)

*

+ + +

cos# 0 %sin#

0 1 0

sin# 0 cos#

&

'

( ( (

)

*

+ + +

cos$ %sin$ 0

sin$ cos$ 0

0 0 0

&

'

( ( (

)

*

+ + +

From World to Camera:

!

P =RPw

+ T

!

T =

tx

ty

tz

"

#

$ $ $

%

&

' ' '

!

=

cos" cos# $cos"sin# sin"

sin%sin" cos# + cos%sin# $sin%sin"sin# + cos% cos# $sin% cos"

$cos%sin" cos# + sin%sin# cos%sin"sin# + sin% cos# cos% cos#

&

'

( ( (

)

*

+ + +

=

r11

r12

r13

r21

r22

r23

r31

r32

r33

&

'

( ( (

)

*

+ + +

世界座標からピクセル座標 (1)

!

˜ P c

=R T

0 0 0 1

"

# $

%

& ' ( ˜ P

w

!

˜ p =1

Zc

f 0 0 0

0 f 0 0

0 0 1 0

"

#

$ $ $

%

&

' ' '

( ˜ P c

!

˜ p '=

ku

"kucot# u

0

0 kvsin# v

0

0 0 1

$

%

& & &

'

(

) ) )

* ˜ p

Pixel coordinate Image coordinate Camera coordinate World coordinate

World to CameraCamera to ImageImage to Pixel

!

˜ p '=

ku "ku cot# u0

0 kv sin# v0

0 0 1

$

%

& & &

'

(

) ) )

*1

Zc

f 0 0 0

0 f 0 0

0 0 1 0

$

%

& & &

'

(

) ) )

*R T

0 0 0 1

$

% &

'

( ) * ˜ P w

世界座標からピクセル座標 (2)

!

˜ p '=1

Zc

fku " fku cot# u0

0 fkv sin# v0

0 0 1

$

%

& & &

'

(

) ) )

* R T[ ] * ˜ P w

!

s " ˜ p '= AM " ˜ P w

!

" = fku

# = f kv sin$

% = & fku cot$

s = Zc

'

(

) )

*

) )

!

A =

" # u0

0 $ v0

0 0 1

%

&

' ' '

(

)

* * *

, M = R T[ ]

A: Intrinsic parameter matrixM: Extrinsic parameter matrix

,

Two View Geometry

エピポーラ幾何

Epipolar planeImage plane: I1Image plane: I2

R, T

Camera: C1

Camera: C2

Epipolar line: l1Epipolar line: l2

Epipole: e1

Epipole: e2

X

x1

x2

Essential 行列

!

x1

T

T" Rx2

+ T( ){ } = 0

x1=(x1,y1,f1), x2=(x2,y2,f2) x1, T, R x2 + T

X

x1x2

x1

T

R x2

!

" x1

T

T[ ]#R( )x2 = 0

!

" x1

T

Ex2

= 0

Essential

!

E = T[ ]"R

!

T[ ]" =

0 #tz ty

tz 0 #tx#ty tx 0

$

%

& & &

'

(

) ) ) !

x1

T

Ex2

= 0

:

Fundamental 行列

!

x1

T

Ex2

= 0

!

" x1

T

A1

#T( )E A2

#1x2( ) = 0

!

" x1

T

Fx2

= 0

X

x1x2

x1

T

R x2

Fundamental

!

F =A1

"TEA

2

"1=A

1

"TT[ ]

#RA

2

"1

!

x1

T

Fx2

= 0

小テスト（氏名：　　　　　　）

•

Documents

ボリュームデータ生成 (1)...R.Hartley and A.Zisserman, “Multiple View Geometry”, Cambridge University press, 2001. V ýG/GeGmFî B㌹GJG GZGoG :#ÕFÝFå ... エピポーラ幾何