1

Camera calibration based on arbitrary

parallelograms

授課教授：連震杰學生：鄭光位

2

Outline

Introduction Camera of warped images Camera calibration by transferring

scene constrain Experiments result Conclusion

3

Introduction(1)

The reconstruction of the three-dimensional (3D) objects from images is one of the most important topics in computer vision

Estimating the camera’s intrinsic parameters or, equivalently, to recover metric invariants such as the absolute conic or the dual absolute quadric is a key issue of 3D metric reconstruction .

There’re three kinds of approaches : using only scene constrains of known targets called calibration objects 、 using only camera assumption without calibration object and combination of two above.

4

Introduction(2) However, all of the above methods, which are based on the camera

constraints, are not applicable to image sets that have geometrically transformed images such as pictures of pictures, scanned images of printed pictures, and images modified with image-processing tools.

In order to use geometrically transformed images in calibrating cameras or metric reconstruction of scenes with the previous methods, it is needed to find a way to transform the scene constraints in the image into another image without camera constraints and this requires additional processes such as estimating epipolar geometry.

5

Introduction(3) In this paper, we propose a unified framework to deal with

geometrically transformed images basing on warping images containing parallelograms.

The image warping converts the problems with scene constraints of parallelograms into the well-known autocalibration problems of intrinsically constrained cameras.

In addition, we propose a method to estimate infinite homographies between views using the parallelograms and how to reconstruct the physical cameras in affine space by using the newly introduced constraints of the warped images without knowledge about the cameras or projective reconstruction.

6

Outline

Introduction Camera of warped images Camera calibration by transferring

scene constrain Experiments result Conclusion

7

Camera of warped images(1)

Assume that there is a plane–plane homography H and a pinhole projection model is given as

[ ]p K R t

[ ] [ ]new new new newP HK R t K R t

By warping the image with the homography H

※Warping an image does not change the apparent camera center because

0PC HPC This means that the camera of the warped image is a rotating camera of the original one

8

Camera of warped images(2)

Assume that there is a rectangle whose aspect ratio is Rm in 3D space ,and we set the reference plane as Z = 0,which is the plane containing the rectangle.

homography that transforms the projected rectangles to canonical coordinates is defined as:

1 11 2 1 2[ ] ( [ ] ( , , ))FP cH v v x K r r t diag a b c

After transforming an image with HFP, the reference plane becomes fronto-parallel one whose aspect ratio is not identical to the actual 3D plane.

v1,v2, and v3 are the vanishing pointsthat correspond to the three orthogonal directions in 3Dxc is the projection of the pre-defined origin Xc on the reference plane

9

Camera of warped images(3)

Theorem: In the warped image by HFP, the ICDCP of the reference plane is given as diage(R2

m; R2FP; 0), where Rm is an aspect ratio of

the real rectangle and RFP is an aspect ratio of the warped rectangle.

Proof: Assume that a projection process of a plane is described with HP. The CDCP is a dual circle whose radius is infinity and we derive it from regular circles. Assume that there is a circle whose radius is r in the model plane (Euclidean world). The conic dual to the projected circle in the projected image is

* 2pA =H (1,1, 1/ ) T

pdiag r H

* 2 2 2 2 2FP FP p FPA =H H (1,1, 1/ ) H (1/ ,1/ , 1/ )T T

pdiag r H diag a b c r

10

Camera of warped images(4)

a point (X,Y,1)T on the reference plane is warped to the point at

( , ,1) ( , ,1) ( , ,1)T TFP FP FP

c cX Y H X Y X Y T

a b

and : :FPm FP FP

FP

YYR R a b R Rm

X X

2 2 2 2 2 2 2 2

FP FP

FP

when r goes to ,

(1/ ,1/ , 1/ ) (1/ ,1/ ,0) ( , ,0)

then ICPs I and J are simply expressed as

[ 0] and J [ 0]

m FP

T TFP m FP m FP

diag a b c r diag a b diag R R

I R iR R iR

11

Camera of warped images(5)◎Intrinsic parameters of fronto-parallel (FP) cameras We can imagine that there is a camera, called a fronto-parallel (FP) camera, that may capture the warped image.

1 1 2 2 2 3 3 3 1

2 2 21 2 2 3 3 1

, , T T T

l v v l v v l v v

w l l l l l l

v1, v2 and v3 are three vanishing points that are in orthogonal directions

2 2 23 1 3

2 2 2 3FP 3 2

2 2 2 2 21 3 2 3 1 2

0

w 0 .....(*)

m mm

m m m

mm m m m m

In warped image , setting vanishing points as

gives the IAC wFP of the FP camera

1 2 3 1 2 3[1 0 0] , [0 1 0] and [ ]T T Tv v v m m m

12

Camera of warped images(6)Because the ICPs of the plane are on the IAC

2 2

2 20 , 0 T T m

FP FP FP FP FP FPFP

RI w I J w J

R

By decomposing (*),the FP camera matrix is

1

2 3 1 2 3

3

1 0

1 , v [ , , ]

FP

TFP

m

mR

K m m m mR

m

1

2

3

cot1

1 , is an angle between two parallel line sets.

FP m

FPm

mR R

K mR

m

13

Camera of warped images(7)

◎Motion constraints of FP cameras

11 2 3 1 2 1 2 3

1

1 2 3 3 3

( ( , , ))

1 1 1 ( , , ) 1 1

FP FP

FP

P H K r r r t K r r t diag a b c K r r r t

diag e e r r t r e K ea b c

1 2 31 0 0 , 0 1 0 , 0 0 1T T T

e e e

the rotation matrix is an identity matrix in the FP camera

it is straightforward to show the following two motion constraints

Constraint 1. Assume that there is a parallelogram in 3D space and twoviews i and j that see the parallelogram. The FP cameras PFPi and PFPj ,which are derived from an identical parallelogram in 3D, are puretranslating cameras.

14

Camera of warped images(8)

Constraint 2. Assume that there are two static parallelograms I and J in general position in 3D space that are seen in two view i and j. The relative rotation between two FP cameras of the view i is identical to that between two FP cameras of the view j.

15

Outline

Introduction Camera of warped images Camera calibration by transferring

scene constrain Experiments result Conclusion

16

Camera calibration by transferring scene constraints

Intrinsic matrix w from the constraint of FP cameraK IAC

camera calibration using geometric constraints of rectangles is equivalent to the autocalibration of rotating cameras whose intrinsic parameters are varying.

A group of cameras DOF : 5

N groups , DOF : 5n

but if we get HijFPI，

then it will remains at 5 ;

and that H will be infinite homography between two cameras

17

Camera calibration by transferring scene constraints

1 1

2 2

3 3

cot cot1 1

1 1 , i j

i j

FP FP

i jFP FP

m m

i j

m mR R

K m K mR R

m m

3 1 2 3

3 1 2 3

[ ] ,

[ ]

i i i i T

j j j j T

v m m m

v m m m

1

1 0

H 0 1

0 0i ij j

xijFP FP FP FP y

z

h

K R K h

h

there are two views i and j that see an unknown parallelogram

where vi3 and vj

3 represent scaled orthogonal vanishing points in the warped images of view I and j, respectively.

because two FP cameras are

purely translating to each other

x y zH has three parameters h ,h and hijFP

1

j i

ij T ijFP FP FP FPw H w H

18

Camera calibration by transferring scene constraints

1

1

1 1

1 1

1

,

= ,

=

=

j i

j j j

j i j i i i

j i i j

j i i

ij T ijFP FP FP FP

TFP FP j FP

T ij T ij Tj FP FP FP FP FP FP FP i FP

T ij T T ijj FP FP FP i FP FP FP

ij T T ijj FP FP FP i FP FP

w H w H

w H w H

w H H w H H w H w H

w H H H w H H H

w H H H w H H

代入上式

再代入

觀察上式可知

i j

1

1

1

ijFP FP FP

=

infinite homography is expressed with fronto-parallel warping matrices

H and H , and an infinite homography H between FP cameras

j

j i

FP

ij T iji

ij ijFP FP FP

H

H w H

H H H H

19

Outline

Introduction Camera of warped images Camera calibration by transferring

scene constrain Experiments result Conclusion

20

Experiments result(1)

captured with a SONY DSC-F717 camera in 640 * 480 resolution

Zhang

728.5874 28.4393 352.8952 721.3052 27.013 335.8925

0 724.9379 285.1658 , K 0 724.9379 247.3248

0 0 1 0 0 1estimatedK

※Camera calibration using static cameras

21

Experiments result(2)

※Reconstruction using image sets including pictures of pictures

22

Experiments result(3)

The angle between major walls was estimated to be 90.69 degree；

the true angle measured 90 degree using SICK laser range finder as blue lines

23

Experiments result(4)

24

Outline

Introduction Camera of warped images Camera calibration by transferring

scene constrain Experiments result Conclusion

25

Conclusion

Conventional camera calibration methods based on the camera constraints are not applicable to images containing transformed images, such as pictures of pictures or scanned images from printed pictures.

we propose a method to convert scene constraints in the form of parallelograms into camera constraints by warping images.

A linear method is also proposed to estimate an infinite homography linearly using fronto-parallel cameras without any assumptions about cameras.

we show that the affine reconstruction of cameras is possible directly from the infinite homography derived from images of parallelograms.