Multiple View Geometry in Computer Visionpksaho01/teaching/Lecture8.pdf · Multiple View Geometry...

Multiple View Geometry

in Computer Vision

Prasanna Sahoo

Department of Mathematics

University of Louisville

1

Camera Models

Lecture 8

2

In this lecture, we will show that a camera is a map-

ping from the 3D world R3 to a 2D image plane R2.

This mapping can be represented by a 3×4 matrix P.

We will examine the model for the following cameras:

• Pinhole camera

• CCD camera

• Finite projective camera

• General projective camera

3

Pinhole Camera

A pinhole camera is a box in which one of the walls

has been pierced to make a small hole through it.

Assuming that the hole is indeed just a point, exactly

one ray from each point in the scene passes through

the pinhole and hits the wall opposite to it. This

results in an inverted image of the scene.

4

Pinhole camera

The word camera has its origins in the Latin camera and the Greek kamara,

both of which refer to a room or a chamber.

5

The inversion of the image is an annoyance.

• However, it can be corrected by considering a virtual

image of the scene on a virtual plane parallel to the

imaging plane but on the opposite side of the pinhole.

6

Basic pinhole camera model

Let the center of projection be the origin of a Eu-

clidean coordinate system. The plane Z = f is called

the focal plane or image plane.

7

A point in space, R3, with coordinates X = (x, y, z)T

is mapped to a point on the image plane where a line

joining the point X to the center of projection meets

the image plane.

It can be easily shown that the point (x, y, z)T is

mapped to the point(f xz , f y

z , f)T

on the image plane.

8

Ignoring the final image coordinate, we see that the

mapping

(x, y, z)T 7→(fx

z,fy

z

)T

describes the central projection mapping from world

to image coordinates.

• This is a mapping from R3 to R2.

9

Some Termnilogies

• The center of projection is called the camera center

or the optical center.

• The plane Z = f is called the focal plane or image

plane.

10

• The line from the camera center perpendicular to

the image plane is called the principal axis or princi-

pal ray of the camera.

• The point where the principal axis meets the image

plane is called the principal point.

• The plane through the camera center parallel to

the image plane is called the principal plane of the

camera.

11

Central Projection Mapping

Using homogeneous coordinates the central projection

map (x, y, z)T 7→ (f x/z, f y/z)T can be described asxyz1

7→f xf yz

=

f 0 0 00 f 0 00 0 1 0

xyz1

= diag(f, f,1) [ I | 0 ]X,

where diag(f, f,1) is a diagonal matrix and [ I | 0 ] is

a matrix divided up into a 3 × 3 block (the identity

matrix) plus a column vector made up of zeros.12

The equation on the last slide can be written as

x = PX

where X denotes the world point represented by the

homogeneous 4-vector (x, y, z,1)T, x the image point

represented by the homogeneous 3-vector (fx, fy, z)T,

and P the 3 × 4 homogeneous camera projection

matrix. Hence

P = diag(f, f,1) [ I | 0 ].13

In deriving the central projection mapping

(x, y, z)T 7→ (fx/z, fx/z)T

it was assumed that the origin of coordinates in the

image plane was at the principal point. However, if

the origin is not at the principal point, then we have

(x, y, z)T 7→ (fx/z + px, fx/z + py)T

where (px, py)T are the coordinates of the principal

point.14

Image and camera coordinate

systems

15

Using homogeneous coordinates the central projection

mapping can be described as

xyz1

7→f x + zpx

f y + zpy

z

=

f 0 px 00 f py 00 0 1 0

xyz1

= K[ I | 0 ]Xcam,

where K is a 3×3 matrix, [ I | 0 ] is a matrix divided up

into a 3× 3 block (the identity matrix) plus a column

vector made up of zeros.

16

The equation on the last slide can be written concisely

asx = K [ I | 0 ]Xcam.

• The matrix K is called camera calibration matrix.

• The homogeneous 4-vector (x, y, z,1)T is written as

Xcam to emphasize that the camera is assumed to be

located at the origin of a Euclidean coordinate system

with the principal axis of the camera pointing straight

down the z-axis.17

Camera Location

The camera is assumed to be located at the origin of a Euclidean coordinate

system with the principal axis of the camera pointing straight down the z-axis.

18

Camera Rotation and Translation

In general, points in R3 will be expressed in terms of

a different Euclidean coordinate frame, known as the

world coordinate frame. The two coordinate frames

are related through a rotation and a translation.

19

World and Camera Coordinate Frames

The two coordinate frames are related through a rotation and a translation.

20

If X̃ ia an inhomogeneous 3-vector representing the

coordinates of point in the world coordinate frame,

and X̃cam represents the same point in the camera

coordinate frame, then

X̃cam = R (X̃− C̃),

where C̃ represents the coordinates of the camera cen-

ter in the world coordinate frame, and R is a 3×3 rota-

tion matrix representing the orientation of the camera

coordinate frame.21

The equation X̃cam = R (X̃ − C̃) can be written in

homogeneous coordinates as

X̃cam =

R −RC̃

0 1

xyz1

=

R −RC̃

0 1

X.

This leads to the following concise formula

x = KR [ I | − C̃ ]X

where X is now in a world coordinate frame.

22

The mapping x 7→ X defined by the formula

x = KR [ I | − C̃ ]X

is the general mapping given by a pinhole camera. A

general pinhole camera

P = KR [ I | − C̃ ]

has 9 degrees of freedom (3 DOF for K, 3 DOF for

R, and 3 DOF for C̃).23

It is often convenient not to make the camera center

explicit in the world to image transformation. Instead

it is represented as

X̃cam = RX̃ + t.

Hence the camera matrix becomes

P = KR [ I | − C̃ ] = K [R | t ]

where t = −RC̃.24

CCD Cameras

In pinhole camera model it is assumed that the image

coordinates have equal scales in both axial directions.

In the case of CCD cameras, this scale factors are

unequal in each direction.

25

EURON Summer School on Visual servoing

Image plane

����

����

����

���

m = (x,y,1)p = (u,v,1)

3D Point

Center of projection

(X,Y,Z)3D Point

Charged Coupled

Device

Image coordinates: p = Km

Camera Geometry – p.9/43

CCD Camera

26

Suppose the scale factors in the directions x and y are

mx and my, respectively. Hence the calibration matrix

K for the CCD camera is given by

K =

mx 0 00 my 00 0 1

f 0 px

0 f py

0 0 1

=

f mx 0 px mx

0 f my py my

0 0 1

.

Hence a CCD camera

P = KR [ I | − C̃ ]

has 10 degrees of freedom (that is 4 + 3 + 3 = 10).27

Note the calibration matrix for the CCD camera can

be written as

K =

αx 0 x0

0 αy y0

0 0 1

where αx = f mx, αy = f my, x0 = px mx, and y0 =

py my.

28

Finite Projective Cameras

A camera P is called a finite projective camera if the

calibration matrix K is of the form

K =

αx s x00 αy y00 0 1

(1)

where s is a parameter known as the skew parameter.

Hence a finite projective camera P is given by

P = KR [ I | − C̃ ].

29

• The 3× 3 submatrix KR of

P = KR [ I | − C̃ ]

is non-singular.

• If P is any 3 × 4 matrix for which the left hand

3 × 3 submatrix, say M, is non-singular, then M can

be decomposed as M = KR, where K is a upper-

triangular matrix of the form (1) and R is a rotation

matrix.30

Therefore if the 3 × 3 submatrix P is non-singular,

then the 3× 4 matrix P can be written as

P = M [ I | M−1 p4 ] = KR[ I | − C̃ ]

where p4 is the last column of P. Thus we have:

Result 5.1. The set of camera matrices of finite

projective cameras is identical with the set of homo-

geneous 3 × 4 matrices for which the left hand 3 × 3

submatrix is non-singular.

31

General Projective Cameras

A camera is called a general projective camera if it

can be represented by an arbitrary homogeneous 3×4

matrix of rank 3.

The rank 3 requirement is needed because if the rank

is less than 3, then the range of the matrix mapping

will be a line or a point but not the whole plane.

32

END

33