Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1
CSE190, Spring 2014
Image Formation
Biometrics CSE 190 Lecture 5
CSE190, Spring 2014
Announcements
• HW1: To be assigned tonight
Pattern Classification, Chapter 2 (Part 3)
3 Discriminant Functions for the Normal Density
• We saw that the minimum error-rate classification can be achieved by the discriminant function gi(x) = ln P(x | ωi) + ln P(ωi)
• Case of multivariate normal
• Decision boundary gi(x) = gj(x) €
gi(x) = −12(x − µi)
tΣi−1(x − µi) −
d2ln2π − 1
2lnΣi + lnP(ω i)
Pattern Classification, Chapter 2 (Part 3)
4
CASE I
Case Σi = σ2I (I stands for the identity matrix)
€
gi(x) = wit x + wi0 (linear discriminant function)
where :
wi =µiσ 2
; wi0 = −1
2σ 2µitµi + lnP(ω i)
(ω i0 is called the threshold for the ith category!)
Pattern Classification, Chapter 2 (Part 3)
5
• The decision surfaces for a linear machine are pieces of hyperplanes defined by:
gi(x) = gj(x)
Pattern Classification, Chapter 2 (Part 3)
6
2
Pattern Classification, Chapter 2 (Part 3)
7
The hyperplane separating Ri and Rj are given by
always orthogonal to the line linking the means!
)()()(ln)(
21
0)(
2
2
0
0
jij
i
ji
ji
ji
t
PP
µµωω
µµ
σµµ
µµ
−−
−+=
−=
=−
x
wxxw
)(21)()( 0 jiji PPif µµωω +== x then
Pattern Classification, Chapter 2 (Part 3)
8
Pattern Classification, Chapter 2 (Part 3)
9 Case II Case Σi = Σ (covariance of all classes are identical but arbitrary!)
Now consider case where P(ωi)=P(ωj),
€
gi(x) = −12(x − µi)
tΣi−1(x − µi) −
d2ln2π − 1
2lnΣi + lnP(ω i)
= −12(x − µi)
tΣ−1(x − µi) −12lnΣ + lnP(ω i)
= −12(x − µi)
tΣ−1(x − µi) + lnP(ω i)
€
gi(x) = −(x − µi)tΣ−1(x − µi)
€
(x − µi)tΣ−1(x − µi) is sometimes called the Mahalanobis distance
Pattern Classification, Chapter 2 (Part 3)
10 Case II Case Σi = Σ (covariance of all classes are
identical but arbitrary!)
Hyperplane separating Ri and Rj
Here the hyperplane separating Ri and Rj is generally not orthogonal to the line between the means!
[ ])(
)()()(/)(ln
)(21
)(
0)(
10
10
jiji
tji
jiji
ji
t
PPx µµ
µµµµ
ωωµµ
µµ
−−Σ−
−+=
−Σ=
=−
−
−wxxw
Pattern Classification, Chapter 2 (Part 3)
11
Pattern Classification, Chapter 2 (Part 3)
12 CASE III
Case Σi = arbitrary The covariance matrices are different for each category
Here the separating surfaces are Hyperquadrics which are: hyperplanes, pairs of hyperplanes, hyperspheres, hyperellipsoids, hyperparaboloids, hyperhyperboloids)
)(Plnln21
21 w
w21W
:wherewxwxWx)x(g
iii1
iti0i
i1
ii
1ii
0itii
ti
ωΣµΣµ
µΣ
Σ
+−−=
=
−=
=+=
−
−
−
3
Pattern Classification, Chapter 2 (Part 3)
13
CSE190, Winter 2011 Biometrics
Image Formation: Outline • Factors in producing images • Projection • Perspective • Vanishing points • Orthographic • Lenses • Sensors • Quantization/Resolution • Illumination • Reflectance
CSE190, Winter 2011 Biometrics
Earliest Surviving Photograph
• First photograph on record, “la table service” by Nicephore Niepce in 1822. • Note: First photograph by Niepce was in 1816.
CSE190, Winter 2011 Biometrics
Images are two-dimensional patterns of brightness values.
They are formed by the projection of 3D objects.
Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.
CSE190, Winter 2011 Biometrics
Effect of Lighting: Monet
CSE190, Winter 2011 Biometrics
Change of Viewpoint: Monet
Haystack at Chailly at Sunrise (1865)
4
CSE190, Winter 2011 Biometrics
Pinhole Camera: Perspective projection
• Abstract camera model - box with a small hole in it
Forsyth&Ponce CSE190, Winter 2011 Biometrics
Parallel lines meet in the image • vanishing point
Image plane
CSE190
Computer Vision I
Equation of Perspective Projection
Cartesian coordinates: • We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, f) • Establishing an image plane coordinate system at C’ aligned with i
and j, we get
€
(x,y, z)→ ( f xz, f yz)
CSE190
Computer Vision I
A Digression
Homogenous Coordinates and
Camera Matrices
CSE190
Computer Vision I
What is the intersection of two lines in a plane?
A Point
CSE190
Computer Vision I
Do two lines in the plane always intersect at a point?
No, Parallel lines don’t meet at a point.
5
CSE190
Computer Vision I
Can the perspective image of two parallel lines meet at a point?
YES
CSE190
Computer Vision I
Projective geometry provides an elegant means for handling these different situations in a unified way and homogenous coordinates are a way to represent entities (points & lines) in projective spaces.
CSE190
Computer Vision I
Homogenous coordinates • Our usual coordinate system is called a Euclidean
or affine coordinate system
• Rotations, translations and projection in Homogenous coordinates can be expressed linearly as matrix multiplies
Euclidean World
3D
Homogenous World
3D
Homogenous Image
2D
Euclidean World
2D
Convert
Convert
Projection
CSE190
Computer Vision I
Changes of coordinates: Euclidean -> Homogenous-> Euclidean
In 2-D • Euclidean -> Homogenous: (x, y) -> k (x,y,1) • Homogenous -> Euclidean: (u,v,w) -> (u/w, v/w)
In 3-D • Euclidean -> Homogenous: (x, y, z) -> k (x,y,z,1) • Homogenous -> Euclidean: (x, y, z, w) -> (x/w, y/w, z/w)
CSE190
Computer Vision I
Homogenous coordinates A way to represent points in a projective space 1. Add an extra coordinate
e.g., (x,y) -> (x,y,1)=(u,v,w)
2. Impose equivalence relation such that (k not 0)
(u,v,w) ≈ k*(u,v,w) i.e., (x,y,1) ≈ (kx, ky, k)
3. “Point at infinity” – zero for last coordinate e.g., (x,y,0)
• Why do this? – Possible to write the
action of a perspective camera as a matrix
– Possible to represent points “at infinity”
• Where parallel lines intersect
• Vanishing points are the projection of points of points at infinity
CSE190
Computer Vision I
The camera matrix
Turn this expression into homogenous coordinates – HC’s for 3D point are
(X,Y,Z,T) – HC’s for point in image
are (U,V,W) €
UVW
"
#
$ $ $
%
&
' ' '
=
1 0 0 00 1 0 00 0 1 f 0
"
#
$ $ $ $
%
&
' ' ' '
XYZT
"
#
$ $ $ $
%
&
' ' ' '
€
(x,y, z)→ ( f xz, f yz)
Perspective Camera Matrix A 3x4 matrix
6
CSE190
Computer Vision I
End of the Digression
CSE190
Computer Vision I
Simplified Camera Models
Perspective Projection
Affine Camera Model
Orthographic Projection
CSE190, Winter 2011 Biometrics
Affine Camera Model
• Take Perspective projection equation, and perform Taylor Series Expansion about (some point (x0,y0,z0).
• Drop terms of higher order than linear. • Resulting expression is affine camera model
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
0
0
0
zyx
Appropriate in Neighborhood About (x0,y0,z0)
CSE190, Winter 2011 Biometrics
• Perspective
• Perform a Taylor series expansion about (x0, y0, z0)
• Dropping higher order terms and regrouping.
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡
yx
zf
vu
uv
!
"#
$
%&=
fz0
x0y0
!
"##
$
%&&−fz02
x0y0
!
"##
$
%&&z− z0( )+
fz0
10
!
"#
$
%& x − x0( )
+fz0
01
!
"#
$
%& y− y0( )+
12fz03
x0y0
!
"##
$
%&&z− z0( )
2+!
uv
!
"#
$
%& ≈
fz0
x0y0
!
"##
$
%&&+
f / z0 0 − fx0 / z02
0 f / z0 − fy0 / z02
!
"
###
$
%
&&&
xyz
!
"
###
$
%
&&&=Ap+b
CSE190, Winter 2011 Biometrics
uv
!
"#
$
%& ≈
fz0
x0y0
!
"##
$
%&&+
f / z0 0 − fx0 / z02
0 f / z0 − fy0 / z02
!
"
###
$
%
&&&
xyz
!
"
###
$
%
&&&=Ap+b
Rewrite Affine camera model in terms of Homogenous Coordinates
uvw
!
"
###
$
%
&&&≈
f / z0 0 − fx0 / z02 fx0 / z0
0 f / z0 − fy0 / z02 fy0 / z0
0 0 0 1
!
"
####
$
%
&&&&
xyz1
!
"
####
$
%
&&&&
Parallel lines project to parallel lines Ratios of distances are preserved under orthographic projection
CSE190, Winter 2011 Biometrics
uv
!
"#
$
%&=
fz0
xy
!
"##
$
%&&
Orthographic projection Starting with Affine camera mode
Take Taylor series about (0, 0, z0) – a point on optical axis
UVW
!
"
###
$
%
&&&=
f / z0 0 0 00 f / z0 0 00 0 0 1
!
"
####
$
%
&&&&
XYZT
!
"
####
$
%
&&&&
Euclidean Coordinates Homogenous Coordinates
7
CSE190 Computer Vision I
What if camera coordinate system differs from object coordinate system?
{c}
P {W}
CSE190 Computer Vision I
Euclidean Coordinate Systems
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⇔++=⇔⎪⎩
⎪⎨
⎧
=
=
=
zyx
zyxOPOPzOPyOPx
Pkjikji
...
CSE190 Computer Vision I
A convenient notation
BP = ABR AP+ BOA
– Points: AP1 – Leading superscript indicates the coordinate system that the coordinates are with respect to – Subscript – an identifier
– Rotation Matrices – Lower left (Going from this system) – Upper left (Going to this system)
– To add vectors, coordinate systems (leading superscript) must agree – To rotate a vector, points coordinate system must agree with lower left of rotation matrix
RBA
CSE190 Computer Vision I
Coordinate Changes: Rigid Transformations
ABAB
AB OPRP +=
Rotation Matrix Translation vector
CSE190 Computer Vision I
More about rotations matrices A rotation matrix R has the following properties:
• Its inverse is equal to its transpose R-1 = RT or RTR = I
• Its determinant is equal to 1: det(R)=1. Or equivalently:
Rows (or columns) of R form a right-handed orthonormal coordinate system.
• Even though a rotation matrix is 3x3 with nine numbers, it only has three degrees of freedom, it can be parameterized with three numbers.
• There are many parameterizations.
CSE190 Computer Vision I
Rotation • About z axis
x' y' z’
=
x y z
cos θ sin θ
0
-sin θ cos θ
0
0 0 1
rot(z,θ) x
y
z
p
p'
θ
Note: z coordinate doesn’t change after rotation
8
CSE190 Computer Vision I
Rotation
• About x axis:
• About y axis:
x' y' z’
=
x y z
0 cos θ sin θ
0 -sin θ cos θ
1 0 0
x' y' z’
=
x y z
cos θ 0
-sin θ
sin θ 0
cos θ
0 1 0
CSE190 Computer Vision I
Roll-Pitch-Yaw ),ˆ(),ˆ(),ˆ( ϕβα krotjrotirotR =
),ˆ(),'ˆ(),''ˆ( ϕβα krotjrotkrotR =
Euler Angles
CSE190 Computer Vision I
Rotation • Rotation by angle θ about (kx, ky, kz),
a unit vector • (Rodrigues Formula)
x' y’ z’
= x y z
kxkx(1-c)+c kykx(1-c)+kzs kzkx(1-c)-kys
kxky(1-c)-kzs kyky(1-c)+c kzyx(1-c)-kxs
kxkz(1-c)+kys kykz(1-c)-kxs kzkz(1-c)+c
where c = cos θ & s = sin θ
Rotate(k, θ)
x
y
z
θ
k
CSE190 Computer Vision I
Block Matrix Multiplication Given
€
A =A11 A12A21 A22
"
# $
%
& ' B =
B11 B12B21 B22
"
# $
%
& '
What is AB ?
⎥⎦
⎤⎢⎣
⎡
++
++=
2222122121221121
2212121121121111
BABABABABABABABA
AB
Homogeneous Representation of Rigid Transformations
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=⎥⎦
⎤⎢⎣
⎡ +=⎥
⎦
⎤⎢⎣
⎡
11111P
TPOROPRP AB
A
A
TA
BBAA
BABA
B
0
Transformation represented by 4 by 4 Matrix
CSE190 Computer Vision I
What if camera coordinate system differs from object coordinate system?
{c}
P {W}
€
wcT = w
cR cOw0T 1
"
# $
%
& '
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜
⎝
⎛
=⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
1010000100001
ZYX
fWVU
CSE190 Computer Vision I
Intrinsic parameters
u
v
3x3 homogenous matrix Focal length: Principal Point: C’ Units (e.g. pixels) Orientation and position of image coordinate system Pixel Aspect ratio
9
CSE190 Computer Vision I
Camera parameters • Extrinsic Parameters: Since camera may not be at the origin,
there is a rigid transformation between the world coordinates and the camera coordinates
• Intrinsic parameters: Since scene units (e.g., cm) differ image units (e.g., pixels) and coordinate system may not be centered in image, we capture that with a 3x3 transformation comprised of focal length, principal point, pixel aspect ratio, angle between axes, etc.
UVW
!
"
###
$
%
&&&=
Transformationrepresented byintrinsic parameters
!
"
###
$
%
&&&
1 0 0 00 1 0 00 0 1 0
!
"
###
$
%
&&&
Rigid Transformationrepresented by extrinsic parameters
!
"
###
$
%
&&&
XYZ1
!
"
####
$
%
&&&&
3 x 3 4 x 4 wcT
CSE190 Computer Vision I
, estimate intrinsic and extrinsic camera parameters • See Text book for how to do it. • Camera Calibration Toolbox for Matlab (Bouguet) http://www.vision.caltech.edu/bouguetj/calib_doc/
Camera Calibration