Upload
woonhyuk-baek
View
860
Download
3
Embed Size (px)
Citation preview
Monocular Model-Based 3D Tracking of Rigid Ob-jects: A Survey
2008. 12. 04.백운혁
Chapter 2. Mathematical Tools
Agenda
Monocular Model-Based 3D Tracking of Rigid Objects : A Survey
Chapter 2. Mathematical Tools 2.1 Camera Representation 2.2 Camera Pose Parameterization 2.3 Estimating the External Parameters
Matrix 2.4 Least-Squares Minimization Techniques 2.5 Robust Estimation 2.6 Bayesian Tracking
the standard pinhole camera model
2.1 Camera Representation
2.1.1 The Perspective Projection Model
1
Z
Y
X
MWorld Coordinates
1
v
u
mImage Coordinates(in the image)
]|[ tRKP Projection Matrix
]|[ tR
K
Image coordinate system
2.1.2 The Camera Calibration Matrix internal parameters
100
0 0
0
v
us
K v
u
f focal length
fkuu
fkvv uk the number of pixels per unit distance
in the uvk the number of pixels per unit distance
in the v
0
0
v
uc principal point
s skew parameter
2.1.2 The Camera Calibration Matrix projection
Z
X
f
mx
f focal length
Z
Xfmx
Image Plane
2.1.2 The Camera Calibration Matrix projection to image
uk the number of pixels per unit distance in the u
vk the number of pixels per unit distance in the v
0
0
v
uc principal point (center of image
plane)
2.1.2 The Camera Calibration Matrix skew
field of view
s referred as the skew, usually
0
fkuu fkvv
image plane size and field of view are as-sumed to be fixed,but not fixed focal length
2.1.3 The External Parameters Matrix world coordinate to camera coordinate
]|[ tRThe 3x4 external pa-rameters
R rotation ma-trix
t translation vector
tRMM wc
wM in the world coordinate system
cM in the camera coordinate system
2.1.3 The External Parameters Matrix
2.1.4 Estimating the Camera Calibration Matrix
internal parameters are assumed to be fixed make use of a calibration pattern of known size
inside the field of view correspondence
between the 3D points and the 2D image points
2.1.5 Handling Lens Distortion
gentialradial duduuu tan
gentialradial dvdvvv tan
urkrkduradial )1( 42
21 radial distor-
tion
uvpvrp
urpuvpdu gential
222
1
2221
tan2)2(
)2(2tangential dis-tortion
(usually ig-nored)
22 vur
can be avoided by locally re-pametrizing the ro-tation
2.2 Camera Pose Parameteriza-tion
2.2.1 Euler Angles
cossin0
sincos0
001
cos0sin
010
sin0cos
100
0cossin
0sincos
R
α,β,γ to be rotation angles around the Z, Y, and X axis respec-tively yields
one rotation has no effect gimbal lock problem
2.2.2 Quaternions
2
1sin,
2
1cos wq
A rotation about the unit vector by an angle
w
a scalar plus a 3-vector),( va
2.2.3 Exponential Map
32
!3
1
!2
1)exp( I
A rotation about the unit vector by an angle
w
w
Let be a 3D vector
Tzyx wwww ,,
2.2.3 Exponential Map
is the skew-symmetric matrix
)( AAT
0
0
0
zy
xx
yz
ww
ww
ww
Rodrigues’ formula2ˆ)cos1(ˆ)(sin)exp()( IR
2)cos1
()sin()exp()(
IR
the exponential map represents a rotation as a 3-vector that gives its axis and magnitude.
iZ
2.2.4 Linearization of Small Rota-tions
RMM
MM
MI
)(
estimated camera positions
(when the internal parameters are known)
2.3 Estimating the External parameters Ma-trix
2.3.1 How many Correspondences are nec-essary?
n=3 known correspondencesproduce 4 possible solution(P3P Problem)
n>=4 known correspondencesproduce 2 possible solution
n>=4 known correspondences(points are coplanar)produce unique solution
n>=6 known correspondences produce unique solution
2.3.2 The Direct Linear Transformation (DLT)
to estimate the whole matrix P by solving a linear systemeven when the internal parameters are
not known Each correspondence
gives rise to two linearly independent equations
iiii
iii uPZPYPXP
PZPYPXP
34333231
14131211
iiii
iii vPZPYPXP
PZPYPXP
34333231
24232221
ii mM
iiii
iii uPZPYPXP
PZPYPXP
34333231
14131211
iiii
iii vPZPYPXP
PZPYPXP
34333231
24232221
03433323114131211 iiiiiiiiii uPuZPuYPuXPPZPYPXP
03433323114131211 iiiiiiiiii vPvZPvYPvXPPZPYPXP
010000
00001
34
11
P
P
vvZvYvXZYX
uuZuYuXZYX
iiiiiiiiii
iiiiiiiiii
0
34
11
P
P
B Stacking all the equation into B yields the lin-ear system :
2.3.2 The Direct Linear Transformation (DLT)
2.3.2 The Direct Linear Transformation (DLT)
6 correspondences must be known
for 3D tracking , using a calibrated cameraand estimating only its orientation and
position
0
34
11
P
P
B is the eigen vector of B corresponding to the smallest eigenvalue of B
]|[ tR
PKtR 1]|[
2.3.3 The Perspective-n-Point (PnP) Prob-lem
2.3.4 Pose estimation from a 3D Plane
The relation between a 3D plane and its image
projection can be represented by a homogeneous 3x3
matrix(homography matrix)
Let us consider the plane
0Z
MPm~~
1
1
1
0
21
321
Y
X
H
Y
X
tRRK
Y
X
tRRRK
2.3.4 Pose estimation from a 3D Plane
The matrix H can be estimatedfrom four correspondencesusing a DLT algorithm
the translation vec-tor
last column is given by the cross-product since the columns of R must be orthonormal
ii mM
tRRKH 21 t
3R 21 RR
2.3.5 non-Linear Reprojection Error
i
ii
tR
mMPdisttR ,~
]|[ 2
]|[
minarg
finding the pose that minimizes a sum of residual errors
2.4 Least-Squares Minimiza-tion Techniques ir
2.4.1 Linear Least-Squares
the function is linear
the camera pose parameters
the unknowns of a set of linear equationsin matrix form as
can be estimated as
f
p
TT AAAA 1)( pseudo-inverse of A
p bAp
bAp
2.4.2 Newton-Based Minimization Algo-rithms
the function is not linear
algorithms start from an initial estimate of the minimum and update it iteratively
is chosen to minimize the residual at itera-tion
and estimated by approximating to the first order
f
iii pp 1
0p
iiii Jpfpf )()(
i 1if
2.4.2 Newton-Based Minimization Algo-rithms
Jacobian matrixthe partial derivatives of all these functions
n
mm
n
x
y
x
y
x
y
x
y
J
1
1
1
1
mn RRf :
bpf ii
)(minarg
i
i
i
J
J
bJpf
minarg
)(minarg
iTT
i JJJ 1)(
iTT
i JIJJ 1)(
I stabilizes the begav-ior
inliers
data whose distribution can be explainedby some set of model parameters
outliers
which are data that do not fit the model
the data can be subject to noise
M-estimatorsgood at finding accurate solutionsrequire an initial estimate to converge correctly
RANSACdoes not require such an initial estimatedoes not take into account all the available datalacks precision
2.5 Robust Estimation
2.5.1 M-Estimators
least-squares estimation the assumption that the observations are in-
dependent and have a Gaussian distribution
Instead of minimizingi
ir2
ir are residual er-rors is an M-estimator that reduce the influence
of outliers
i
ir )(
2.5.1 M-Estimators
otherwisec
xc
cxifx
xHub
2
2)(
2
otherwisec
cxifc
xc
xTuk
6
116)(
2
322
Huber estimator
Tukey estimator
Huber estimator : linear to reduce the influence of large residual errors
Tukey estimator : flat so that large residual errors have no influ-ence at all
2.5.1 M-Estimators
2.5.2 RANSAC
samples of data points are randomly selected estimate model parameters find the subset of points (consistent with
the estimate) the largest is retained
and refined by least-squares minimization
N n
ip
SSi
iS
Sa set of mea-surements nthe model parameters require a
minimum of
2.5.2 RANSAC linear least-square esti-mation
2.5.2 RANSAC random sampling
2.5.2 RANSAC random sampling
2.5.2 RANSAC random sampling
2.5.2 RANSAC random sampling
estimating the density of successive states
in the space of possible camera poses.
2.6 Bayesian Trackingts
Thank you for your attention