Monocular Model-Based 3D Tracking of Rigid Ob-jects: A Survey
2008. 12. 15.백운혁
Chapter 4. Natural Features, Model-Based Tracking
Agenda
Monocular Model-Based 3D Track-ing of Rigid Objects : A Survey
Chapter 4. Natural Features, Model-Based Tracking 4.1. Edge-Based Methods 4.2. Optical Flow-Based Methods 4.3. Template Matching 4.4. Interest Point-Based Methods 4.5. Tracking Without 3D Models
4.1 Edge-Based Methodsstraight line segments and to fit the model outlines
4.1.1 RAPiD
4.1.1 RAPiD
Origin
Control point
Control point in camera coordinates
Motion
z
y
x
T
T
T
T
z
y
x
P
P
P
P
Rtp
Z
Y
X
PTM
4.1.1 RAPiD
RPtTM
PtMM IR
)( PtKmm
pWmm
z
y
x
z
y
x
t
t
tp
4.1.1 RAPiD
IRiii lpWn ~ )(~ mmnl T dis-
tance
2)~(minarg ii
iip
lpWnp
is vector made of the dis-tances
L il
pAL
ALAAp T 1)(
4.1.2 Making RAPiD Robust
Minimize the distance
Control points lying on the same object edge are grouped into primitives. And a whole primitive can be rejected from the pose es-timation.
RANSAC methodology
The number of edge strength maxima visible
ji
ijipp
mMPdistp,
)),((minarg
4.1.3 Explicit Edge Extraction
),,,( lccX yx The middle point, the orientation and the length of the segment
mX Of a model seg-ment
dX Of a an extracted seg-ment
)()()( 1dmdm
Tdm XXXXd
Is the covariance matrix
Mahalanobis distance
The pose is then estimated by minimizing
i
im
id
id
Tim
id pXXpXX ))(())((
p
4.2 Optical Flow-Based Meth-ods
dtv
umm
m Its corresponding location in the next image
m The projection of a point in an image at time
I t
4.2.1 Using Optical Flow Alone
Normal optical flow
For large motions Causes error accumulation
0)(,
t
Imm
v
I
u
I
4.2.2 Combining Optical Flow and Edges
To avoid error accumulation0 tIpB
B Depends of the pose and the image spatial gradi-ents at time
p t
tI Is a vector made of the temporal gradient at the cho-sen locations
4.3 Template MatchingTo register a 2D template to an image under a family of deformations
4.3.1 2D Tracking
j
jjt mTpmfIpO 2))());((()(
To find the parameters of some deformation
That warps a template into the input image
p f
T tI
j
jjt mTpmfIpO 2))());((()(
iAi
is the pseudo-inverse of the Jacobian matrix of computed at A J ));(( pmfI jt jP
ITT
i JJJ 1)(
4.4 Interest Point-Based MethodsUse localized features
Rely on matching individual features across images and are therefore easy to robustify against partial occlusions or matching errors
4.4.1 Interest Point Detection
Harris-Stephen detector / Shi-Tomasi detector
The pixels can be classified from the behavior of the eigen values of
2
2
vvu
vuu
III
IIIZ
The coefficients of are the sums over a window
of the first derivatives and of image intensi-ties
with respect to pixel coordinates
Z
uI vI
),( vu
Z
4.4.2 Interest Point Matching
to use 7x7 correlation windows reject matches for which measure is less than 0.8 search of correspondents for a maximum movement
of 50 pixels
Kanade-Lucas-Tomasi tracker
Keep the points that choose each other
2))());((( j
jiijf mTpmfI
4.4.3 Pose Estimation by Tracking Planes Pose Estimation for Planar Structures
010
121 w
tt
tt
tw HHHHH
)),(( 11 IdjydixI
Thanks for your attention