Embed Size (px)
Citation preview
T o m a s P a j d l a
presents work of
M. Bujnak, Z. Kukelova & T. Pajdla
Making Minimal Problems Fast
Czech Technical University PragueCenter for Machine Perception
M o t i v a t i o n
3 D R E C O N S T R U C T I O N
100 users1000 scenes2000 rec’s
3 D r e c o n s t r u c t i o n p i p e l i n e
K e y e l e m e n t s o f
3 D R E C O N S T R U C T I O N
R E L A T I V E C A M E R A P O S E
P R O B L E M
S O L U T I O N S 2004(6‐8)
D. NisterAn efficient solution to the five‐point relative poseIEEE PAMI, 2004.
H. Stewenius, C. Engels, and D. Nister. Recent developments on direct relative orientationISPRS J. of Photogrammetry and Remote Sensing, 2006.
Z. Kukelova, M. Bujnak, T. PajdlaPolynomial eigenvalue solution to the 5‐pt and 6‐pt relative pose problemsBMVC 2008
x1 x2
Algebraic equations
solve
R E L A T I V E C A M E R A P O S E P R O B L E M
5 matches are necessary and sufficient
1 2
3
4 5
1 2
3
45
M I N I M A L P R O B L E M S & R A N S A C
RANSAC: find F to maximize the # of good matchesselect 5 matches → compute F → record the support
A B S O L U T E C A M E R A P O S E
D E T E R M I N A T I O N
( w i t h u n k n o w n f )
Algebraic equations
solve
S O L U T I O N f o r u n k n o w n f 2008
M. Bujnak, Z. Kukelova, T. Pajdla A general solution to the P4P problem for camera with unknown focal lengthCVPR 2008
• completely general solution (even for the planar case)
• the non‐planar case in
Abidi, T. Chandra. A new efficient and direct solution for pose estimation using quadrangular targets: Algorithm and evaluation. PAMI 1995.
S O M E C L A S S I C A L P R O B L E M S
• 5‐pt calibrated camera relative pose• 6‐pt unknown f camera relative pose
2006 H. Stewenius, C. Engels, D. Nister. Recent developments on direct relative orientation. (Groebner basis)
• 6‐pt one‐known f camera relative pose
2001 M. Urbanek, R. Horaud, and P. Sturm. Combining Off‐ and On‐line Calibration of a Digital Camera. (Non‐minimal, Linear)
• 8‐pt unknown radial distortion camera relative pose
2007 Z. Kukelova, T. Pajdla A minimal solution to the autocalibration of radial distortion. (Groebner basis)
1. Problem formulation → algebraic equations
2. Solving algebraic equations
S O L V I N G M I N I M A L P R O B L E M S
Solving algebraic equations: what is (is not) difficult
1. Groebner Basis Solutions
2. Polynomial Eigenvalue Solutions
3. Making it fast(er)
S O L V I N G A L G E B R A I C E Q U A T I O N S
1+1=?
idealactio
n m
atrix
Quotient ring
xy2+7y+..
field
S O L V I N G A L G E B R A I C E Q U A T I O N S
!?
L O O K S A S “t h e M A T H E M A T I C S” !!!L E T’ S T A K E A N E N G I N E E R I N G A P P R O A C H
S O L V I N G A L G E B R A I C E Q U A T I O N S
1 equation, 1 variable
... a simple rule
→ companion matrix → eigenvalues
S O L V I N G A L G E B R A I C E Q U A T I O N S & GB
m equations, n variables (an example for m = 2 & n = 2)
→ Groebner basis(polynomials with the same solutions but easy to solve)
← how to get it ?
… a polynomial combination of
M A T R I X F O R M O F P O L Y N O M I A L S
A D D I N G M U L T I P L E S
G A U S S I A N E L I M I N A T I O N
Gaussian elimination
S O L V I N G P O L Y N O M I A L E Q U A T I O N S B Y C O N S T R U C T I N G G R O E B N E R B A S I S
A generalization of the Gaussian elimination
multiplication by scalars↓
multiplication by scalars & variables
S O L V I N G A L G E B R A I C E Q U A T I O N S
m equations, n variables
→ Groebner basis → generalizedcompanion matrix
S O L V I N G A L G E B R A I C E Q U A T I O N S
→ Groebner basis
m equations, n variables
→ generalizedcompanion matrix
→ eigenvectors
T H E D I F F I C U L T P A R T
no simple rule:
1. NP-complete in m, n (~ 3-coloring of graphs)(takes very long time to compute)
2. EXPSPACE-complete problem (needs huge space to remember intermediate results)
Equations → Groebner basis
1888 David Hilbert: Finitness theoremEvery ideal has a finite generating set
H I S T O R Y
1965 Bruno Burchberger: Groebner basesComputational procedure for solving systemsof polynomial equations(Extremely simple: 20 lines of Maple code!)
1999 Jean-Charles Faugere: F4 algorithmAn efficient computational tool for cryptography
1998 Hans Stetter: Multiplication matrixA stable numerical procedure via eigenvectors
1. “Standard” (Hironaka 1964) and Groebner (Burchberger 1965) bases
G B C O M P U T A T I O N A L G O R I T H M S
2. 1965: Buchberger’s algorithm
– a generalization of the Gauss-Jordan elimination – extremely simple: 7 (+ 13 for the rem division)
lines of code– not efficient – not good for numerical approximations
3. 1999 (2005): F4 (F5) algorithm (J.-C. Faugere)
– more efficient, more robust, more complex
G B C O M P U T A T I O N A L G O R I T H M S
Manipulation ... polynomial multiplication & pseudo-division
?
How to find coefficients?
Bucheberger’salgorithm(generalizedGauss elimination)
C O M P U T I N G G B M A Y B E V E R Y H A R D
have extremely simple Groebner basis
Example: 4 polynomials, 3 variables, degree ≤ 6, small integer coeffs
H O W E V E R
when computed by the Buchberger’s algorithm over the rational numbers w.r.t. the grevlex ordering x > y > z,the following polynomial appears during the computation:
~ 80,000 digits
W H Y ?C
o m
p l
e x
i t y
Input Groebner basis
General algorithms can construct all GBs but often generate many complicated polynomials
G e n e r a la l g o r i t h m
S p e c i f i ca l g o r i t h m
S P E C I F I C G B C O N S T R U C T I O N A L G’ S
Find a short path towards the GB which is independentfrom the actual coefficients, implement it eficiently.
Use floating-point arithmetics to do the manipulationsto avoid huge coefficients.
1.
2.
C O M P U T A T I O N
will run very long.
The problem is in remembering very long coefficients.
C O M P U T A T I O N
returns a very similar basis in a fraction of second
S H O R T P A T H T O G B
try P = 1, 2, 3, 5, 7, ..., 30011, 30013, 30029, ...
untill the result stabilizes (always does)
1. Use a computer algebra system (Macaulay2) to computethe Groebner basis in a finite field (fast!) for randomcoefficients and remember the path:
“lucky” prime number (always exists)
2. Remember the “path of construction” and hard-code it.
F L O A T I N G P O I N T A R I T H M E T I C S
Strictly speaking, polymomial manipulations must be done in exact arithmetics
Double canceling may fail when rounding occurs
in floating point arithmetics
F L O A T I N G P O I N T A R I T H M E T I C S
1. J.C. Faugere’s F4 algorithm better than Buchberger algortithm when rounding occurs.
2. For some computer vision problems rounding does notdestroy the result (but for some other problem does).
3. All manipulations have to be done with care and checkedon typical data in randomized experiments.
Offline
S O L V E R G E N E R A T O R
Z. Kukelova, M. Bujnak, T. Pajdla, Automatic Generator of Minimal Problem SolversECCV 2008,
Online solver
www.cmp.felk.cvut.cz/minimal
M A K I N G M I N I M A L S O L V E R S F A S T
P r o f i l i n g o n l i n e s o l v e r s
→ Let’s get rid of the general eigenvalue computation!
T H E F A S T E S T (U S E F U L) S O L V E R
D. Nister An efficient solution to the five‐point relative pose problemIEEE PAMI, 26(6):756–770, 2004
D. Nister. An efficient solution to the five‐point relative pose problem
S T U R M T h e o r e m
S T U R M T h e o r e m
If there are roots in an interval … split it … test again
S T U R M T h e o r e m
+
+
–
–
+
+
–
+
S T U R M T h e o r e m
+ only real roots are computed+ search only roots with meaningful values+ precision can be controlled
→ Often faster for many computer vision problems
S O L V I N G A L G E B R A I C E Q U A T I O N S
→ Groebner basis → generalizedcompanion matrix
↓1-variable polynomial (char. pol. of )
↓Sturm sequences
→ eigenvectors
A generalprocedure?
C L A S S I C A L M E T H O D S f o r C h a r . P o l.
• Krylov’s method (Cayley‐Hamilton)
• FGML (Basis convertion method)
• Faddeev‐Laverrier method (Traces of matrices)
• Danilevsky method (Transforming to a companion mtrx)
C L A S S I C A L M E T H O D S
D A N I L E V S K I M e t h o d
transform
Danilevsky
•14.2μs
mFGML
•13.7μs
Faddeev
•17.2μs
GB+eig
•61.2μs
Nister
•10.6μs
S P E E D
5‐pt Relative Pose
N U M E R I C A L S t a b i l i t y 4‐pt Absolute Pose + f
samples
A general procedure for 1‐variable equation → & Sturm seqncs
M. Bujnak, Z. Kukelova, T. Pajdla. Making Minimal Solvers Fast. CVPR 2012
Danilevsky
14.2μs
GB+eig
61.2μs
5‐pt Relative Pose 6‐pt Relative Pose + f 4‐pt Absolute Pose + f
Danilevsky
2.6μs
GB+eig
76.3μs
Danilevsky
7.4μs
GB+eig
27.4μs
> 4x > 7x > 2x
Nister 10.6 μs
C O N C L U S I O N
1. Groebner Basis technique can be always used (may be hard to compute)
2. Special Groebner Basis Solvers can be automatically generated
3. and 1-variable polynomial + Sturm sequences used in general
. . . A N D M O R E cmp.felk.cvut.cz/minimal
S O L V I N G P O L Y N O M I A L E Q U A T I O N S B Y T H E G R O E B N E R B A S I S M E T H O D
… I S A N A R T!
S I M P L E R S O L V E R S (F O R S O M E P R O B L E M S)
+ F O R A L L P R O B L E M S– H A R D T O I M P L E M E N T– ``B A D’’ N U M E R I C S
G R O E B N E RB A S I SS O L V E R
P O L Y N O M I A L E I G E N V A L U ES O L V E R
– F O R S O M E P R O B L E M S + E A S I E R T O I M P L E M E N T+ S T A N D A R D N U M E R I C S
1 U N K N O W N → E I G E N V A L U E S
1 equation, 1 variable
... a simple rule
→ companion matrix → eigenvalues
E I G E N V A L U E S S O L V E A L G E Q N S
linear bilinear (linear in λ)
Eigenvalues solve a special system of algebraic eqns via the hidden variable method
P O L Y N O M I A L E I G E N V A L U E S O L U T I O N
“Hide” y (the “hidden variable method”)
m equations, n variables
2 × 3 matrix →∞ sols !!! →!?
P O L Y N O M I A L E I G E N V A L U E S O L U T I O N
→ add
“Hide” y (the “hidden variable method”)
P O L Y N O M I A L E I G E N V A L U E P R O B L E M
Quadratic eigenvalue problem⇔
… generalization of
… can be rewritten as
Generalized eigenvalue problem
Higher order PolyEigs:A. Wallack, I. Z. Emiris, D. Manocha. MARS ‐ A Maple‐Matlab‐C Resultant‐based Solver.
S O M E C L A S S I C A L P R O B L E M S
• 5‐pt calibrated camera relative pose• 6‐pt unknown f camera relative pose
2006 H. Stewenius, C. Engels, D. Nister. Recent developments on direct relative orientation. (Groebner basis)
• 6‐pt one‐known f camera relative pose
2001 M. Urbanek, R. Horaud, and P. Sturm. Combining Off‐ and On‐line Calibration of a Digital Camera. (Non‐minimal, Linear)
• 8‐pt unknown radial distortion camera relative pose
2007 Z. Kukelova, T. Pajdla A minimal solution to the autocalibration of radial distortion. (Groebner basis)
P O L Y E I G S O L U T I O N S
• 5‐pt calibrated camera relative pose (easier implementation)• 6‐pt unknown f camera relative pose (better num, faster)
2008 Z. Kukelova, M. Bujnak, T. Pajdla. Polynomial eigenvalue solution to the 5‐pt and 6‐pt relative pose problems. BMVC 2008
• 6‐pt one‐known f camera relative pose (better num, faster)
2009 Z. Kukelova, M. Bujnak, T. Pajdla. 3D reconstruction from image collections with a single known focal length. ICCV 2009
• 8‐pt unknown radial distortion camera relative pose (similar)
Z. Kukelova, T. Pajdla. Unpublished.
6‐pt U N K N O W N f C A M E R A R E L A T I V E P O S E
Epipolar constraint
9 unknowns – 6 correspondences = 3 dimensional nullspace
Fundamental matrix is singular
Transfer F to the essential matrix
Constraints on E
• 10 x 3rd and 5th order polynomial equations in 3 variables and with 30 monomials
6‐pt U N K N O W N f C A M E R A R E L A T I V E P O S E
• Quadratic polynomial eigenvalue problem (in MATLAB)
5‐pt C A L I B R A T E D C A M E R A R E L A T I V E P O S E
Epipolar constraint
9 unknowns – 5 correspondences = 4 dimensional nullspace
10 x 3rd order polynomial equations in x, y, z with 20 monomials
Constraints on E
3rd order PolyEig problem
• PolyEig gives the same or somewhat more precise results than GB on synthetic noise free data sets.
• PolyEig 6‐pt solver is faster and outperforms all previous solutions in precision of the focal length estimation.
Failure ratio test – % scenes with precision worse than a given threshold
Relative error on synthetic data.
5‐pt & 6‐pt C A M E R A R E L A T I V E P O S E
PolyEig
C O N C L U S I O N
1. Groebner Basis technique can be always used (may be hard to compute)
2. Some interesting & important problems can be solved as Polynomial Eigenvalue Problems
a. Better numerical accuracyb. Easier implementation
3. Q: What can and what can’t be solved efficientlyas a PolyEig problem?
. . . A N D M O R E cmp.felk.cvut.cz/minimal
6‐pt 1 K N O W N f R E L A T I V E P O S E P R O B L E M
Formulation very similar to 6‐pt 2 unknown f’s
Somewhat simpler system leads to a generalized eigenvalue problem
6‐pt 1 K N O W N f R E L A T I V E P O S E P R O B L E M
6‐pt 1 K N O W N f R E L A T I V E P O S E P R O B L E M
From Google Street View to 3D City ModelsTomas Pajdla ([email protected])
Torii, M. Havlena, T. Pajdla. From Google Street View to 3D City Models. OMNIVIS 2009
Structure from Motion – A Sequential Approach
1. SIFT, SURF tracks2. Threading camera resec’s3. Tf‐idf based loop closing4. Bundle adjustment
3D Reconstruction
SFM→ Images (EXIFs, …) → cameras & accurate calibration & sparse pts
MVS→ point cloud, 3D surface, texture
