Pose Estimation 2010. 3. 16. TUE.
Kim Kyungkoo
Active Grasp
2
• Introduction
• Pose Estimation– Object modeling with features– Real-time pose estimation
• Demo
• Future works
Contents
Introduction• Importance of object recognition and pose estimation
3
Pose Estimation• Problem Definition
– Robot knows• The target object to grasp• The corresponded 3D model• The grasp point on a 3D model
– BUT! Do not know• The grasp point in real-environment
4
Orientation matching between an object and a 3D model is needed
Pose Estimation• System overview
– Object modeling
– Automatic pose estimation
5
Stereo CameraLive video
Tracking
Reconstruction
Partial modelModel features
Transformation
Stereo CameraLive video
Featurematching
Pose estimation
3D model of an object
Object Modeling with features• Object Modeling process
6
2D image
3D depth Image
Disparity
Bi-layer Segmentati
on
2D image
3D depth Image
Object Segmentati
on
Object depth image
SURF Feature Tracking
2D image
3D depth Image
Merged Object
Depth image
Merging
DisparityImage
Homogeneous Matrix
Calculation
Merged Foreground Depth image
Merged Image Set
Captured Image Set
Accumulated Image Set
Depth Image Reconstruction
Object Modeling with features• Object feature list creation during modeling process
– Features• Using SURF algorithm to extract features• Each feature consist of a 3D coordinate and a descriptor
– Storing features extracted from object region of each frame• As the system extracts features from each image, it accumulates
the features with a previous feature list– It stores all features for the first image in image stream
7
A 3D Featur
e
SURFFeature Match
Updated 3D
Feature list
Transformed 3D
Feature list
Matched?YES NO
Add feature descriptor into same
ID
Create new ID for corresponding
points
Feature list creation on an object• Example of feature list
8
ID of corresponding point Feature information Feature information Feature information
1 (23, 25, 100, desc) (24, 24, 99, desc)
2 (60, 11, 89, desc) (58, 11, 90, desc) (61, 10, 90, desc)
3 (201, 5, 120, desc) (201, 3, 121, desc) (203, 5, 121, desc)
4 (21, 15, 110, desc)
5 (81, 93, 81, desc) (80, 95, 79, desc)
6 (101, 115, 120, desc) (101, 116, 119, desc)
7 (356, 345, 80, desc)
Real Time Pose Estimation• Feature matching between feature list of an object and
features of current image– Using SURF feature extraction and matching algorithm– Each feature consist of a 3D coordinate and a descriptor– Acquisition of 3D corresponding points
• Transformation– The 3D model of an object is transformed to fit a current image
using 3D corresponding points
– Method?
9
Pose Estimation of current view• Transformation of the 3D model for pose estimation
– Using three corresponding points– Calculate the best transformation matrix with three correspond-
ing points using RANSAC algorithm
10
Pose Estimation of current view• Transformation of the 3D model for pose estimation
11
H
Pose Estimation of current view• Transformation of the 3D model for pose estimation
12
3 쌍의 corresponding points 중 random 하게 한 쌍을 선택하여 선택된 각 점을 3 차원 공간상 0,0,0
으로 이동
T1 T2
Pose Estimation of current view• Transformation of the 3D model for pose estimation
13
남은 2 쌍의 corresponding points 중 한 쌍을 선택하여 그 점이 같은 축 위에 존재 하도록 회전
R1
Pose Estimation of current view• Transformation of the 3D model for pose estimation
14
같은 축 위로 회전된 corresponding point 를 기준으로 scaling
Pose Estimation of current view• Transformation of the 3D model for pose estimation
15
마지막 남은 corresponding point 를 다른 쪽에 맞도록 회전
Pose Estimation of current view• Transformation of the 3D model for pose estimation
1. Choose three corresponding points randomly2. Calculate a transformation matrix3. Transform all the corresponding point of model using the
transformation matrix4. Sum the distance between each corresponding point5. Repeat 1st to 4th process6. Select the transformation matrix which contains minimum dis-
tance summation value7. Transform all the point of an object model using the inverse
matrix of the selected transformation matrix in 6th process
16
Demo• Modeling process
17
Demo• Pose estimation
18
Future Works• Accuracy
• Transformation
• Feature list
19