Pose Estimation 2010. 3. 16. TUE.

Kim Kyungkoo

Active Grasp

2

• Introduction

• Pose Estimation– Object modeling with features– Real-time pose estimation

• Demo

• Future works

Contents

Introduction• Importance of object recognition and pose estimation

3

Pose Estimation• Problem Definition

– Robot knows• The target object to grasp• The corresponded 3D model• The grasp point on a 3D model

– BUT! Do not know• The grasp point in real-environment

4

Orientation matching between an object and a 3D model is needed

Pose Estimation• System overview

– Object modeling

– Automatic pose estimation

5

Stereo CameraLive video

Tracking

Reconstruction

Partial modelModel features

Transformation

Stereo CameraLive video

Featurematching

Pose estimation

3D model of an object

Object Modeling with features• Object Modeling process

6

2D image

3D depth Image

Disparity

Bi-layer Segmentati

on

2D image

3D depth Image

Object Segmentati

on

Object depth image

SURF Feature Tracking

2D image

3D depth Image

Merged Object

Depth image

Merging

DisparityImage

Homogeneous Matrix

Calculation

Merged Foreground Depth image

Merged Image Set

Captured Image Set

Accumulated Image Set

Depth Image Reconstruction

Object Modeling with features• Object feature list creation during modeling process

– Features• Using SURF algorithm to extract features• Each feature consist of a 3D coordinate and a descriptor

– Storing features extracted from object region of each frame• As the system extracts features from each image, it accumulates

the features with a previous feature list– It stores all features for the first image in image stream

7

A 3D Featur

e

SURFFeature Match

Updated 3D

Feature list

Transformed 3D

Feature list

Matched?YES NO

Add feature descriptor into same

ID

Create new ID for corresponding

points

Feature list creation on an object• Example of feature list

8

ID of corresponding point Feature information Feature information Feature information

1 (23, 25, 100, desc) (24, 24, 99, desc)

2 (60, 11, 89, desc) (58, 11, 90, desc) (61, 10, 90, desc)

3 (201, 5, 120, desc) (201, 3, 121, desc) (203, 5, 121, desc)

4 (21, 15, 110, desc)

5 (81, 93, 81, desc) (80, 95, 79, desc)

6 (101, 115, 120, desc) (101, 116, 119, desc)

7 (356, 345, 80, desc)

Real Time Pose Estimation• Feature matching between feature list of an object and

features of current image– Using SURF feature extraction and matching algorithm– Each feature consist of a 3D coordinate and a descriptor– Acquisition of 3D corresponding points

• Transformation– The 3D model of an object is transformed to fit a current image

using 3D corresponding points

– Method?

9

Pose Estimation of current view• Transformation of the 3D model for pose estimation

– Using three corresponding points– Calculate the best transformation matrix with three correspond-

ing points using RANSAC algorithm

10

Pose Estimation of current view• Transformation of the 3D model for pose estimation

11

H

Pose Estimation of current view• Transformation of the 3D model for pose estimation

12

3 쌍의 corresponding points 중 random 하게 한 쌍을 선택하여 선택된 각 점을 3 차원 공간상 0,0,0

으로 이동

T1 T2

Pose Estimation of current view• Transformation of the 3D model for pose estimation

13

남은 2 쌍의 corresponding points 중 한 쌍을 선택하여 그 점이 같은 축 위에 존재 하도록 회전

R1

Pose Estimation of current view• Transformation of the 3D model for pose estimation

14

같은 축 위로 회전된 corresponding point 를 기준으로 scaling

Pose Estimation of current view• Transformation of the 3D model for pose estimation

15

마지막 남은 corresponding point 를 다른 쪽에 맞도록 회전

Pose Estimation of current view• Transformation of the 3D model for pose estimation

1. Choose three corresponding points randomly2. Calculate a transformation matrix3. Transform all the corresponding point of model using the

transformation matrix4. Sum the distance between each corresponding point5. Repeat 1st to 4th process6. Select the transformation matrix which contains minimum dis-

tance summation value7. Transform all the point of an object model using the inverse

matrix of the selected transformation matrix in 6th process

16

Demo• Modeling process

17

Demo• Pose estimation

18

Future Works• Accuracy

• Transformation

• Feature list

19