Object Trackinng PHd Thesis

Embed Size (px)

Text of Object Trackinng PHd Thesis

Master Erasmus Mundus in Color in Informatics and Media Technology (CIMET)

ObjectTracking:StateofTheArtandCAMSHIFTImprovementUsing MultidominantColorsTracking MasterThesisReport Presentedby

PriyantoHidayatullah anddefendedatthe

UniversityofJeanMonnetSaintEtienne,France nd 22 June2010 JuryCommittee: Prof.AlainTremeau Prof.JonYngveHardeberg FaouziAlayaCheikh,Ph.D JavierHernndezAndrs,Ph.D DamienMuselet,Ph.D EricDinet,Ph.D Supervisor: HubertKonik,Ph.D

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

Abstract

Object tracking is a wide area in which a lot of methods available and wide variety of applications. One of the applications would be tracking an object in a clickable hypervideo to enrich the interactivity of video application. In this thesis, some state of the art of object tracking methods are reviewed and closely observed. We then select one of object tracking state of the art methods to improve. Our selection goes to CAMSHIFT which has been very well accepted as one of the most prominent methods in object tracking which has real time speed performance and more suitable for clickable hypervideo. CAMSHIFT is very good for single hue object tracking and in the condition where objects color is different with backgrounds colors. In this thesis, we try to improve the robustness of CAMSHIFT for multihued object tracking and the situation where objects colors are similar with backgrounds colors. To improve robustness on the condition where objects colors are similar to backgrounds colors, we use object localization by selecting each dominant color object part using combination of Mean-Shift segmentation and region growing. Hue-distance, saturation and value color histogram are used to describe the object. We also track the dominant color object parts separately and combine them together to improve robustness of the tracking on multihued object. Our experiments showed that those methods improved CAMSHIFT significantly. This improvement hopefully will be useful for object tracking in clickable hypervideo. Keywords: Object tracking, CAMSHIFT, Segmentation, Mean-Shift, Hypervideo.

i

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

Table of ContentsAbstract ...................................................................................................................... i Table of Contents ....................................................................................................... ii Table of Figures ......................................................................................................... iv 1 1.1 2 2.1 2.2 Introduction ...................................................................................................... 1 The General Aim of The Master Thesis ....................................................... 1 Previous Work ................................................................................................... 2 Test Videos ................................................................................................ 2 Object Tracking Categorization .................................................................. 4

2.3 Corner Detector Combined with Optical Flow ............................................ 4 2.3.1 Corner detection .................................................................................... 5 2.3.2 Optical flow ........................................................................................... 5 2.4 2.5 Speeded Up Robust Features (SURF) ......................................................... 7 Mean Shift Tracking .................................................................................. 9

2.6 CAMSHIFT Tracking ................................................................................ 11 2.6.1 Color probability distribution and histogram back projection ............... 12 2.6.2 Mass center calculation ....................................................................... 13 2.6.3 CAMSHIFT advantages and disadvantages .......................................... 14 2.7 2.8 2.9 Local Binary Pattern ................................................................................ 15 Beyond Semi-Supervised Online Boosting Tracking ................................. 17 Method that We Choose ........................................................................... 19

2.10 CAMSHIFT/Mean-Shift Improvement in Literatures ...............................20 2.10.1 Mean-Shift tracking combined with texture histogram.....................20 2.10.2 CAMSHIFT and Mean-Shift combined with interest points .............. 21 2.10.3 CAMSHIFT improvement using new HSV model ............................. 24 2.10.4 CAMSHIFT improvement using hue-distance and saturation features25 2.10.5 CAMSHIFT with improvement of object localization........................ 27 2.10.6 CAMSHIFT improvement using adaptive background (ABCShift) .... 28 2.10.7 CAMSHIFT improvement by background subtraction...................... 31 2.10.8 The CAMSHIFT improvement method that we choose ..................... 32 2.10.9 The more specific aim of the master thesis ....................................... 32 3 Proposed Method ............................................................................................. 34 3.1 Object Localization .................................................................................. 34 3.1.1 Preprocessing ...................................................................................... 34 3.1.2 Image color transformation ................................................................. 36 3.1.3 Object Selection................................................................................... 36 3.1.4 Minimum and maximum values storing ............................................... 36 3.2 3.3 Object Modeling ...................................................................................... 37 Making Color Mask.................................................................................. 38

ii

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

3.4 3.5 3.6 4 4.1 4.2 5

Segmentation .......................................................................................... 38 Histogram Back Projection ...................................................................... 39 Tracking .................................................................................................. 39 Implementations ..................................................................................... 42 Experiments Setting ................................................................................ 43

Implementations and Experiments .................................................................. 42

Results and Discussions ................................................................................... 44 5.1 Results .................................................................................................... 44 5.1.1 First Experiment Results ..................................................................... 44 5.1.2 Second Experiment Results ................................................................. 46 5.1.3 Third Experiment Results.................................................................... 49 5.1.4 Forth Experiment Results.................................................................... 49 5.2 Discussion ............................................................................................... 52 5.2.1 Some Advantages ................................................................................ 52 5.2.2 Some Limitations ................................................................................ 52

6

Conclusions and Future Works ........................................................................ 54 6.1 6.2 Conclusions ............................................................................................. 54 Future Works .......................................................................................... 54

7

Bibliography .................................................................................................... 55

iii

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

Table of FiguresFigure 2.1 Test Videos. ............................................................................................... 2 Figure 2.2 Illustration of optical flow[28]. .................................................................. 6 Figure 2.3 Shi Tomasi corner detector also detect the background corners inside objects rectangle .................................................................................................................... 7 Figure 2.4 SURF Tracker Result. ................................................................................ 9 Figure 2.5 Intuitive description of Mean-Shift.[14] ................................................... 10 Figure 2.6 Summary of CAMSHIFT algorithm. ........................................................ 13 Figure 2.7 LBP and CS-LBP features for a neighborhood of 8 pixels [16]................... 15 Figure 2.8 Example of LBP calculation[16]............................................................... 16 Figure 2.9 LBP Tracker Result in second video.......................................................... 16 Figure 2.10 The core classifier system: detector, recognizer and tracker.[20] ............ 18 Figure 2.11 Comparing LBP Image and its back projection image. ............................ 21 Figure 2.12 SURF and CAMSHIFT 1......................................................................... 23 Figure 2.13 SURF and CA