Upload
upton
View
45
Download
0
Embed Size (px)
DESCRIPTION
Object Recognition by Discriminative Combinations of Line Segments, Ellipses and Appearance Features. Professor: S. J. Wang Student : Y. S. Wang. Outline. Background System Overview Shape-Token Code-Book of Shape-Token Code-Word Combination Hybrid Detector Experimental Result - PowerPoint PPT Presentation
Citation preview
1
Object Recognition by Discriminative Combinations of Line Segments,
Ellipses and Appearance Features
Professor: S. J. WangStudent : Y. S. Wang
2
OutlineBackgroundSystem OverviewShape-TokenCode-Book of Shape-TokenCode-Word CombinationHybrid DetectorExperimental ResultConclusion
3
BackgroundContour Based Detection Method
Problem of Contour Fragment:◦Storage requirement is large for training.◦Slow matching speed.◦Not scale invariant.
Solution provided is Shape-Token.
4
System Overview
5
Shape TokenWhat is Shape-Tokens?Constructing Shape-TokensDescribing Shape-TokensMatching Shape-Tokens
6
What is Shape-Tokens?Use the combination of line and ellipse
to represent the contour fragments.◦Line for line.◦Ellipse for curve.
Example:
Why shape-tokens?◦Several parameters are enough for us to
describe the contour fragment.
7
Constructing Shape-TokensExtract Shape Primitives of line
segments and ellipses by [16] [17]. Pairing reference primitive to its
neighboring primitive. ◦Different type combination:
Take ellipse as reference.◦Same type combination:
Consider each as reference in turn.Three types of Shape-Tokens:
◦Line-Line, Ellipse-Line, Ellipse-Ellipse.
8
Constructing Shape-TokensLine-Line
◦Combine neighboring line which has any point falling in trapezium searching area.
Ellipse-Line & Ellipse-Ellipse◦Circular Search Area. Consider
primitives has any point within searching area and weakly is connected to reference ellipse.
9
Describing Shape-Tokensd
◦ : Orientation of a Primitive.◦ : Unit vector from center of reference
primitive to center of its neighbor.
◦ : Distance between centers of primitives.◦ : Length and Width for each primitives.
10
Matching Shape-TokensDissimilarity Measure (Shape
Distance)
11
Matching Shape-TokensMore general for multiple scale
matching◦Normalize descriptor against object
scale
12
Codebook of Shape-TokensExtracting Shape-Tokens inside
bounding boxes of training images.
Producing Code-words◦Clustering by Shape◦Clustering by Relative Positions
Selecting representative code-words into codebook for specific target object.
13
K-Medoid MethodSimilar to the k-means method.Procedure:
◦ Randomly select k of the n data points as medoids.
◦ Associate each data point to the closest medoid.◦ For each medoid m
For each non-medoid data point o Swap m and o and compute the total cost of the configuration.
◦ Select the configuration with the lowest cost.◦ Repeat the steps above until there is no change
in the medoid.
14
K-Medoid MethodFirst two steps
15
K-Medoid MethodThird to Fourth step
16
Clustering by ShapeMethod:
◦Use k-medoid method to cluster the shape-tokens for each type separately.
◦Repeat the step above until the dissimilarity value for each cluster is lower then a specific threshold. Metric: Dissimilarity Value: average shape
distance between the medoid and its members.
Threshold: 20% of the maximum of D(.).
17
Clustering by relative positionsTarget:
◦Partition the clusters obtained from previous step by to attain sub-clusters whose members have similar shape and position relative to the centroid of object.
◦ : vector direct from object centroid to the
shape-token centroid.Method: Mean-Shift.
18
Candidate Code-Word Medoid for each sub-cluster.Parameters:
Shape Distance Threshold :Mean shape distance of the cluster plus one standard deviation.
Relative Position Center :Mean of vectors of the sub-clusters members.
Radius :Euclidean distance between to of each sub-cluster member plus one standard deviation.
19
Candidate Code-WordsExample: the Weizmann horse
dataset.
20
Selecting Candidates into CodebookIntuition: Size of cluster.Problem: Lots of selected
candidates belong to background clutter.
What kind of candidates we prefer ?◦Distinctive Shape.◦Flexible enough to accommodate
intra-class variations.◦Precise Location for its members.
21
Selecting Candidates into CodebookInstead of using cluster size directly,
the author scores each candidate by a product “” consists of three values.◦Intra-cluster shape similarity value
“” where is the maximum of the range of shape distance for the type of candidate currently considered.
◦The number of unique training bounding boxes its members are extracted from.
◦Its value of .
22
Selecting Candidates into CodebookOne more problem left:
◦If use to choose the candidate directly, it may cause not ideal spatial distribution.
Solution: Radial Ranking Method
23
Selecting Candidates into CodebookExample: the Weizmann horse
dataset.
24
Code-Word CombinationWhy code-word combination ?
◦One can use a single code-word that is matched in test image to predict object location. => Less discriminative and easy to matched in background.
◦Instead, a combination of several code-words can be more discriminative.
25
Code-Word CombinationMatching a code-word
combination◦Way to match code-word
combination.Finding all matched code-word
combinations in training images◦Exhaustive set of code-word
combinations.Learning discriminative xCC (x-
codeword combination)
26
Matching a Code-Word Combination Criteria:
◦Shape Constraint :Shape distance between each code-word and shape-token in image should be less then shape distance threshold .
◦Geometric Constraint:Centroid prediction by all code-words in the combination concur.
27
Matching a Code-Word Combination Example:
28
Finding all matched code-word combinations in training imagesGoal:
Finding an exhaustive set of possible candidates of code-word combinations.
Method: (Similar to Sliding-Window Search)◦For each candidate window at scale
and location in image I, we try to find there is any match for each code-word or not. And the combination of each matched code-word will be a possible combination candidate.
29
Finding all matched code-word combinations in training imagesSpecify a variable to represent
the matching condition of a specific code-word .
30
Finding all matched code-word combinations in training imagesIf ,then we say that the code-
word is matched at scale and location .
Any combination of these matched code-word will produce a candidate combination.
Why not consider the geometric constraint?
31
Finding all matched code-word combinations in training images
32
Learning Discriminative xCCWe’d like to obtain a xCC which
satisfies the following three constraint.◦Shape Constraint :
Highly related Code-Book Establishment
◦Geometric Constraint: Object Location Agreement.
◦Structural Constraint :Reasonable code-word combination for different poses of object.
33
Learning Discriminative xCCExample:
34
Learning Discriminative xCCBinary Tree to represent a xCC.
◦Each node is a decision statement:
35
Learning Discriminative xCCAdaBoost Training Procedure to
produce one xCC from each iteration.
The Binary Tree depth “k” can be obtained by 3-fold cross validation.
36
Learning Discriminative xCCExample:
37
Learning Discriminative xCCExample:
38
Learning Discriminative xCCExample:
39
Hybrid Detector xMCCIncorporating SIFT as appearance
information to enhance the performance.
Procedure: (same as previous section)
40
Hybrid Detector xMCCExample:
41
Hybrid Detector xMCCExample:
42
Hybrid Detector xMCCExample:
43
Experimental ResultContour only result under
viewpoint change. (train on side-view only)
44
Experimental ResultContour only result for
discriminating similar shape object classes.
45
Experimental ResultCompare with Shotton [6] on
Weizmann Horse test set.
Shotton [6]: Use contour fragment, fixed number of code-words for each combination.
46
Experimental ResultWeizmann Horse Test Set.
47
Experimental ResultGraz-17 classes.
48
Experimental ResultGraz-17 dataset.
49
Experimental ResultHybrid-Method result
50
ConclusionThis article provide a contour based
method that exploits very simple and generic shape primitives of line segments and ellipses for image classification and object detection.
Novelty:◦Shape-Token to reduce the time cost for
matching and the need of memory storage.◦No restriction on the number of shape-tokens
for combinations.◦Allow combination of different feature types.