Embed Size (px)
Citation preview
1
Color-Based Image Salient Region Segmentation Using Novel Region Merging Strategy
IEEE Transaction on Multimedia 2008Yu-Hsin Kuan, Chung Ming Kuo, and Nai-Chung Yang
授課教授 連震杰 教授
指導教授 吳宗憲 教授
實驗室 多媒體人機通訊實驗室
組員 P78961265 林仁俊P78971155 魏文麗P76971191 劉家瑞
2
OutlineIntroductionRelated workThe proposed method
Dominant color extraction and image quantization
Region merging strategyExperimental resultsConclusion
3
Introduction
4
Introduction (cont.)Color images are extensively used in multimedia
applications (retrieval, index).Low-level visual features such as color, shape,
texture (i.e. global features) has received much attention in recent years.Retrieve too many unrelated imagesPerformances are unsatisfactory
5
Introduction (cont.)High-level semantic descriptors (object, scene,
place) should be more consistent with human perception.
How to narrow down the gap between low-level features and human perception?Use spatial local features instead of global features
of imagesThe main purpose of this paper
Find the salient regions that are relatively meaningful to human perception
6
Related workWhat is salient region?
It should be compact , complete and significant enough.
(a) (b)
7
Related work (cont.)Region-based methods [1]
To depend on initial seedsOver-segmentation
Boundary-based methods [2]Noise, unconnected edgesOver-segmentation
Hybrid –based methods [3]Integrate the region and edge informationEnhance the drawbacks
8
Related work (cont.)Histogram-based methods [4]
Generally deal with gray-level imagesColor images represented by 3-D histogramSelect a global threshold or dominant color in 3-D
space is difficultGraph-based methods [5]
By minimizing the weight that cut a graph into sub-graphs
High computational complexity
9
The proposed method
10
The proposed method (cont.)Dominant color extraction and image
quantizationThe dominant colors are extracted based on
nonparametric density estimation
1
1( ) ( )
N
ii
f x K x xN
2 2
1
2 2
2
1( )
2xK x e
Kernel Density Estimator
X is sample dataN is the total pixel number of imageσis the bandwidth for the kernel
11
The proposed method (cont.)
3 local maxima 3 local maxima 3 local maxima
12
Dominant color extraction ─ Q&AQuestion 1:
Why Gaussian smoothingAnswer :
避免過多的 local maxima(too many dominant color – over segmentation)
Question 2: 的改變，對 histogram 的影響 ?
Answer :會有 over smoothing ，或是不夠 smooth 的情形發生
13
The proposed method
14
The proposed method (cont.)
Dominant color extraction and image quantization
1 Y1 U1 V1
2 Y1 U1 V2
3 Y1 U1 V3
4 Y1 U2 V1
5 Y1 U2 V2
6 Y1 U2 V3
7 Y2 U1 V1
8 Y2 U1 V2
9 Y2 U1 V3
10 Y2 U2 V1
11 Y2 U2 V2
12 Y2 U2 V3
Dominant colors
15
The proposed method (cont.)
Dominant color extraction and image quantization
1 Y1 U1 V1
2 Y1 U1 V2
3 Y1 U1 V3
4 Y1 U2 V1
5 Y1 U2 V2
6 Y1 U2 V3
7 Y2 U1 V1
8 Y2 U1 V2
9 Y2 U1 V3
10 Y2 U2 V1
11 Y2 U2 V2
12 Y2 U2 V3
1 Y1 U1 V1
2 Y1 U1 V2
3 Y1 U2 V1
4 Y1 U2 V2
5 Y2 U1 V1
6 Y2 U1 V2
7 Y2 U2 V1
It may cause too many candidates of dominant colors.
We eliminate the candidates that the image pixels assignment is lower than a pre-defined threshold.
16
The proposed method (cont.)
Source image Quantized image
17
112/04/19
The proposed method (cont.)
18
The proposed method (cont.)
Region merging strategyImportant index computation
1 1 1
i ij j
i i
i ij j
R Rij m mn
R Rj i j
N NImp R
N N
The number of pixels
Total number of pixels with color label i
Total number of pixels of an image (image size)
A region with color label i, Region index j
aa
a
bb
1
2
3
21
c
19
The proposed method (cont.)
Region merging strategyThreshold : Tm Ti
j mImp R
Tij mImp R
Segmentation result
Merge into an adjacent region
20
The proposed method (cont.)
21
The proposed method (cont.)Region merging strategy
Attraction computation
Assume is a region to be merged and are its neighboring regions
1 22
mmF G
D
1 2
1 2 21 2
,,
R RRegionSize RegionSizeAttraction R R
ColorDistance R R
a , ,b c d
2,
,kRegionSize
Attraction a kColorDistance a k
22
The proposed method (cont.)
Region merging strategyAttraction computation
( ( , )), , ,
2d
max d a kT k b c d
2 , , ,
,, , ,
d
d
max d a k d a k TColorDistance a k
d a k d a k T
1 2 1 2 1 2
2 2 2
1 2, R R R R R Rd R R y y u u v v
23
The proposed method (cont.)
Initial region After region merging strategy Final segmentation result
24
Experimental results
Two parameters need to be presetBandwidth of the convolution kernelMerge threshold
For CIF format images , the average speed is around 0.6 second for each imagePentium 4 PC , 2.66 GHz CPU with 512MB RAM
Tm
25
Experimental results (cont.)
(a) Source image (b) After quantized and region merge (c) Segmentation result
(a)(c)(b) (c)(a) (b)
26
Experimental results (cont.)
Source image Over-segmentation[25]
Our method
Source image Over-segmentation[25] Our method
D. Comaniciu and P. Meer, “Robust analysis of feature spaces: color image segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997
27
Conclusion
The proposed approach efficiently extracts salient regions in color images.
Segmentation results satisfied our definition of saliency.
Effectively addressed the over-segmentation problem.
28
Reference [1] M. G. Montoya, C. Gil, and I. Garcia, “The load unbalancing problem for region
growing image segmentation algorithms,” J. Parallel Distrib. Comput., vol. 63, pp. 387–395, 2003
[2] W. Y. Ma and B. S. Manjunath, “Edge flow: a technique for boundary detection and image segmentation,” IEEE Trans. Image Process., vol. 9, no. 8, pp. 1375–1388, Aug. 2000
[3] T. Gevers, “Adaptive image segmentation by combining photometric invariant region and edge information,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 6, pp. 848–852, Jun. 2002
[4] H. D. Cheng, X. H. Jiang, and J. Wang, “Color image segmentation based on homogram thresholding and region merging,” Pattern Recognit., vol. 35, pp. 373–393, Feb. 2002
[5] A. Tremeau and P. Colantoni, “Regions adjacency graph applied to color image segmentation,” IEEE Trans. Image Process., vol. 9, pp. 735–744, Apr. 2000
29
Thank you~
