陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题： Combining powerful local and...

陈 静2009.09.04JDL视觉建模与识别组

文献列表

标题： Combining powerful local and global statistics for texture description作者： Y. Xu, S.B. Huang, H. Ji, C. Fermuller

Paper I — #0872 Paper I — #0872

Paper II — #0600 Paper II — #0600

标题： Appearance-based Keypoint Clustering作者： F. Estrada, P.Fua, V.Lepetit, S. Susstrunk

Appearance-based Keypoint Clustering Paper I

1st Author——Francisco J. Estrada

• Biography– University of Toronto at Scarborough

• Department of Computer and Mathematical Sciences

– Ecole Polytechnique Federale de Lausanne (EPFL)

• Images and Visual Representation Group (IVRG)

• Computer Vision Lab (CVLAB)

– Centre for Vision Research (CVR) at York University

• Elderlab

– University of Toronto (UofT)• Computational Vision group in the

department of Computer Science过去

现在

From: http://www.cs.utoronto.ca/~strider

1st Author——Francisco J. Estrada

• Research Interest & Publications– Perceptual Grouping

F. J. Estrada, P. Fua, V. Lepetit, and , S. Süsstrunk, “Appearance Based Keypoint Clustering”,In CVPR, 2009.

– Image SegmentationF. J. Estrada, and A. D. Jepson, “Benchmarking Image Degmentation Algorithms”, In IJCV,2009.

– Image ProcessingF. J. Estrada, D. J. Fleet, and A. D. Jepson, “ Stochastic Image Denoising”,In BMCV, 2009.

C. Fredembach, F. Estrada, and S. Süsstrunk, “Memory Colour Segmentation and Classification Using Class-specific Eigenregions”, accepted to IEEE TIP, 2009

R. Achanta, S. Hemami, F. Estrada, and S. Süsstrunk, “Frequency-tuned Salient Region Detection”, In CVPR, 2009

C. Fredembach, F. Estrada, and S. Süsstrunk, “Segmenting Memory Colours”, Color Imaging Conference, pp. 315-320, 2008

R. Achanta, F. Estrada, P. Wils, and S. Süsstrunk, “Salient Region Detection and Segmentation”, International Conference on Computer Vision Systems, pp. 66-75, 2008

– Single View ReconstructionP. Denis, J. Elder, and F. Estrada, “Efficient Edge-Based Methods for Estimating Manhattan Frames in Urban Imagery”, In ECCV, 2008

2009-8-27 中国科学院计算技术研究所 JDL实验室

5

From: http://www.cs.utoronto.ca/~strider

2nd Author-Pascal Fua

• Biography ：– Ecole Polytechnique , Paris, 1984.– PhD, Université d'Orsay, 1989.– Chair for Computer Vision Lab,since 2002

• Current Research Interests– Shape modeling and motion recovery.– Human body modeling.– Optimization algorithms for image processing and image

registration.– Automated feature extraction.– Robotics and Augmented Reality applications.

• Publication ：– 2009 ： PAMI 5’, CVPR 7’, IJCV 1’, ICCV 2’

2009-8-27 中国科学院计算技术研究所 JDL实验室

6

3rd Author and 4th Author

第四作者： Sabine SusstrunkDirectory of the Images and Visual Representation Group (IVRG) since 1999.Reserch areas:Computational photography, color imaging, image quality metrics, image indexing and archiving

第三作者： Vincent LepetitResearch and Teaching Associate IN CVLabResearch Area:Object Detection and RecognitionRigid and Deformable 3D Tracking and Registration,Biomedical Applications, Augmented Reality.

英文摘要• We present an algorithm for clustering sets of detected

interest points into groups that correspond to visually distinct structure.

• Through the use of a suitable colour and texture representation, our clustering method is able to identify key points that belong to separate objects or background regions.

• These clusters are then used to constrain the matching of key points over pairs of images, resulting in greatly improved matching under difficult conditions.

• We present a thorough evaluation of each component of the algorithm and show its usefulness on difficult matching problems.

2009-8-27 中国科学院计算技术研究所 JDL实验室

8

中文摘要• 主要内容

– 聚类算法：将各关键点按视觉结构来进行分组• 关键技术

– Suitable 颜色和纹理表示 – 谱嵌入

• 主要应用– 关键点匹配

• 实验设计– 算法的各个组成部分的评估– 在匹配问题上的评估

本文要解决的问题• Keypoints Perceptual Grouping

Motivation

• lead to more efficient and reliable algorithms for– Object detection– Tracking– Scene reconstruction

总的流程

KeypointsDetectionKeypointsDetection

Feature RepresentationFeature Representation

Spectral Embedding

Mean-shift ClustingMean-shift Clusting

局部表观表示• 颜色信息表示

– RGB 颜色空间– 标准直方图

ip''''''''''''''

局部表观表示• 纹理信息表示

2 2 2

2

1

1

:

: the number of pixels in

, , , ,

1

i

i

T

j jij p

i

j

j

j p

C v v v vN

where

N p

I I I I Iv

x y x y x y

v vN

''''''''''''''

''''''''''''''

''''''''''''''

ip''''''''''''''

说明：1 。这种纹理表示方法有很好的性能，特 别是在物体分类问题2 。本文的贡献在于，将它引入多线索视觉分类算法中

相似度计算

2

2 2 ,

:

: colour histogram for patch

: colour histogram for patch

, 12

i

i jijk

i i

j j

i j

H k H kH H

H k

where

H p

H p

H k H kH k k n

''''''''''''''''''''''''''''''''''''''''''''''''''''''''

''''''''''''''

''''''''''''''''''''''''''''

''''''''''''''''''''''''''''

''''''''''''''''''''''''''''''''''''''''''

52

1

1...5

, ln

:

: ( 1...5)

0, 0

ij i j kk

k i j kk

k k kk i j

C C

where

C C k

C u C u u

和 的广义特征值，即 满足：

:ij i jd p p''''''''''''''''''''''''''''

块 和 中心间的欧式距离

22 22

2 2 2

ijij ijd

ija e

参数 通过最大化 F 来获得

*

.5* .5*

:

: :

p rF

p r

where

p r

正确率， 召回率

谱嵌入• 基本思想

1

2

4

.6

.15

3 .7

.15

.6

.2

.6 .2

.05 .0

基本假设：1.如果转移概率能表征节点间的相似度，那么Random walk 将会 favous 那些和它相似的节点2.属于相同物体的图像块的 Kernel diffusion 很相似

节点 Vi ：对应于图像块边 E(i, j) ：表示块 和块 之间的相似度jp

''''''''''''''ip

'''''''''''''' ip''''''''''''''

Normalizing each column of EM: Markov matrix

0j t jtd M d

''''''''''''''''''''''''''''

谱嵌入

Figure 2. Left: Original image and first 3 dimensions of the embedding.Second and third columns: similarity of sampled image patches (blue dots) with regard to a selected patch shown in red, brightness is proportional to similarity.

实 验

• Figure 5. Tracking of a lightly textured object on a heavily cluttered background.

• Left Column: Unconstrained matching between image pairs.

• Right Column: Our approach to

• matching.

• Bottom Row: Keypoint clusters produced by our algorithm for the four target images.

2009-8-27 中国科学院计算技术研究所 JDL实验室

18

实 验

2009-8-27 中国科学院计算技术研究所 JDL实验室

19

Figure 6. Tracking a cheetah against a complex background.

Left column: Standard unconstrained matching between a video frame and the reference image that appears in the top-left corner of Fig.1.

Middle column: Foreground/background clusters produced by our method for the target video frame.

Right column: Constrained matching results.

实 验

Figure 7. Additional examples of constrained SIFT matching. Top row: conventional SIFT matching.Bottom row: constrained matching using the clusters detected by our method.

Combining powerful local and global statistics texture description

Paper II

英文摘要• A texture descriptor is proposed, which combines local highly

discriminative features with the global statistics of fractal geometry to achieve high descriptive power, but also invariance to geometric and illumination transformations.

• As local measurements SIFT features are estimated densely at multiple window sizes and discretized. On each of the discretized measurements the fractal dimension is computed to obtain the so-called multifractal spectrum, which is invariant to geometric transformations and illumination changes. Finally to achieve robustness to scale changes, a multi-scale representation of the multifractal spectrum is developed using a framelet system, that is, a redundant tight wavelet frame system.

• Experiments on classification demonstrate that the descriptor outperforms existing methods on the UIUC as well as the UMD high-resolution dataset.

主要内容• 纹理描述 - 强的描述能力

– 局部判别性较高的特征– 全局的统计特征

总体思路

ComputeMulti-levelOrientationHistogram

ConstructMFS

ConstructTexture Description

总体思路

ComputeMulti-levelOrientationHistogram

ConstructMFS

ConstructTexture Description

Compute Multi-level Orientation Histogram

Figure 2. Orientation histogram when using the neighborhood of size 5 × 5.

Construct MFS

Figure 4. One rotated element and one mirror-reflected elementfrom the basic elements shown in the right column.

Figure 3. Basic elements of 29 orientation histogram templates

Construct texture description

基础知识• 小波分解 - 》卷积

基础知识• 卷积 - 》矩阵乘法

mnm

Construct texture description

实 验

实 验

致谢

向被我打扰过的洪晓鹏师兄、李安南师兄、翟德明师姐和阚美娜师姐表示感谢

感谢各位在这听我“瞎扯”