Content-Based Image Retrieval - Approaches and Trends of the New Age

Ritendra Datta, Jia Li, and James Z. WangThe Pennsylvania State University

MIR2005

INTRODUCTION 為什麼 image 無法處理的像 text 一樣好

Text is man’s creation, images are a mere replica of what man has seen

Interpretation of what we see is hard to characterize

visual similarity != semantic similarity CBIR has grown tremendously after 2000,

not just in terms of size, but also in the number of new directions explored

INTRODUCTION

INTRODUCTION The theoretical foundation behind ho

w we humans interpret images is still an open problem A brief scanning of about 300 relevant pa

pers published in the last five years revealed that less than 20% were concerned with applications or real-world systems

CBIR 領域研究方向 Feature Extraction Approaches to Retrieval Annotation and Concept Detection Relevance Feedback and Learning Hardware and Interface Support

Feature Extraction 如何抽 Color Feature

“An Efficient Color Representation for Image Retrieval” ( 比傳統 histograms 好 )

“Multiresolution Histograms and Their Use for Recognition” ( 用在 textured image)

“Image retrieval using color histograms generated by Gauss mixture vector quantization” ( 利用 GMVQ 抽 color histogram)

Feature Extraction Color + Texture 抽取

“Wavelet-Based Texture Retrieval Using Generalized Gaussian Density and Kullback-Leibler Distance”

Shape “Shape Matching and Object Recognition Usi

ng Shape Contexts” (is fairly compact yet robust to a number of geometric transformations)

Feature Extraction Segmentation

“Normalized Cuts and Image Segmentation” ( 最重要的方向之一 )

“Blobworld: Image Segmentation Using Expectation-maximization and Its Application to Image Querying” ( 我之前用過的方法 )

“Segmentation of Brain MR Images Through a Hidden Markov Random Field Model and the Expectation-Maximization Algorithm” ( 處理medical imaging)

Feature Extraction 線條相似度

“Image retrieval using wavelet-based salient points”

如何選擇 feature Application-specific feature sets ( 最直觀的 ) “SIMPLIcity:Semantics-Sensitive Integrated M

atching for Picture Libraries” (semantics-sensitive feature selection)

“Feature Selection for SVMs” ( 用 classifier)

Approaches to Retrieval Region based image retrieval

“A Scalable Integrated Region-Based Image Retrieval System”

region-based querying (BlobWorld) Vector quantization (VQ) on image blocks

“Keyblock: An Approach for Content-based Image Retrieval” (generate codebooks for representation and retrieval, taking inspiration from data compression and text-based strategies)

Approaches to Retrieval Windowed search

“Object-Based Image Retrieval Using the Statistical Structure of Images” (more effective than methods based on inaccurate segmentation)

Anchoring-based image retrieval “A Study of Image Retrieval by Anchoring” (A

nchoring is based on the idea of finding a set of representative “anchor” images and deciding semantic proximity between an arbitrary image pair in terms of their similarity to these anchors)

Approaches to Retrieval Probabilistic frameworks for image re

trieval “A Probabilistic Architecture for Conten

t-based Image Retrieval”

Annotation and Concept Detection

Supervised classification “Image Classification for Content-Based Index

ing” (involving simple concepts such as city, landscape, sunset,and forest, have been achieved with high accuracy)

Translation approach “Object recognition as machine translation: L

earning a lexicon for a fixed image vocabulary” ( 我們在 clef 2004 就是 follow 這方法 )

Annotation and Concept Detection

為何如此困難 We humans segment objects better than machi

nes, having learned to associate over a long period of time, through multiple viewpoints, and literally through a “streaming video” at all times

The association of words and blobs become truly meaningful only when blobs isolate objects well

Relevance Feedback and Learning

“Relevance Feedback in Image Retrieval: A Comprehensive Review”

Problems One problem with RF is that after every round o

f user interaction, usually the top results with respect to the query have to be recomputed

Another issue is the user’s patience in supporting multi-round feedbacks

REAL-WORLD REQUIREMENTS

Performance Semantic learning Volume of Data Concurrent Usage Heterogeneity Multi-modal features User-interface Operating Speed System Evaluation

CURRENT RESEARCH TRENDS

Journals IEEE T. Pattern Analysis and Machine Intellige

nce (PAMI) IEEE T. Image Processing (TIP) IEEE T. Circuits and Systems for Video Techno

logy (CSVT) IEEE T. Multimedia (TOM) J. Machine Learning Research (JMLR) International J. Computer Vision (IJCV)

CURRENT RESEARCH TRENDS

Pattern Recognition Letters (PRL) ACM Computing Surveys (SURV)

Conferences IEEE Computer Vision and Pattern Recognition

(CVPR) International Conference on Computer Vision (IC

CV) European Conference on Computer Vision (ECC

V) IEEE International Conference on Image Processi

ng (ICIP)

CURRENT RESEARCH TRENDS

ACM Multimedia (MM) ACM SIG Information Retrieval (IR) ACM Human Factors in Computing Systems (C

HI)

CONCLUSIONS We have presented a brief survey on work

related to the young and exciting fields of content-based image retrieval and automated image annotation, spanning 120 publications in the current decade

We have laid out some guidelines for building practical, real-world systems