Intelligent Database Systems Lab N.Y.U.S.T. I. M. A cluster validity measure with a hybrid parameter search method for the support vector clustering algorithm

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

A cluster validity measure with a hybrid parameter search method

for the support vector clustering algorithm

Presenter : Lin, Shu-Han

Authors : Jeen-Shing Wang, Jen-Chieh Chiang

PR (2008)


N.Y.U.S.T.I. M.

2

Outline

Introduction of SVC Motivation Objective Methodology Experiments Conclusion Comments


N.Y.U.S.T.I. M.SVC

SVC is from SVMs SVMs is supervised clustering technique

Fast convergence

Good generalization performance

Robustness for noise

SVC is unsupervised approach

1. Data points map to HD feature space using a Gaussian kernel.

2. Look for smallest sphere enclose data.

3. Map sphere back to data space to form set of contours.

4. Contours are treated as the cluster boundaries.

3


N.Y.U.S.T.I. M.SVC - Sphere Analysis

To find the minimal enclose sphere with soft margin:

To solve this problem, the Lagrangian function:

4

a



5



Karush-Kuhn-Tucker complementarity:

6


N.Y.U.S.T.I. M.SVC -Sphere Analysis

To find the minimal enclose sphere with soft margin:

C : existence of outliers allowed

7

Wolfe dual optimization

problem a

Bound SV; Outlier


N.Y.U.S.T.I. M.SVC -Sphere Analysis

The distance (similarity) between x and a:

q : |clusters| & the smoothness/tightness of the cluster boundaries.

8

Mercer kernelKernel: Gaussian

a

Gaussian function:


N.Y.U.S.T.I. M.Motivation

9

Drawbacks of Cluster validation Compactness

Different densities or size

As the # of clusters increases, it will monotonic decrease

Separation

Irregular cluster structures


N.Y.U.S.T.I. M.Motivation

10

Their previous study Can handle

Different sizes Different densities Arbitrary shape

But…


N.Y.U.S.T.I. M.

Objectives – A cluster validity method and a parameter search algorithm for SVC

Auto determine the two parameter: Increasing q lead to increasing # of clusters

C regulates the existence of outliers and overlapping clusters

To Identify the optimal structure

11


N.Y.U.S.T.I. M.

Methodology- Idea

12

q is related to the densities of the clusters Each cluster structure corresponds to an interval of q Identify the optimal structure is equivalent to finding the

largest interval

N=64, max # of cluster = , 8 N


N.Y.U.S.T.I. M.

Methodology- Problem

13

How to locate overall search range of q How to detect outliers/noises How to identify the largest interval


N.Y.U.S.T.I. M.Methodology – Locate range of q

14

Lower bound

Upper bound: Employ K-Means to get clusters, and get variance of each clusters vi

N

Ascending order: cluster size

n =3, the biggest 3 clusters’ variance


N.Y.U.S.T.I. M.Methodology – Outlier Detection

Set q = qmax ,the tightest of q

15

outliersingleton

And we get Copt, remove these outlier


N.Y.U.S.T.I. M.Methodology – the largest interval

16

qopt


N.Y.U.S.T.I. M.Methodology – the largest interval

17

Fibonacci search: locate the interval wherethe cluster structure is the same

Bisection search

n: iteration


N.Y.U.S.T.I. M.

Methodology – Overview

18

Locate range of q

Outlier Detection

the largest interval


N.Y.U.S.T.I. M.

Experiments - Benchmark and Artificial Examples

19


N.Y.U.S.T.I. M.

Experiments - Outlier

20

Copt


N.Y.U.S.T.I. M.Experiments

21

？


N.Y.U.S.T.I. M.

22

Conclusions

A new measure: Inspired from the observations of q

Determine the optimal cluster structure with its corresponding range of q and C

qC


N.Y.U.S.T.I. M.

23

Comments

Advantage Inspired from observation of parameter

Drawback …

Application SVC

DBSCAN: MinPts / Eps

Documents

Intelligent Database Systems Lab N.Y.U.S.T. I. M. A cluster validity measure with a hybrid parameter search method for the support vector clustering algorithm