A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

A fast nearest neighbor classifier based on

self-organizing incremental neural network(SOINN)

Neuron Networks (NN, 2008)

Presenter : Lin, Shu-HanAuthors : Shen Furao,, Osamu Hasegawa


N.Y.U.S.T.I. M.

2

Outline

Introduction Motivation Objective Methodology Experiments Conclusion Comments


N.Y.U.S.T.I. M.

3

Introduction - self-organizing incremental neural network (SOINN)

Distance: Too far

Node = prototype


N.Y.U.S.T.I. M.

4


Link age


N.Y.U.S.T.I. M.

5


Age: Too old


N.Y.U.S.T.I. M.

6


Run two times

Insert node if error is large

Cancel Insertion if insert is no use


N.Y.U.S.T.I. M.

7


Run two times

Delete outlier:Nodes without

neighbor(low-density assumption)


N.Y.U.S.T.I. M.Motivation

SOINN classifier (their first research in 2005) Use 6 user determined parameters Do not mentioned about noise Too many prototypes Unsupervised learning

Their second research (in 2007)talk aboutthese weakness

8


N.Y.U.S.T.I. M.

9

Objectives

Propose a Improved version of SOINN, ASC (Adjust SOINN Classifier) FASTER: delete/less prototype

Training phase Classification phase

CLASSIFIER: 1-NN (prototype) rule INCREMENTAL LEARNING ONE LAYER: easy to understand the setting,

less parameters~ MORE STABLE: help of k-means


N.Y.U.S.T.I. M.Methodology – Adjusted SOINN

10

Distance: Too far

A node is a cluster



11

Link age



12

Winner

Neighbor



13

Age: Too old > ad



14

Delete outlier:Nodes without

neighbor(low-density assumption)



15

Lambda = iterations


N.Y.U.S.T.I. M.Methodology – k-means

16

Help of k-means clustering, k = # of neurons Adjust the result prototypes: assume that each node nearby the centroid

of class


N.Y.U.S.T.I. M.Methodology – noise-reduction

17

Help of k-Edit Neighbors Classifier (ENC), k=? Delete the node which label are differs from the majority voting of its

k-neighbors: assume that are generated by noise


N.Y.U.S.T.I. M.Methodology – center-cleaning

18

Delete neurons: if it has never been the nearest neuron to other class: assume that are lies in the central part of class


N.Y.U.S.T.I. M.Experiments: Artificial dataset

19

dataset Adjusted SOINN

ASC

Error: sameSpeed: faster



20


ASC

Error: sameSpeed: faster



21


ASC

Error: betterSpeed: faster



22


ASC

Error: betterSpeed: faster


N.Y.U.S.T.I. M.Experiments: Real dataset

23

Compression ratio (%)

Speed up ratio (%)


N.Y.U.S.T.I. M.

Experiments: Compare with other prototype-based classification method

24

Nearest Subclass Classifier (NSC) k-Means Classifier (KMC) k-NN Classifier (NNC) Learning Vector Quantization (LVQ)


N.Y.U.S.T.I. M.

Experiments: Compare with other prototype-based classification method

25


N.Y.U.S.T.I. M.

26

Conclusions

ASC Learns the number of nodes needed to determine the decision boundary Incremental neural network Robust to noisy training data Fast classification Fewer parameters: 3 parameters


N.Y.U.S.T.I. M.

27

Comments

Advantage Improve many things A previous paper to demonstrate the thing they want to modify

Drawback NO Suggestion of parameters

Application A work from unsupervised learning to supervised learning

Documents

A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)