Intelligent Database Systems Lab

國立雲林科技大學National Yunlin University of Science and Technology

1

Modeling Semantic Similarities inMultiple Maps

Presenter : Wei-Hao Huang 　Authors : Laurens van der Maaten, Geoffrey Hinton

EWI-ICT TR, 2009

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

2

Outlines Motivation Objectives Methodology Experiments Conclusions Comments

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

3

Motivation· Semantic space models cannot faithfully

represent intransitive pairwise similarities or the similarities of words that have multiple meanings.─ Triangle inequality─ Nearest neighbor is limited─ Similarities are symmetric

tie

suittuxedo

ropeknot

Animal

dogdog

dog

dog

dogChina

North Korea

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Objectives

4

• To propose multiple map SNE to solve fundamental limitations of metric spaces

tie

suittuxedo

ropeknot

tie

suittuxedo

tie

rope

knot

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

5

Methodology· Stochastic neighbor embedding· Multiple maps SNE

Map Map2Map1 Map3

Data

SNE Multiple maps SNE

Data

Mixing proportion

(importance)

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology· Stochastic neighbor embedding

6

tie

suittuxedo

ropeknot

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology· Multiple map SNE

7

tieropesuit

tie animal animal

dog dog

dog

dog dog

dog

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology· Multiple map SNE

8

A*C=1*1/2=1/2

B*C=1*1/2=1/2

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments· Visualization Experiments

─ Florida State University word association dataset─ Selecting 5019 words

· Generalization Experiments─ To evidence their model for semantic representation─ Training data: 80%─ Validation data: 10%─ Test data: 10%

9

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments· Visualization

10

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments

11

Sport

Clothing

Statue of Liberty

Cheerleader

Cheerleader

Tie

Tiemonarchy

monarchy

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments· Generalization

12

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments· Comparing multiple maps SNE with other

method.─ Semantic space models─ Semantic networks ─ Topic models

13

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Conclusions· The multiple maps SNE alleviates the

fundamental limitations of metric spaces.· Multiple map model has characteristics that

are similar to those of topic models.

14

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

15

Comments· Advantages

─ Multiple maps SNE alleviates the fundamental limitations of metric spaces

· Applications─ Data visualization─ Semantic similarities