Unsupervised Ensemble Learning with Dependent Classifiers

Ariel Jaffe, Ethan Fetaya, Boaz Nadler Weizmann Institute of Science

Tingting Jiang, Yuval Kluger Yale University School of Medicine

B4

1

AISTATS 2016

Unsupervised Ensemble Learning with Dependent Classifiers

↓

3

A

4

AB

A

5

AB

A

A

6

crowdsourcing

computational biology medicine and decision science

7

http://www.kantei.go.jp/jp/asophoto/2009/04/07kokkai.html

8

assuming conditional independence between classifiers

※

9

GErr(fMens) = EX0

�1

MVar(X0) + Bias(X0)

2 +

�1 � 1

M

�Cov(X0)

�+ �2

assuming conditional independence between classifiers

※

10

assuming conditional independence between classifiers

※

11

http://www.kantei.go.jp/jp/asophoto/2009/04/07kokkai.html

12

1. Introduction 2. Problem Setup 3. Estimating the assignment function

4. The latent spectral meta learner

5. Experiments

13

1. Introduction 2. Problem Setup 3. Estimating the assignment function

4. The latent spectral meta learner

5. Experiments

14

motivated by DREAM challenge DNA

100 100,000

m {fi}mi=1

15

m {fi}mi=1

xj

fi zij

16

m {fi}mi=1

sensitivity : �i = Pr(fi(X) = 1|Y = 1)

specificity : �i = Pr(fi(X) = �1|Y = �1)

overall accuracy : �i =1

2(�i + �i)

i �i, �i

17

proposed by Dawid and Skene [6]

Pr(f1 = a1, . . . , fm = am|Y ) =�

i

Pr(fi = ai|Y )

18

proposed by Dawid and Skene [6]

y = sign(m�

i=1

wifi(x) + w0), wi = w(�i, �i)

�, �

�, �

19

proposed by Dawid and Skene [6]

↓

�, �

20

21

�, �

22

�, �xj

fi zij

23

1. Introduction 2. Problem Setup 3. Estimating the assignment function

4. The latent spectral meta learner

5. Experiments

24

{�i}K

k=1 K � m

fi assignment function c(i)�

c(i) �= c(j) : fi, fj Y

c(i) = c(j) : fi, fj �c(i)

K

assignment function c(i)

25

26

27

↓

28

rij = E[(fi � E[fi])(fj � E[fj ])]

assignment function c

von, voff � Rm

29

R :

⬛

⬛

30

voff · (voff )T von · (von)T

31

voff · (voff )T von · (von)T

32

rij = Ic(i, j)voni von

j + (1 � Ic(i, j))voffi voff

j

rij =

�voff

i · voffj if c(i) �= c(j)

voni · von

j if c(i) = c(j)

von, voff , c

Ic(i, j) =

�1 c(i) = c(j)

0 otherwise

33

rij = E[(fi � E[fi])(fj � E[fj ])]

�(von, voff , c) =�

i �=j

Ic(i, j)(voni von

j � rij)2+

(1 � Ic(i, j))(voffi voff

j � rij)2

von, voff , c

rij = Ic(i, j)voni von

j + (1 � Ic(i, j))voffi voff

j

34

von, voff , cR

35

NP-hard

von, voff , cR

36

NP-hard

von, voff , cR

S = S(R)

sij =�

k �=i,j;l �=i,j

|rij rkl � rilrkj |

37

Lemma 2

S = S(R)

fi, fj i, j

sij =�

k �=i,j;l �=i,j

|rij rkl � rilrkj |

fi, fj

38

Lemma 2

Lemma 4

S = S(R)

fi, fj i, j

sij =�

k �=i,j;l �=i,j

|rij rkl � rilrkj |

fi, fj

fi, fj

39

maxc(i)�=c(j)

sij < minc(i)=c(j)

sij

von, voff , c

40

41

42

43

44

45

DREAM challenge S1 dataset 124 Fig2. Fig3.

46

47

time complexity

100

m O(m4)

49

→

50

→

51

1. Introduction 2. Problem Setup 3. Estimating the assignment function

4. The latent spectral meta learner

5. Experiments

52

�k

��i , ��

i

��, ��

53

��, ��

54

��, ��

55

reference [10] AISTATS 2015

ensemble pruning : most accurate classifiers from different group

56

→

→��, ��

57

1. Introduction 2. Problem Setup 3. Estimating the assignment function

4. The latent spectral meta learner

5. Experiments

58

UCI DREAM challenge

59

Majority voting SML+EM Oracle-CI

L-SML

60

20 10000

xj

fi zij

61

20 10000

62

※

63

UCI

11 19000

Random Forest SVM

64

65

DREAM challenge dataset

14 100,000

100

66

67

68

Oracle-CI

69

Appendix