Upload
nhan-nguyen
View
11
Download
0
Embed Size (px)
DESCRIPTION
May Hoc
Citation preview
My Hc (Machine Learning)
TS. on Thanh Ngh
An Giang University
Long Xuyn, Ngy 4 thng 3 nm 2014
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Chng 3 - Cc phng php hc da trn xc sut
1
Gii thiu
2
Xc sut hu nghim cc i (MAP)
3
nh gi kh nng c th nht (MLE)
4
Phn loi Nave Bayes
5
Cc i ha k vng EM
2 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Cc phng php hc da trn xc sut
Cc phng php thng k cho bi ton phn loi
Phn loi da trn m hnh xc sut c s
Vic phn loi da trn kh nng xy ra (probabilities) ca
cc lp
Cc ch chnh:
Gii thiu v xc sut
nh l Bayes
Xc sut hu nghim cc i (Maximum a posteriori)
nh gi kh nng c th nht (Maximum likelihood
estimation)
Phn loi Nave Bayes
Cc i ha k vng (Expectation maximization)
3 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Cc khi nim c bn v xc sut
Gi s chng ta c mt th nghim (v d: mt qun xc
sc) m kt qu ca n mang tnh ngu nhin
Khng gian cc kh nng S . Tp hp tt c cc kt qu cth xy ra
V d: S = {1, 2, 3, 4, 5, 6} i vi th nghim qun xc scS kin E . Mt tp con ca khng gian cc kh nngV d: E = {1}: kt qu qun sc xc ra l 1V d: E = {1, 3, 5}: kt qu qun sc xc ra l mt s lKhng gian cc s kin W . Khng gian (world) m cc kt
qu ca s kin c th xy ra
W bao gm tt c cc ln sc xc
Bin ngu nhin A. Mt bin ngu nhin biu din (din t)mt s kin, v mt mc v kh nng xy ra s kin ny
4 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Biu din
P(A): Phn ca khng gian (world) m trong A l ng
Khng gian tt c cc gi tr c th xy ra ca A
khng gian m trong A l ng
khng gian m trong A l sai
5 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Cc bin ngu nhin 2 gi tr
Mt bin ngu nhin 2 gi tr (nh phn) c th nhn mt
trong 2 gi tr ng (true) hoc sai (false)
Cc tin
0 P(A) 1P(true) = 1
P(false) = 0
P(A B) = P(A) + P(B) P(A B)Cc h qu
P(not A) P(A) = 1 P(A)P(A) = P(A B) + P(A B)
6 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Cc bin ngu nhin a tr
Mt bin ngu nhin nhiu gi tr c th nhn mt trong s
k(> 2) gi tr v1, v2, ..., vk
Cc tin
P(A = vi A = vj) = 0 if i 6= jP(A = v1 A = v2 ... A = vk) = 1P(A = v1 A = v2 ... A = vi ) =
ij=1 P(A = vj)k
j=1 P(A = vj) = 1
P(B [A = v1 A = v2 ... A = vi ] =i
j=1 P(B A = vj)
7 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Xc sut c iu kin (1)
P(A|B) l phn khng gian m trong A l ng, vi iukin ( bit) l B ng
V d:
A: Ti s i bng vo ngy maiB: Tri s khng ma vo ngy maiP(A|B): Xc sut ca vic ti i bng vo ngy mai nu( bit rng) tri s khng ma (vo ngy mai)
8 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Xc sut c iu kin (2)
nh ngha:
P(A|B) = P(A,B)P(B)Cc h qu
P(A,B) = P(A|B) P(B)P(A|B) + P(A|B) = 1k
i=1 = P(A = vi |B) = 1
khng gian m trong B l ng
khng gian m trong A l ng
9 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Cc bin c lp v xc sut (1)
Hai s kin A v B c gi l c lp v xc sut nu xcsut ca s kin A l nh nhau i vi cc trng hp:
Khi s kin B xy ra, hocKhi s kin B khng xy ra, hocKhng c thng tin (khng bit) v vic xy ra ca s kin B
V d:
A: Ti s i bng vo ngy maiB: Tun s tham gia trn bng ngy mai
P(A|B) = P(A)D Tun c tham gia trn bng ngy mai hay khng cng
khng nh hng ti quyt nh ca ti v vic i bng vo
ngy mai
10 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Cc bin c lp v xc sut (2)
T nh ngha ca cc bin c lp v xc sut
P(A|B) = P(A), chng ta thu c cc lut nh sau:P(A|B) = P(A)P(B|A) = P(B)P(A,B) = P(A) P(B)P(A,B) = P(A) P(B)P(A, B) = P(A) P(B)P(A, B) = P(A) P(B)
11 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Xc sut c iu kin vi nhiu hn 2 bin
P(A|B,C ) l xc sut ca A i vi (bit) B v C
V d:
A: Ti i do b sng vo sng maiB: Thi tit sng mai rt pC : Ti s dy sm vo sng maiP(A|B,C ): Xc sut ca vic ti s ido dc b sng vo sng mai, nu (
bit rng) thi tit sng mai rt p v
ti s dy sm vo sng mai
B C
AP(A | B, C)
12 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
c lp c iu kin
Hai bin A v C c gi l c lp c iu kin i vibin B, nu xc sut ca A i vi B bng xc sut ca A ivi B v C
Cng thc nh ngha: P(A|B,C ) = P(A|B)V d:
A: Ti s i bng vo ngy maiB: Trn bng ngy mai s din ra trong nhC : Ngy mai tri s khng ma
P(A|B,C ) = P(A|B)Nu bit rng trn u ngy mai s din ra trong nh, th xc
sut ca vic ti s i bng vo ngy mai khng ph thuc
vo thi tit
13 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Cc quy tc quan trng ca xc sut
Quy tc chui (chain rule)
P(A,B) = P(A|B) P(B) = P(B|A) P(A)P(A|B) = P(A,B)/P(B) = P(B|A) P(A)/P(B)P(A,B|C ) = P(A,B,C )/P(C ) = P(A|B,C ) P(B,C )/P(C )
= P(A|B,C ) P(B,C )c lp v xc v c lp c iu kin
P(A|B) = P(A); nu A v B c lp v xc sutP(A,B|C ) = P(A|C ) P(B|C ); nu A v B l c lp c iukin i vi C
P(A1, ...,An|C ) = P(A1|C )...P(An|C ); nu A1, ...,An l clp c iu kin i vi C
14 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
nh l Bayes
P(h|D) = P(D|h)P(h)P(D)
P(h): Xc sut trc (tin nghim) ca gi thit (phn loi) h
P(D): Xc sut trc (tin nghim) ca vic quan st cd liu D
P(D|h): Xc sut (c iu kin) ca vic quan st c dliu D, nu bit gi thit (phn loi) h l ng
P(h|D): Xc sut (c iu kin) ca gi thit (phn loi)h lng, nu quan st c d liu D
Cc phng php phn loi da trn xc sut s s dng xc
sut c iu kin (posterior probability) ny!
15 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
nh l Bayes - V d (1)
Gi s chng ta c tp d liu sau (d on 1 ngi c chi tennis
hay khng?)
Ngy Ngoi tri Nhit m Gi Chi tennis
N1 Nng Nng Cao Yu Khng
N2 Nng Nng Cao Mnh Khng
N3 m u Nng Cao Yu C
N4 Ma Bnh thng Cao Yu C
N5 Ma Mt m Bnh thng Yu C
N6 Ma Mt m Bnh thng Mnh Khng
N7 m u Mt m Bnh thng Mnh C
N8 Nng Bnh thng Cao Yu Khng
N9 Nng Mt m Bnh thng Yu C
N10 Ma Bnh thng Bnh thng Yu C
N11 Nng Bnh thng Bnh thng Mnh C
N12 m u Bnh thng Cao Mnh C
16 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
nh l Bayes - V d (2)
D liu D. Ngoi tri l nng v Gi l mnh
Gi thit (phn loi) h. Anh ta chi tennis
Xc sut trc P(h). Xc sut rng anh ta chi tennis (bt kNgoi tri nh th no v Gi ra sao)
Xc sut trc P(D). Xc sut rng Ngoi tri l nng vGi l mnh
Xc sut trc P(D|h). Xc sut Ngoi tri l nng v Gil mnh, nu bit rng anh ta chi tennis
Xc sut trc P(h|D). Xc sut anh ta chi tennis, nu bitrng Ngoi tri l nng v Gi l mnh
Chng ta quan tm n gi tr xc sut sau (posteriorprobability) ny!
17 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
Xc sut hu nghim cc i (MAP)
Vi mt tp cc gi thit (cc phn lp) c th H, h thnghc s tm gi thit c th xy ra nht (the most probable
hypothesis) h( H) i vi cc d liu quan st c DGi thit h ny c gi l gi thit c xc sut hu nghimcc i (Maximum a posteriori MAP)
hMAP = arg maxhH P(h|D)hMAP = arg maxhH
P(D|h)P(h)P(D) (bi nh l Bayes)
hMAP = arg maxhH P(D|h) P(h) (v p(D) l nh nhaui vi cc gi thit h)
18 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
MAP - V d
Tp H bao gm 2 gi thit (c th)h1: Anh ta chi tennish2: Anh ta khng chi tennis
Tnh gi tr ca 2 xc sut c iu kin: P(h1|D),P(h2|D)Gi thit c th nht hMAP = h1 nu P(h1|D) P(h2|D);ngc li th hMAP = h2
Bi v P(D) = P(D, h1) +P(D, h2) l nh nhau i vi c haigi thit h1 v h2, nn c th b qua i lng p(D)
V vy, cn tnh 2 biu thc: P(D|h1) P(h1) vP(D|h2) P(h2), v a ra quyt nh tng ngNu P(D|h1) P(h1) P(D|h2) P(h2), th kt lun l anh tachi tennis
Ngc li, th kt lun l anh ta khng chi tennis
19 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
nh gi kh nng c th nht (MLE)
Phng php MAP: Vi mt tp cc gi thit c th H, cntm mt gi thit cc i ha gi tr: P(D|h) P(h)Gi s (assumption) trong phng php nh gi kh nng c
th nht (Maximum likelihood estimation MLE): Tt c cc
gi thit u c gi tr xc sut trc nh nhau:
P(hi ) = P(hj),hi , hj HPhng php MLE tm gi thit cc i ha gi tr P(D|h);trong P(D|h) c gi l kh nng c th (likelihood) cad liu D i vi h
Gi thit c kh nng nht (maximum likelihood hypothesis)
hML = arg maxhH P(D|h)
20 / 44
Gii thiu
Xc sut hu nghim cc i (MAP)
nh gi kh nng c th nht (MLE)
Phn loi Nave Bayes
Cc i ha k vng EM
MLE V d
Tp H bao gm 2 gi thit c thh1: Anh ta chi tennish2: Anh ta khng chi tennis
D: Tp d liu (cc ngy) m trong thuc tnh Outlook cgi tr Sunny v thuc tnh Wind c gi tr StrongTnh 2 gi tr kh nng xy ra (likelihood values) ca d liu
D i vi 2 gi thit: P(D|h1) v P(D|h2)P(Outlook = Sunny ,Wind = Strong |h1) = 1/8P(Outlook = Sunny ,Wind = Strong |h2) = 1/4Gi thit MLE hMLE = h1 nu P(D|h1) P(D|h2); v ngcli th hMLE = h2Bi v P(Outlook = Sunny ,Wind = Strong |h1)