1
M U
1. Tnh cp thit ca ti
Trong rt nhiu lnh vc nh iu khin, t ng ha, cng ngh
thng tin, nhn dng c i tng l vn mu cht quyt
nh s thnh cng ca bi ton.
Mt nhc im khi dng mng nron l cha c phng php lun
chung khi thit k cu trc mng cho cc bi ton nhn dng v iu
khin m phi cn ti kin thc ca chuyn gia. Mt khc khi xp x
mng nron vi mt h phi tuyn s kh khn khi luyn mng v c
th khng tm c im ti u ton cc... Hin nay, vic nghin cu
cc thut ton tm nghim ti u ton cc khi luyn mng nron
c mt s tc gi nghin cu p dng. Tuy nhin khi s dng
mng nron xp x mt s i tng phi tuyn m mt li sinh ra
c dng lng khe, vic hun luyn mng gp rt nhiu kh khn.
Ni dung ti s i nghin cu mt thut ton tm im ti u ton
cc trong qu trnh luyn mng nron bng thut ton vt khe c s
kt hp vi gii thut di truyn.
2. Mc tiu ca lun n
- xut m hnh kt hp thut ton vt khe v gii thut di truyn
hun luyn mng nron.
- Xy dng b cng c phn mm luyn mng nron cho mt s bi
ton c mt li c bit, lm c s b sung vo Neural Toolbox
Matlab.
3. Ni dung chnh ca lun n
- Nghin cu l thuyt v thut ton vt khe v xy dng thut ton
tnh bc hc vt khe.
- Xy dng thut ton hun luyn mng nron bng k thut lan
tuyn ngc kt hp vi thut ton vt khe.
2
- xut thut ton hun luyn mng nron bng k thut lan truyn
ngc c s dng gii thut di truyn kt hp vi thut ton vt
khe.
- Vit v ci t chng trnh hun luyn mng nron trn C++.
- Vit v ci t chng trnh hun luyn mng nron trn Matlab.
CHNG 1
MNG NRON V QU TRNH HC CA MNG NRON
1.1. Gii thiu v mng nron v qu trnh hc ca mng
nron
1.1.1. Mng nron v cc phng php hc
Mng nron nhn to, gi tt l mng nron, l mt m hnh x l
thng tin phng theo cch thc x l thng tin ca cc h nron sinh
hc. N c to ln t mt s lng ln cc phn t (gi l nron)
kt ni vi nhau thng qua cc lin kt (gi l trng s lin kt) lm
vic nh mt th thng nht gii quyt mt vn c th no .
Mt mng nron nhn to c cu hnh cho mt ng dng c th
(nhn dng mu, phn loi d liu,...) thng qua mt qu trnh hc t
tp cc mu hun luyn. V bn cht hc chnh l qu trnh hiu
chnh trng s lin kt gia cc nron sao cho gi tr hm li l nh
nht.
C ba phng php hc ph bin l hc c gim st, hc khng gim
st v hc tng cng. Hc c gim st l phng php c s
dng ph bin nht, trong tiu biu l k thut lan truyn ngc.
1.1.2. nh gi cc nhn t ca qu trnh hc
1.1.2.1. Khi to cc trng s
Do bn cht ca gii thut hc lan truyn ngc sai s l phng
php gim lch gradient nn vic khi to cc gi tr ban u ca
cc trng s cc gi tr nh ngu nhin s lm cho mng hi t v cc
gi tr cc tiu khc nhau.
3
1.1.2.2. Bc hc
Vic chn hng s hc ban u l rt quan trng. Vi mi bi ton ta
li c phng n chn h s hc khc nhau. Khi mt qu trnh hun
luyn theo k thut lan truyn ngc hi t, ta cha th khng nh
c n hi t n phng n ti u. Ta cn phi th vi mt s
iu kin ban u m bo thu c phng n ti u.
1.2. Nhn dng h thng s dng mng nron
1.2.1. Nhn dng h thng
1.2.1.1. Ti sao phi nhn dng
Bi ton nhn dng l mt vn t ln hng u trong nhiu cc
lnh vc khc nhau nh: in t y sinh, in t vin thng, h thng
in, t ng ha v iu khin V d nh: nhn dng vn tay,
nhn dng k t, nh, ting ni, pht hin v chn on bnh...
1.2.2. Nhn dng h thng s dng mng nron
1.2.2.1. Kh nng s dng mng nron trong nhn dng
Xt trng hp i tng phi tuyn c phc tp cao, nu s dng
phng php gii tch thng thng nhn dng s rt kh khn,
thm ch khng thc hin c do s hiu bit ngho nn v i
tng. V vy cc nh khoa hc a ra tng l s dng cng c
tnh ton mm nh h m, mng nron, i s gia t xp x -
chnh l nhn dng i tng. Mng nron l mt trong nhng cng
c hu hiu nhn dng m hnh i tng, bng phng php ny
ta khng bit c m hnh ton thc s ca i tng nhng hon
ton c th s dng kt qu xp x thay th i tng.
1.2.2.2. M hnh nhn dng h thng s dng mng nron
Nhn dng gm: nhn dng m hnh v nhn dng tham s.
Nhn dng m hnh l qu trnh xc nh m hnh ca i tng v
thng s trn c s u vo v u ra ca i tng. M hnh thu
c sau khi nhn dng gi l tt nu n th hin c ng i
4
tng. Nh vy c th s dng m hnh thay cho i tng d
bo, kim tra v iu khin.
Mng nron c hun luyn
m hnh ha quan h vo ra
ca i tng. Nh vy quy
trnh nhn dng m hnh c
bn cht l thut ton luyn
mng. Cu trc mng nron
gii bi ton nhn dng m hnh rt a dng, ty thuc vo tng bi
ton c th.
Nhn dng tham s chnh l hun luyn mng, c biu din trn
Hnh 1.2. Tn hiu sai s e y y l c s cho qu trnh luyn mng.
Mng nron y c th l mng nhiu lp hoc cc dng khc v
c th s dng nhiu thut luyn mng khc nhau.
1.2.2.3. Nhn dng h thng s dng mng nron
Nhn dng h thng cn hai giai on l la chn m hnh v ti u
tham s. i vi mng nron la chn s nt n, s lp n (cu trc
ca mng) tng ng vi m hnh la chn. Mng c th c
hun luyn theo kiu gim st vi k thut lan truyn ngc, da vo
lut hc sai s hiu chnh. Tn hiu sai
s c lan truyn ngc qua mng.
K thut lan truyn ngc s dng
phng php gim gradient xc
nh cc trng ca mng v vy tng
ng vi ti u tham s.
1.3. Mt li c bit khi luyn
mng nron
1.3.1. Mt li c bit khi luyn
mng nron Hnh 1.3: Mt sai s dng lng khe
i tng
Mng nron
u y
y
e -
Hnh 1.2: M hnh nhn dng c bn
5
Hnh 1.3 m t mt mt sai s, c mt vi iu c bit cn ch i
vi mt sai s ny: dc bin i mt cch mnh m trn khng
gian tham s. V l do , n s kh m la chn mt tc hc
ph hp cho thut ton gim dc nht.
1.3.2. V d v bi ton dn n mt li c bit
c im khe ca cc bi ton ti -u ho trong ngnh nhit[28]
S dng mng nron nhn dng i tng
Vi cc h thng c phi tuyn cao th lm th no nhn dng
i tng lun l mt cu hi t ra vi chng ta. V tnh phi tuyn
ca cc mng nron (hm kch hot phi tuyn), chng c dng
m t cc h thng phi tuyn phc tp.
Luyn mng nron c hai qu trnh, qu trnh nh x v qu trnh
hc. Hc thc cht l qu trnh lan truyn ngc. Thc hin k thut
lan truyn ngc chnh l gii bi ton ti u tnh vi hm mc tiu
l mt sai s.
Hnh dng ca mt sai s ph thuc vo s lp nron v loi hm
kch hot. Trong khi mt sai s vi mng tuyn tnh mt lp c mt
cc tiu n v dc khng i, mt sai s vi mng nhiu lp c
th c nhiu im cc tiu cc b, c th b ko di, un cong to
thnh khe, trc khe v dc c th thay i mt di rng trong
cc vng khc nhau ca khng gian tham s.
Thc t, vic chn hm kch hot nh th no, chn s lp mng
nron bng bao nhiu ph thuc vo i tng cn xp x. Nh vy,
do phc tp ca i tng cn xp x khc nhau nn hm mc tiu
rt khc nhau v dn n qu trnh hc (gii bi ton ti u) c th
rt phc tp. c bit khi i tng cn xp x dn n hm mc tiu
c dng lng khe (v d nh i tng nhit) th qu trnh hc rt
kh khn thm ch khng hi t nu ta s dng cc b cng c c
trong Toolbox ca Matlab.
6
1.4. M phng qu trnh luyn mng nron khi s dng
Toolbox ca Matlab
1.4.1. M phng hun luyn mng nron c mt li bnh
thng
Xt h thng phi tuyn cn nhn
dng c m hnh ton hc sau:
f (u) = 0.6 sin( .u) + 0.3 sin(3. .u) + 0.1
sin (5. .u)
Tn hiu vo: u (k) = sin(2 .k/250)
Mng nron s dng l mng truyn
thng 3 lp c mt u vo v mt
u ra.
1.4.2. M phng hun luyn mng nron c mt li c bit
minh ha, tc gi xut cu trc mng n ron nhn dng cc
ch s: 0, 1, 2,...,9. Trong hm sigmoid c s dng lm hm
kch hot. 1/ (1 exp(-x))f
Hnh 1.6 trnh by kt qu ca qu trnh luyn mng cho bi ton
nhn dng ch vi cc k thut lan truyn ngc sai s theo phng
php Batch Gradient Descent (traingd), Batch Gradient Descent with
Momentum (traingdm), Variable Learning Rate (traingda, traingdx).
Cc phng php ny u c tch hp trn Neural Network
Hnh 1.5: Cu trc mng nron cho nhn dng ch
Hnh 1.4: K nguyn luyn mng v d 1
7
Toolbox ca Matlab. Nhn chung cc phng php u cho kt qu
kh tt, tuy nhin t c chnh xc nh mong mun th thi
gian cn thit cho luyn mng l kh ln. Thm ch c trng hp tn
hiu li hu nh thay i rt t qua cc chu k luyn mng.
1.5. Tng quan v tnh hnh nghin cu trong v ngoi nc
1.6. Kt lun chng 1
Trong chng 1, tc gi phn tch cc nhn t trong qu trnh hc
ca mng nron. Tc gi nhn thy rng, kt qu luyn mng nron
ph thuc rt ln vo gi tr ban u ca vec-t trng s v bc
hc. Vic mng s hi t n im ti u ton cc hay khng nhiu
khi cn ph thuc vo s may mn do vic chn gi tr khi to l
ngu nhin. Thm na, vic la chn bc hc s bng bao nhiu
c th hi t hay t nht l tng tc hi t l mt cu hi cng
c t ra, c bit khi mt li c dng c bit. minh chng
cho iu tc gi a ra 2 v d: v d 1, khi mt li dng
bnh thng, s dng b cng c trong Toolbox ca Matlab luyn
mng, mng luyn thnh cng sau 65 bc tnh. n v d th 2
v nhn dng ch vit tay th thi gian luyn mng lu hn rt nhiu,
thm ch tn hiu li cn thay i rt t qua cc chu k luyn mng.
Hnh 1.6: Cc kt qu luyn mng n ron vi cc phng php
lan truyn ngc khc nhau (traingd, traingdm, traindx, trainda)
8
gii quyt vn ny, cn thit phi tm ra mt thut ton hiu
chnh cc bc hc nhm rt ngn thi gian hi t ca mng ng
thi cng trnh c vn cc tr a phng.
CHNG 2: THUT TON VT KHE TRONG
QU TRNH LUYN MNG NRON
2.1. Thut ton vt khe
Cho bi ton ti u v gii bi ton ti u khng iu kin:
MinJ(u) u En (2.1)
u l vec-t trong khng gian
Euclide n chiu
Cng thc lp bc th k:
uk+1
= uk+ k s
k, k = 0,1,(2.2)
trong : u : vect bin ca hm
mc tiu J(u) ti bc lp th k;
k l di bc ca hm theo hng chuyn ng sk.