Click here to load reader

知识发现(数据挖掘 ) 第八章

  • View
    85

  • Download
    5

Embed Size (px)

DESCRIPTION

知识发现(数据挖掘 ) 第八章. 史忠植 中国科学院计算技术研究所 http://www.intsci.ac.cn/. 神经 网络 Neural Networks. 目 录. 1. 神经计算 2. 并行分布式理论框架 3. 交互与竞争神经网络 4. 误差反向传播神经网络 5. Hopfield 神经网络 6. 自组织特征映射网络 7. 自适应共振理论 8. 脉冲耦合神经网络. 神经 网络. 一个神经网络是由简单处理元构成的规模宏大的并行分布处理器。天然具有存储经验知识和使之可用的特性。 神经网络从两个方面上模拟大脑: - PowerPoint PPT Presentation

Text of 知识发现(数据挖掘 ) 第八章

  • **)

    http://www.intsci.ac.cn/Neural Networks

  • ** 1. 2. 3. 4. 5. Hopfield6. 7. 8.

  • **

  • ** 2040 1949 1943McCullochPittsM-PBulletin of Methematical Biophysics 949D. O. HebbHebb

  • **1950~1968 Marvin MinskyFrank RosenblattBernard WidrowPerceptron

  • **1969~1982 M. L. MinskyS. PapertPerceptronMIT Press1969 7080

  • **1983~1990 1982J. HopfieldHopfieldLyapunovANNANNANN

  • **1983~1990 1984 J. HopfieldHopfield-Tank TSP 1985UCSDHintonSejnowskyRumelhartPDPHopfieldBoltzmann

  • ** 1986RumelhartBPPaker1982Werbos1974 ART

  • ** Hinton Helmboltz Ying-Yang ( S.Amari) ,

    199012

  • ** 1986UCSDRumellhartMcClellandHinton Parallel and Distributed Processing, MIT Press, Cambridge

  • **PDP1 PEAN2 ai3 fi4 5 wijoi6Fi7 8

  • **

    Various types of neuronsVarious network architecturesVarious learning algorithmsVarious applications

  • **IAC

  • **

  • **

  • **Winner-Take-All Winner Take All

  • **

  • **

  • **3. 310

  • **

    o1 oj om

    W1 Wj Wm

    x1 x2 xi xn

    _971335777.unknown

    _971335778.unknown

    _971337494.unknown

    _971335776.unknown

  • **

  • **(1)X=(x1 ,x2)T w1jx1+w2jx2-Tj=0

  • **

    x1

    * *

    * * O

    * * O

    * * O O

    * O x2

    * * O O

    O O

  • **(2)X=(x1,x2,x3)T w1jx1+w2jx2+w3j x3Tj=0 (3.4)

  • **

    x1

    * *

    * * O

    * * O

    * * O O

    * O x2

    * * O O

    O

    x3

  • **

  • **

    x1x2y1y2o001011100111

  • **

    x1x2y1y2o001010101111

  • **

    x1x2y1y2o0011011010011111

  • **

    x1x2y1y2o00110011011001111110

  • **

  • **BPBP

    o1 ok ol

    W1 Wk Wl

    y1 y2 yj ym

    V1 Vm

    x1 x2 xi xn-1 xn

    _971335777.unknown

    _971335778.unknown

    _971337494.unknown

    _971335776.unknown

  • **BP X=(x1,x2,,xi,,xn)T Y=(y1,y2,,yj,,ym)T O=(o1,o2,,ok,,ol)Td=(d1, d2,,dk,,dl)TV=(V1,V2,,Vj,,Vm)W=(W1,W2,,Wk,,Wl)BP

  • **3.4.1 BPBP

  • **3.4.1 BPBP

  • ** BP

  • **BP

  • **BP(0,1)j=0,1,2,,m; k=1,2,,l i=0,1,2,,n; j=1,2,,mBP

  • **(3.4.9a)BP

  • **(1) (4) (5) (6) (7) (2)X Xpd dp(3)BP

    VWEmin,

    q=1p=1,E=0

    Y

    p1,q1 p

  • **BP

    VW

    q=1p=1

    Y

    p1 p

  • **(1) -BPnm()

  • **(2) (3) ()

  • ** nw E nw+1 nw BP

  • ** BP

    f (x)

    1.0

    0.5

    x

    0

  • ** BP

  • **BP 3BP

  • **1 2 =(1 )BP

  • **3 BP

    o

    1 (=1

    (>1

    net

    0

    _1061210790.unknown

  • ** HopfieldJ.J.Hopfield1982Hopfield1984Hopfield(TSP)() HopfieldDHNN (Discrete Hopfield Neural Network) CHNN (Continues Hopfield Neural Network) Hello,Im John Hopfield

  • **Hopfield

  • **Hopfield

  • **Hopfield i j . , Hopfieldn t

  • **Hopfield

  • **Hopfield Hopfield(1)() ()(2)()

  • **Hopfield () (2-5)i (2-6)

  • **Hopfield

  • **Hopfield Hopfield

  • **Hopfield

    +

    +

    1

    N

    j

    i

    +

    +

  • **Hopfield

    Hopfield

  • **Hopfield Hopfield1)HopfieldI/OSigmoid2)3)4) Hopfield

  • **Hopfield MATLABMATLABHopfield

    satlin( )satlins( )newhop( )Hopfieldnnt2hop( )NNT 2.0 Hopfield

  • **Hopfield MATLAB MATLABHopfield newhop( ) Hopfield net = newhop(T) netTTQR*Q-11HopfieldHopfieldsatlins( )

  • **Hopfield MATLABMATLABHopfieldsatlins( ) A = satlins(N)ANS*QANA[01]N-11-1-111

  • **Hopfield MATLAB10 101-1Hopfield 12

  • **

  • ** CohenGrossberg[1983]:Hopfield Hopfield0 LyapunovHopfield

  • **Lyapunov wijoiojxjojjoj

  • **ANkokok 1ANk

  • **ANkokokwkk=0

  • **=-(netk-k)okANkok=(ok-ok)ok=0 =0ok>0ok=1& ok=0ok01netk>knetk-k>0-(netk-k)ok
  • **ANkokok2ANk

  • **ANkokokANk 0

  • **

    Bidirectional Associative MemoryBAM

  • **

  • ** Y=F(XW)X=F(YWT)X=(x1x2xn)Y=(y1y2ym)FS

  • **0 1if neti>0yi= 0if neti
  • **BAMKosko(1987)WWHopfield

  • ** S={(X1,Y1)(X2,Y2),(Xs,Ys)}

  • ** Kosko1987min HainesHecht-Nielson19882min R. J. McElieceE. C. PosnerE. R. RodemichL4+log2min1-198%

  • **HopfieldTSP1985J. J. HopfieldD. W. TankTSP3030 nn!/(2n) n,n*n

  • **HopfieldTSPdxyXYyxiXi 1Xiyxi= 0Xiwxi,yjXiYj

  • **HopfieldTSPXYZW

  • **HopfieldTSP wxi,yj= -Axy(1-ij) Bij(1-xy) C dxy(ji+1+ji-1)

    1i=jij= 0ij

  • **

  • ** 0ABCD1

  • **02

  • **nn03

  • ** 4

  • **HopfieldTSP HopfieldTSP10TSP

    n!/2n=10!/20=181440

    20

  • ** ,

  • ** (self-Organizing MapSOM)(Adaptive Resonance TheoryART)(Counter PropagationCP)(Synergetic Neural NetworkSNN)

  • **Teuvo Kohonen1981

    , Kohonen Im Teuvo Kohonen

  • **SOM Kohonen1234

  • **SOM

  • **SOM

  • **SOM SOM

    (a) (b)

  • **SOM SOM

  • **SOM SOM

  • **

    Kohonen

  • **(1)(2)(3)

  • **(4) (5)

    (6)(2)

  • **

    Nj*(0)

    Nj*(0)

    Nj*(1)

    Nj*(1)

    Nj*(2)

    Nj*(2)

  • **Kohonen

    W

    j=1,2,m

    Nj*(0)

    ((t)

    p({1,2,,P}

    j=1,2,m

    j*

    Nj*(t)

    Nj*(t)

    i=1,2,n j(Nj*(t)

    N

  • **Hierarchical and Partitive ApproachesPartitive algorithmDetermine the number of clusters.Initialize the cluster centers.Compute partitioning for data.Compute (update) cluster centers.If the partitioning is unchanged (or the algorithm has converged), stop; otherwise, return to step 3k-means error functionTo minimize error function

  • **Hierarchical and Partitive ApproachesHierarchical clustering algorithm (Dendrogram)Initialize: Assign each vector to its own clusterCompute distances between all clusters.Merge the two clusters that are closest to each other.Return to step 2 until there is only one cluster left.Partition strategyCut at different level

  • **Hierarchical SOMGHSOM Growing Hierarchical Self-Organizing Mapgrow in size in order to represent a collection of data at a particular level of detail

  • **MATLABMATLAB

    newsom()plotsom()vec2ind()compet()midpoint()learnsom()

  • **MATLABMATLABnewsom() net = newsom(PR[D1D2...]TFCNDFCNOLROSTEPSTLRTND) netBPPR[Pmin Pmax][D1D2...]TFCNhextopDFCNlinkdistOLR0.9OSTEPS1000TLR0.02TND1

  • **MATLAB plotsom() (1) plotsom(pos) (2) plotsom(WDND) pos1WDND1ND

  • **MATLAByec2ind() ind = vec2ind(vec) vecmnxxi10 indn1

  • **MATLAB1 1999100.5512 0.51230.50870.50010.60120.52980.50000.49650.51030.5003;0.4488 0.48770.49130.49990.39880.47020.50000.50350.48970.4997

  • **MATLAB1

  • **1 SOM

  • **1 SOM125435

  • **SOMWord category map Hash

  • **Ii(d)idIi(1)ixii{d1,,dN}

    d=1d=-1

  • ** ={}

  • **wi(i=1,2,,N) ni wi cijwiwj SOM

  • **

  • **SOM50 1 127 184 63 773

  • **SOMWord category map180SOM 13 432 63 773 13 432

  • **SOMDocument map 13 432 (Random mapping method) 315 315 SOM http://websom.hut.fi/websom/

  • **SOM

  • **SOMSelf-organizing MapsSOM

  • ** EckhornPulse-Coupled Neural Network, PCNN )PCNNPCNN

  • ** 1952HodgkinHuxley[216]1987Charles M. Gray[172 173]1989,Reinhard Eckhorn Charles M. Gray[106173]1990Reinhard Eckhorn[107]1994JohnsonPCNN[244]EckhornPCNN1999IEEE. 2090

  • ** Ekblad, U.J.M. Kinser2004

Search related