Hopfield Neural Networks for Optimization Content Introduction A Simple Example Race Traffic Problem Example A/D Converter Example Traveling Salesperson Problem Hopfield Neural Networks for Optimization Introduction Energy Function of a Hopfield NN Interaction btw neurons Interaction to the external constant Running a Hopfield NN asynchronously, its energy is monotonically non-increasing. Solving Optimization Problems Using Hopfield NNs Reformulating the cost of a problem in the form of energy function of a Hopfield NN. Build a Hopfield NN based on such an energy function. Running the NN asynchronously until the NN settles down. Read the answer reported by the NN. Hopfield Neural Networks for Optimization A Simple Example Race Traffic Problem A Simple Hopfield NN 11 11 22 22 I1I1 I2I2 The Race Traffic Problem +1 11 11 v1v1 v2v2 The Race Traffic Problem 11 11 22 22 00 11 11 11 22 22 00 11 11 11 1 Stable State The Race Traffic Problem 11 11 22 22 00 11 11 11 1 Stable State The Race Traffic Problem 11 11 22 22 00 11 11 11 How about if to run synchronously? Hopfield Neural Networks for Optimization Example A/D Converter Reference Tank, D.W., and Hopfield, J.J., Simple "neural" optimization networks: An A/D converter, signal decision circuit and a linear programming circuit, IEEE Transactions on Circuits and Systems, Vol. CAS-33 (1986) Analog A/D Converter A/D v0v0 v1v1 v2v2 v3v I Using Unipolar Neurons A/D Converter Using Unipolar Neurons A/D Converter v0v0 v1v1 v2v2 v3v3 I0I0 I1I1 I2I2 I3I3 Hopfield Neural Networks for Optimization Example Traveling Salesperson Problem Reference J. J. Hopfield and D. W. Tank, Neural computation of decisions in optimization problems, Biological Cybernetics, Vol. 52, pp , 1985. Traveling Salesperson Problem Given n cities with distances d ij, what is the shortest tour? Traveling Salesperson Problem Traveling Salesperson Problem Distance Matrix Find a minimum cost Hamiltonian Cycle. Search Space Find a minimum cost Hamiltonian Cycle. Assume we are given a fully connection graph with n vertices and symmetric costs ( d ij =d ji ). The size of search space is Problem Representation Using NNs Time City Problem Representation Using NNs Time City The salesperson reaches city 5 at time 3. Problem Representation Using NNs Time City Goal: Find a minimum cost Hamiltonian Cycle. The Hamiltonian Constraint Time City Goal: Find a minimum cost Hamiltonian Cycle. Each row and column can have only one neuron on. For a n -city problem, n neurons will be on. Each row and column can have only one neuron on. For a n -city problem, n neurons will be on. Cost Minimization Time City Goal: Find a minimum cost Hamiltonian Cycle. The total distance of the valid tour have to be very low. d 35 d 54 d 42 d 25 d 51 The summation of these d ij s is very low. Indices of Neurons Time City v xi x i Energy Function Hamiltonian-Cycle Satisfaction Cost Minimization Energy Function Each row one or zero neuron on Each column one or zero neuron on n neurons on Energy Function Total distance of the tour Energy Function Build NN for TSP Energy function of a 2-D neural network Mapping Analog Hopfield NN for 10-City TSP The shortest path Analog Hopfield NN for 10-City TSP The shortest path Analog Hopfield NN for 30-City TSP