Hopfield modeli yapay sinir ağları ve uygulamaları

Abstract

Yapay sinir ağlarına olan ilgi gün geçtikçe artmaktadır. İnsan ve hayvanların kolayca gerçekleştirdikleri bazı işlevleri günümüzde kullanılan bilgisayarlar ve sayısal devreler ile gerçekleştirmek çok zordur. Hopfield modeli, yapay sinir ağları arasında önemli bir yeri olan ve yeni çalışmalar için daima bir çıkış noktası oluşturan bir modeldir. Hopfield modeli pek çok optimizasyon problemine uygulanmış olup, gerçek biyolojik sistemlere oldukça uygun bir yapıya sahiptir. Bir enerji fonksiyonundan hareketle ağ bağlantı yapısının belirlenmesi önemli bir özelliktir. Ayrıca Hopfield modelinin devre gerçeklemesi de kolaydır. Bu çalışmada, Hopfield modeli yapay sinir ağları incelenmiş ve bir bilgisayar simülasyon programı oluşturulmuştur. Simülasyon programı kullanılarak modelin çeşitli uygulamalarda davranışı incelenmiş, global minimum bulunması için çalışmalar yapılmıştır. Belirli bir problem için, problemin enerji fonksiyonu olarak ifadesinin ve parametrelerin belirlenişinin önemi görülmüştür. Gezgin satıcı probleminin çözümü için enerji fonksiyonu yeni bir biçimde oluşturulmuştur. Hopfield modeli yapay sinir ağları için, devre parametrelerinin çözülecek problemden hareketle belirlenebilmesi üzerinde çalışılmış ise de, çözümü garanti eden belirli bir yeter koşul elde edilememiştir.

Brain is a computer made of organic materials and wet chemistry but, is one of the world's best computers. There are problems in which, like multiplication, every data bit must be taken very seriously. Brains are very bad at such problems. There are other problems -such as recognition of faces- which people do well but which digital electronic machines do very ineffectively. One can try to make quantitative the idea of how good or how bad brains and digital machines are in computation. In so doing, howewer, one should factor out the issue of intrinsic device speed. For comparison, silicon devices have an intrinsic speed about 100000 times faster than that of natural neurobiological devices. On the other hand, for solving problems like face recognition it becomes apperant that the neurobiological system is more effective by a factor of 10. This facts illustrate why people are interested in artificial neural networks. Increasingly, computers are being asked to do tasks that were formerly only done by their biological counterparts. And digital computers, as often as not, can not do those tasks. The question then arises as to whether we can make electronic architectures that are more like neurobiological architectures and that are better than conventional architectures for applications like face recognition or decoding of natural language. Brain is analog and parallel. Brain is consists of an enarmous number of simple cells, called neurons and neurons are working in a highly parallel architecture.A neuron can be studied as an input/output device. While details vary, neurons fundementally transmit pulse coded analog information. The input/output relation of this pulse-coded system is a simple sigmoid, easily built electronically. It is not simply an amplifier. The sigmoid characteristic is very important to neural computational properties in general. For the most part, linear systems do not make decisions and, thus, do not compute, although they are important adjuncts to computational systems. Neurobiologic system behaves like an analytical, dynamic system. Neural system computation is described by an equation of motion. Neurobiological system uses time delays as a part of its processing. Electronic circuit is thought of as a fixed circuit, in neurobiology the structure of the circuit changes every time learning occurs. All of the above-described features of neurobiology could be adapted into electronic devices. Today, engineers try to build more brain-like computers out of neuron-like parts. It was the view that led to the development of many of the earlier models of neurons and neural networks. McCullough and Pitts in the 1940s showed that the neuron can be modeled as a simple threshold device to perform logic function. Their work was significant because of showing that the simple units can perform complex function when working in parallel. By the late 1950s and early 1960s neuron models were further defined into Rosenblatt's Perceptron, Widrow and Hoff's Adaline and Steinbuch's Learning Matrix. The Perceptron received considerable excitement when it was first introduced because of its conceptual simplicity. However was short lived when Minsky and Papert proved mathematically that it can not be use to realize the complex logic functions which are not linearly seperable. On the other hand, the faith of Adaline was quite different. Because of its linear and adaptive nature, the technic has developed into a powerful tool for adaptive signal processing. The present interest on neural network research is due inpart to the paper Hopfield published in 1982. This and the other in 1984 are two highly readable papers in neural networks. In some circles, there developed a confusion that Hopfield had invented VIneurocoroputing, Hopfield presented a model of neural computation that was based on the interaction of neurons. The model consisted of a set of first order non-linear differential equations that minimize a certain energy function. Hopfield and Tank, started a new generation of neural networks research which led the world to a new kind of thinking. Hopfield' s first model was discrete and stochastic. But the second model was based on continuous variables and responses. The second model retains all the significant behaviours of the first model. The continuous model can be represented as follows: The output variable Vi for neuron i is a continuous and monotone-increasing function of the instantaneous input Ui to neuron i. The typical input-output relation gi (ui ) is sigmoid. For neurons exhibiting action potentials, ui could be thought of as the mean soma potential of a neuron from the total effect of its excitatory and inhibitory inputs. Vi can be viewed as the short-term average of the firing rate of the cell i. In terms of electrical circuits, gi(ui) represents the input-output characteristic of a nonlinear amplifier with negligible response time. In a biological system, ui will lag behind the instanteneous outputs Vj of the other cells because of the input capacitance Ci of the cell membranes, the transmembrane resistance R, and the finite impedance Tl j between the output Vj and the cell body of cell i. Thus there is a resistance-capacitance (RC) charging equation thet determines the rate of change of ui. C du /dt i i V. = g(u. ) -u /R + V T V +1 i i. i j j i TijVj represents the electrical current input to cell i due to the present potential of cell j, and Tij is thus the synapse efficacy. Tij of both signs should occur. I i is any other (fixed) input current to neuron i. vi lThe same set of equations represents the resistively connected network of electrical amplifiers sketched in Figure 1. Inhibition can be represented by an inverting amplifier. The magnitude of Ti j is 1/Rij, where Ri j is the resistor connecting the output of j to the input line i, while the sign of Ti j is determined by the choice of the positive or negative output of amplifier j at the connection site. Ri is 1/R. = 1/p. + I 1/R. where pi is the input resistance of amplifier i. Ci is the total input capacitance of the amplifier i and its associated input lead. neuron V amplifier. resistor in Tjt network Y, inverting amplifier Figure 1. An electrical circuit for Hopf ield model There is a Liapunov function for the circuit: E - -1/2 E E T..V.V. + l (1/R. ) 1J1J * 4 J J - i nJ ı, J 1 g^tVJdV 1 - I ı.v, Its time derivative for a symmetric T. is dE/dt - - V dV /dt (V T V - u /R + I ) VI 11dE/dt = - V C {dV/dt}{du /dt ) i i dE/dt = - I C.gT1' (V. ){dV./dt): Since gi (V ) is a monotone-increasing function and Ci is positive, each term in this sum is nonnegative. Therefore dE/dt^O; dE/dt=0 - - dV. /dt=0 for all i E is bounded. The time evolution of the system is a motion in state space that seeks out minima in E and comes to a stop at such points. This minima is local. Hopfield network can be used for solving optimization problems rapidly. But a globally optimal solution to the problem is not guaranteed. In difficult problems where rapidly calculated good solutions may be more benefical than slowly computed globally optimal solutions. But for applications which globally optimal solutions must be found, the hardware annealing method can be used. In this method, gain of the amplifiers is increased from minimum value to the maximum. Hopfield network can be used in many applications such A/D converter, signal decomposition, linear programming, travelling salesman problem, system identification, object recognition, stereo vision correspondance. In these applications, circuit parameters is specified by corresponding energy function of the Hopfield network and the function of the problem to be minimized. But this specification generally consists some trials. Here must be done more research. In this work, a simulation programme for Hopfield continuous model is realized and used for many applications. In these applications, with hardware annealing method good results is generally obtained. In simulation, instead of solving differential equations of the system by numerical integration methods like Euler or Runge-Kutta, programming directly of differance equations which is obtained by discrete model of the system is used. These differance equations of the system are IX-T/r -T/T u (k+l) = (l-e * )R [I +£T V U)]+e Ju (k) İ 1 i " 1 j j 1 j The determination of circuit parameters for application may be a further research.