Physics- and biology-inspired learning dynamics for artificial neural networks

Abstract

Artificial neural networks (ANNs) with their expressive power have shown great progress recently and have found applications across a wide range of disciplines. Although ANNs' hidden graph structures allow them to be promising in a given task, their training method does not guarantee that the network will perform well on the task. ANNs were originally inspired by biological neural networks in the brain. However, their evolution has followed a different trajectory, i.e., their training in modern architectures and models is considered biologically implausible. According to the Hebb rule, the strengths of the synapses between two neurons are determined by the activities of those neurons. This is called the locality principle. Synaptic plasticity in a biological neuron, along with the locality principle, is also regulated by a neuromodulatory signaling mechanism that reflects global contextual and sensory information, such as attention, mood, and arousal, operating on certain synapses and neurons, thereby regulating the efficacy of neurons and synapses. However, most modern ANN models and architectures do not involve such a mechanism. Furthermore, a majority of modern ANN models employ the backpropagation (BP) algorithm in their training, which violates the locality principle. In this regard, there are attempts to develop training algorithms that obey biological principles, which are called biologically plausible learning algorithms. In particular, the ones developed based on the locality principle are known as Hebbian learning rules. Given the training of ANNs, the stochastic gradient descent (SGD) algorithm and its variations play a central role and can be adopted for Hebbian rules as well. More importantly, it has a physics origin in nonequilibrium statistical physics, in conjunction with the Langevin dynamics. This connection allows SGD to be improved and investigated thoroughly via methods of statistical physics. Furthermore, the physics influence on ANNs has led to the development of energy-based models, a feedback (or recurrent) neural network type, e.g., Hopfield networks and Boltzmann machines (BMs), which are closely related to spin systems with respect to their energy functions. In particular, their training does not violate the locality principle. Within the scope of this thesis, we first provide a detailed survey on essential tools for ANNs, in particular, restricted BMs (RBMs) and their training by means of physics origin. The RBMs are a family of BMs, and their generative property is grounded on the Boltzmann distribution. The interacting graph structure of the RBMs (or more generally, the family of BM models) makes them nontrivial to train due to the intractability of partition function computation. Therefore, we review some approximations developed in statistical physics for their training, such as Markov chain Monte Carlo and mean-field-like approximations. Furthermore, we provide a review on the biological origin of learning and biologically plausible learning algorithms based on the Hebbian rules. Finally, we introduce our proposal, in which we employ the Krotov-Hopfield (KH) algorithm, a biologically plausible learning algorithm, as a neuromodulatory signaling mechanism for RBMs, a class of generative feedback networks, and multilayer perceptrons (MLPs), a class of discriminative, feedforward networks. The KH algorithm, with its built-in competition scheme arising from homeostatic constraint and global inhibition employs a simplified attention mechanism. This constitutes a proper neuromodulatory signaling framework for ANNs, as neuromodulation serves as a secondary communication among the neurons without the need for direct synapse connections. In our proposal, we adopt this algorithm in a stochastic gradient Langevin dynamics casting at the training process of RBMs and MLPs. We implement this concept with two different schemes, namely top-down (cognition-driven) and bottom-up (sensory-driven). We first apply KH modulation in RBMs for the reconstruction task using MNIST and Kuzushiji MNIST datasets. We then extend the application of KH modulation onto class RBMs (cRBMs), a variant of the RBM developed for the classification task. In each RBM configuration, we use different hidden unit sizes, different weight initializations, and different Gibbs sampling steps. With the framework we proposed, we observed a tenfold reduction in the number of parameters and superior accuracy in cRBMs. Similarly, in the reconstruction performed with RBMs, our proposal has demonstrated a better reconstruction. Furthermore, the framework has shown robustness to overfitting and tolerance to weight initialization in all of the RBM varieties. Since the adoption of the neuromodulation mechanism forms a Langevin-like casting, we conducted experiments to observe whether the performance improvement stems from the steering updates provided by the neuromodulation, or whether those updates form a noisy structure, thereby leading to finding better solutions. We found that the correlation between modulation and the standard gradient update exhibits a slight negative correlation, leading to exploration in the energy landscape. To conclude this, we also performed an experiment in which modulation updates were shuffled to form a noisy structure, which did not yield any performance gain. This explicitly demonstrated that performance gains are due to the steering neuromodulation rather than noisy updates. Similarly, we repeated the KH modulation in the MLP, accompanying its standard training. The experiments showed that the KH modulation schemes are superior to the standard MLP. We also repeated the experiment in which we measured the correlation between KH modulation and standard gradient updates for the MLPs. The results verified the findings we obtained for the RBMs. Our findings, overall, suggest that by employing schemes similar to the proposed framework in more complex and modern architectures, one can bridge the gap between biologically plausible learning algorithms and state-of-the-art learning models. Furthermore, this interaction may lead to the development of novel high-performance techniques.