Theory of Learning in Neural Networks with Small Weight Perturbations

Abstract

Learning multiple tasks in a non-stationary world requires continual learning (CL) -- the ability to accumulate and refine knowledge and skills over time. In neural networks (NN), realizing CL requires balancing stability (retaining benefits of previous learning in weights), and plasticity (efficient acquisition of new information). While CL occurs mundanely for biological NNs in the brain, artificial NNs in machine learning (ML) often fail catastrophically. This contrast poses two questions: (1) how does the brain handle the stability-plasticity dilemma and realize CL? (2) what causes CL to fail in artificial NNs and what could be done to rescue it? Towards answering these questions, this work presents a theoretical treatment of CL in NNs equipped with a simple, analytically tractable mechanism -- a weight-perturbation penalty that constrains the learning process to make small perturbations to weights.

We first tested how the need to reduce learning-induced perturbations can explain neural mechanisms behind perceptual learning (PL) -- a well-studied experimental paradigm where animals exhibit long-lasting improvement in perceptual tasks following extensive training. While PL-induced physiological changes in sensory cortical areas are well documented, normative and mechanistic explanations of them are lacking. We hypothesized that the criticality of these areas for a broad range downstream tasks gives stability paramount importance. Thus, such areas should be modified with \textit{minimum} perturbations (MP). To study its implications, we modeled the sensory hierarchy as a deep NN and developed a mean-field theory of the network in the limit of a large number of neurons and large number of examples. Our theory suggests that the input-output function of the network can be exactly mapped to that of a deep linear network, allowing us to characterize the space of solutions for the task as well as the MP solution within it. Interestingly, MP plasticity induces changes to weights and neural representations in all layers of the network, except for the readout weight vector. While weight changes in higher layers are not necessary for learning, they help reduce overall perturbation to the network. MP plasticity predicts physiological and behavioral changes that are largely consistent with experimental observations, suggesting MP as one of the potential learning principles in sensory areas in the adult brain.

Generalizing beyond the setting of PL, we then used tools from statistical physics to develop a comprehensive theory of deep NNs learning sequences of \textit{arbitrary} tasks with small weight perturbations. Our analytical results exactly describe how the input-output mapping of the network evolves as more tasks are learned sequentially. The degree of forgetting and transfer during CL is theoretically connected to relations between tasks, the network's architecture, and hyperparameters of the learning process. Of note, the theory identifies two scalar order parameters (OP) that succinctly capture input and rule similarity between tasks and suggests that they play related but diverging roles in determining CL outcomes. These OPs, directly computed from task data, are highly predictive of CL performance across a wide range of settings. The analysis also reveals how the architecture, including depth and whether there are task-dedicated readouts, strongly modulates the connection between task relations and CL performance. In particular, when the network contains task-dedicated readouts, our theory predicts three dramatically different CL regimes (or “phases”), determined by the task OPs and the amount of training data available. Sequentially learning tasks that are too dissimilar, as measured by the OPs, can lead to the surprising phenomenon of “catastrophic anterograde interference”, where the network reaches zero training error on the new task but fails to generalize. Our results provide a rigorous treatment of the rich phenomena of CL in deep NNs and distinguish critical factors that promote or hinder CL.

In conclusion, this work presents a theoretical analysis of how the need of stability-plasticity balance shapes learning in NNs. We hope that the results lay groundwork for further insights into neural mechanisms underlying CL in the brain as well as inspire practical algorithms for CL in artificial intelligence systems.