Ensemble-Based Theories of Neural Computation

Abstract

In this thesis, I present several studies toward understanding the properties of ensembles of neurons and neural networks trained or constructed in supervised settings, with applications to both machine learning and cerebellar neuroscience. In Part I, I present studies of learning in ensembles of ridge regression models, with particular attention to the effects of randomly constructed or subsampled input features on ensemble variance and generalization. I suggest ensembles of heterogeneous size as a robust method to mitigate catastrophic overfitting. I then study the trade-off between ensemble size and the size of each ensemble member, proving that no ensemble of random feature models can outperform a single model with the same total number of random features when the ridge parameter is optimally tuned, and identifying conditions under which ensembles may achieve near-optimal performance. In Part II, I turn to cerebellar neuroscience. I first present a model of ensemble-based computation in the cerebellar cortex, suggesting that low-probability error signaling in Purkinje cells implements a biological analogue of the ``bagging'' algorithm from ensemble machine learning, providing computational benefits and unifying a host of experimental observations. Finally, I present a novel analysis of connectivity from parallel fibers to Purkinje cells, alongside a statistical model of synapse formation which suggests that the non-random structure in connectivity is inherited from the phase structure of instructive signals provided by the inferior olive. Together, these studies illuminate effects of randomness in feature selection, regularization, and scale in ensemble learning, and apply an ensemble learning framework and statistical modeling to advance our understanding of the cerebellum.