Turing’s Child Machine: A Deep Learning Model of Neural Development

Abstract

The connection between general intelligence and development was first raised by Turing in the 1950s. Noting the incredible complexity of engineering the adult mind, he proposed instead building a child machine with a simple mind that develops into a complex adult one. The goal of building an intelligent machine, then, is refocused around engineering the neurological stages of development. It was later discovered that during development the child’s brain undergoes pervasive network expansion, increasing at least an order of magnitude in size, followed by activity driven pruning - a process for which the computational role is still unknown. In the parallel world of artificial intelligence research, hand-designing deep neural networks involves architecture decisions which are often guided by intuition and trial-and-error. Architectures typically stay fixed during parameter learning, a stark contrast to the extreme architecture modifications that take place during development in a child’s brain. Furthermore, it is widely understood that there is a ’small network design problem’ when hand-designing deep networks. Building a smaller deep architecture that generalizes as well as a big one requires much more effort as well as more complex layers and connections. Here we model biological development using densely connected as well as convolutional deep neural networks as a means to further our understanding of neurological biological intelligence and computational artificial intelligence. Empirical results suggest the computational role of synaptic overgrowth and pruning in biology is as an unsupervised architecture search process that finds exponentially smaller architectures that generalize well. Resultant ’adult’ convolutional networks that develop this way also show similarities to hand-designed networks.