Automated neuron tracking inside moving and deforming C. elegans using deep learning and targeted augmentation

Abstract

Recent advances allow sustained rapid 3D functional imaging of large numbers of neurons inside behaving animals. To decode neural activity, individual neurons must be segmented and tracked. This can be challenging inside brains that rapidly move and deform within flexible, behaving animals. No general method is able to effectively solve this problem. We present an analysis method based on a convolutional neural network (CNN) with specific enhancements, which we apply to freely moving C. elegans. For a traditional CNN to track neurons across brain images with different postures, the CNN must be trained with ground truth (GT) annotations of similar postures. When these postures are diverse, adequate numbers of manual GT annotations can become prohibitively large to generate. We introduce `targeted augmentation', a method to automatically synthesize reliable annotations from a few manual annotations. Our method effectively learns the internal deformations of the brain. These learned deformations are used to synthesize annotations for new postures by deforming the manual GT annotations of similar postures. When synthetic annotations are added to training datasets, the need for manual GT annotation and final proofreading are reduced. We provide a GUI for our method in an end-to-end pipeline from manual GT annotation to final proofreading. We demonstrate our method by segmenting and tracking neurons -- where individual neurons can be represented either as points or 3D volumes -- in a variety of C. elegans strains across multiple animals. In a study of olfactory processing, our method uncovered rich patterns in interneuron dynamics in freely moving animals exposed to periodic odor pulses, including dynamic on-off switching of neuronal entrainment to time-varying stimuli.