The Learning Hypothesis on Spatial Receptive Field Remapping

Abstract

The spatial pattern of grid cells changes due to information such as the location of rewards. Past research has discovered that after rats learn the location of rewards in a cheeseboard environment, their grid cells activate more near reward locations, distorting the grid pattern. However, it is unknown whether this distortion has any purpose at a population level. In this work, we use reinforcement learning (RL) theory to explain how changes in spatial activation patterns could be the result of the agent trying to optimize learning progress such as learning speed or convergence learning error. Here, we assume that distortions of the grid patterns correspond to changes in features in temporal difference (TD) learning. First, we show that with fixed features, parameters such as learning rate (η) and batch size (B) linearly change the TD learning convergence learning error. Specifically, the convergence learning error is proportional to O(η/B). Though this result depends on a Gaussian approximation when feature dimension is high, we demonstrate the linear scaling phenomenon on continuous complex environments such as MountainCar-v0. Second, we show that using an evolutionary method to adjust radial basis function features (place cell features) would evolve features that are centered near reward locations. Finally, we use experimental data from rats' medial entorhinal cortex (MEC) to show that the features after learning provide faster TD learning convergence speed and lower convergence error. These results support the theory that animals change their perceptions to optimize learning progress.