Understanding the Algorithms Underlying Human Motor Adaptation and Motor Reinforcement Learning

Abstract

Motor learning allows us to skillfully interact with our environments and is responsible for many of our actions such as hitting a tennis serve, driving a car, or brushing teeth. Yet despite its crucial role in our everyday lives, little is known about the algorithms of the brain that govern this process. This work focuses on understanding 3 important features of the algorithms underlying motor learning: the role of motor variability in motor learning, the interactions between explicit strategies and implicit motor learning, and the spatial reference frames behind impaired motor learning in Alzheimer’s disease. Motor variability is the ubiquitous, often unwanted, variation in our actions from one movement to the next that makes it nearly impossible to repeat a movement exactly. This motor variability is often viewed as noise in our motor system; however, there is another point of view stemming from the theory of reinforcement learning. This view posits that the variability of actions can be a key ingredient for learning by providing exploration of different actions, which can lead to quicker discovery of solutions in motor space. Inspired by reinforcement learning theory, we investigated the possibility that motor variability can facilitate learning. In line with this idea, we found that higher levels of task-relevant motor variability predicted faster learning both across individuals and across tasks in two different paradigms, one relying on reward-based learning and the other relying on error-based learning. Remarkably, we also found that repeated training on a task causes the motor system to correspondingly increase its task-relevant motor variability along with its learning rate, while not changing its task-irrelevant variability. These results suggest that not only does motor variability facilitate motor learning, but also that the motor system may take advantage of this relationship by regulating the structure of its variability according to what it needs to learn. In the second part of this work, I focus on understanding how explicit strategies interact with implicit motor learning. Performance in a motor task can be improved by overtly using explicit strategies or by displaying implicit adaptation; however, little is understood about how these processes interact and potentially interfere with each other. In typical motor adaptation tasks that assess implicit and explicit learning, studying their interactions is challenging because they are not only responding to each other but also are both responding to the perturbation. Here we designed a motor adaptation task that allows us to isolate the interactions between implicit and explicit learning from their responses to the perturbation. In doing so, we reveal that implicit motor learning robustly compensates for the inappropriate behavior of low-fidelity explicit learning, providing a rigorous, mechanistic basis for why strategies may be detrimental, in line with the popular advice to ‘avoid overthinking’ in motor skill learning. The result that implicit learning responds to explicit strategy, however, raises the question of “how.” Previous work has shown that implicit motor learning is driven by sensory-prediction error, whereas explicit strategies are driven by performance error. However, since strategy use can only affect performance errors and not sensory prediction errors, if implicit learning were driven purely by sensory-prediction error, it would have to proceed independently of explicit strategy. Thus we wondered if different parts of implicit learning respond differently to sensory-prediction error and performance error. To this end, we dissected implicit learning into components based on their temporal stability to understand how they respond to different error signals. By probing how these processes decay across rest breaks, we decomposed implicit learning into its temporally-stable and temporally-labile components, and found that temporally-labile implicit learning strongly depends on different levels of strategy, whereas temporally-stable implicit learning is invariant to different levels of strategy. These results indicate that implicit learning responds to strategy through its temporally-labile component: temporally-labile implicit learning responds to strategy, driven by performance error, whereas temporally-stable implicit learning proceeds independently of strategy, driven by sensory-prediction error. In the final part of this work, I investigate how spatial reference frames for motor learning are impaired in Alzheimer’s disease. Although Alzheimer’s disease is best known for its impairments of cognitive abilities, it also profoundly impairs motor skills that are critical to daily life. However, it is unclear why this disease impairs some motor skills but not others. We wondered if the impairment of these motor skills might be related to the use of spatial information, as motor learning inherently relies on spatial information in planning movements. In particular, motor learning can depend on two important types of spatial information: extrinsic spatial information relative to objects in the world, and intrinsic spatial information relative to the body. In this study, we dissociated intrinsic and extrinsic learning in Alzheimer’s disease by investigating how motor learning transfers from one posture to another. We found that the ability to form and retain motor memories is largely intact in Alzheimer’s disease participants when either type of spatial information can be used to perform the task, exhibiting similar levels of overall motor learning as control participants. However, when we probe the intrinsic vs extrinsic composition of this learning, we found that Alzheimer’s disease participants demonstrate largely intrinsic, rather than extrinsic representations of this learning compared to control participants. These results indicate a specific impairment of extrinsic learning in Alzheimer’s disease and suggest rehabilitation strategies for preserving motor skills that are critical for maintaining independence.