A computational framework for auditory selective attention

Abstract

Attention is essential for interaction with the world. Human selective listening is perhaps the most ubiquitous of selective attention abilities, enabling daily communication by allowing humans to listen to one person talking while ignoring others. Neurophysiology implicates multiplicative feature gains as a mechanism of attentional selection, enhancing the neural representation of the attended source. However, the role of such gains in mediating real-world attention-driven behavior has remained unclear. Decades of psychophysical experiments have characterized how attentional selection depends on both the physical properties of a stimulus (e.g., the color, pitch or location of an item) and task constraints (e.g., reporting a spoken word or finding an object in a scene), but little is known about why attentional selection is successful in some conditions but not others. Parallel advances in artificial intelligence have delivered artificial neural networks that can perform at human-level and, in some cases, represent information similarly to the brain. However, reproducing the phenotypes of human attentional behaviors in such systems has remained an open challenge. This dissertation introduces a computational framework linking multiplicative feature gains to selective listening behaviors, training artificial neural networks to perform selective listening (the “cocktail party problem”). There are three main contributions of this dissertation:

Chapter 2: We optimized an artificial neural network with stimulus-computable feature gains to recognize a cued talker’s speech from binaural audio in ‘cocktail party’ scenarios. Though not trained to mimic humans, the model produced human-like performance across diverse real-world conditions, exhibiting selection based both on voice qualities and on spatial location as well as selection failures in conditions where humans tended to fail. It also predicted novel attentional effects that we confirmed in human experiments and exhibited signatures of ‘late selection’ like those seen in human auditory cortex. The results suggest that human-like attentional strategies naturally arise from the optimization of feature gains for selective listening.

Chapter 3: We introduce a computational motif for how listening effort interacts with feature-gains to enable attentional selection. We conceive of effort as the extent to which feature-specific-gains are applied in an artificial neural network optimized for selective listening. When effort is high, features that are not in the target of attention are attenuated; when effort is low, all features are “passed” through the model. We tested the modeling framework by measuring the effect of effort on selective listening tasks. The model predicts graded effects of effort, showing benefits of high effort when attending to speech targets among speech or spatially separated distractors, but not among non-speech distractors.

Chapter 4: Music is another domain in which auditory attention is important, as it allows people to selectively listen to specific melodies within multi-instrument pieces. As a prelude to modeling attention in this domain, we introduce models of human relative pitch judgements. We optimized a repertoire of artificial neural networks to perform musical interval transcription under various training constraints to test whether and under what conditions the phenotypes of human pitch perception emerge. We found that several training objectives yielded models that qualitatively reproduce signatures of human relative pitch judgments. We show that explicitly optimizing for absolute (f0 estimation) rather than relative pitch does not yield the same effects. The results suggest that aspects of human pitch perception reflect an optimized strategy for relative pitch estimation.