The Neural Representation of Structured Complex Ideas in Different Formats

Abstract

Darth Vader has waded knee-deep into the Black Sea and is filtering some of the water with a Brita pitcher. Even though you’d never read that sentence before, you immediately understood its specific, improbable meaning. Moreover, you can imagine that scene, even though you’ve never seen it before. How is it that humans are able to understand and imagine novel, complex ideas when we first encounter or conceive them? This example highlights the intersection of two defining features of human cognition. First, thought is compositional. We can combine familiar conceptual components in precise ways to build novel composite thoughts. Second, we can represent ideas in different formats—including both symbolic, language-like representations and depictive mental images—and the brain can translate complex ideas between representational formats. In this dissertation, I describe three neuroimaging experiments directed at advancing our understanding of how the brain implements compositional representation and translates composite ideas from one representational format to another. Experiment 1 investigated where and how the brain represents composite ideas that are defined by spatial structure. Participants undergoing fMRI read descriptions of arrangements of shapes and imagined those arrangements. Structure-dependent information was found bilaterally in the supramarginal gyrus and superior parietal lobule, along the left intraparietal sulcus, and in left anterior middle frontal gyrus. Experiment 2 took a similar approach for ideas with temporal structure, using sequences of described and imagined sounds. Experiment 3 used an individual-focused approach to investigate the translation of composite ideas between formats. We collected over 25 hours of fMRI data from each of three individuals as they compared the meaning of ideas conveyed in images and sentences. We identified a number of regions—most notably the left lateral posterior temporal cortex—that were preferentially engaged in all three participants when comparing ideas expressed in different representational formats.