Preaching to the Choir: An AI-Based Analysis of Religious Demand in U.S. Church Sermons, 2000-2023

Abstract

Christianity retains an outsized impact in American society, yet it is very difficult to quantify what congregants hear at church and how it differs across space and time. In this thesis, I provide evidence that sermons respond to demand from the congregation. I construct a dataset with over 150,000 full-text sermons given at churches throughout the United States over the last two decades. I develop a novel text analysis method based on large language models (LLMs) to extract sentiment and other rhetorical attributes from text. I show that this AI-based text quantification tool can easily be applied in any social science research setting, automating the task of human labeling with 1800x reductions in cost. Using this method on the sermons, I show that pastors respond to both political demand and to economic shocks, even when controlling for denominational effects. When economic times get tough, sermons become increasingly pessimistic, less charitable, and less compassionate. Yet there also exists a large racial divide in how churches react to economic hardship. As poverty increases at the census tract level, sermons in white communities become less compassionate and charitable, while those in Black neighborhoods actually increase in compassion and optimism. I verify the analysis through a Bartik-like event study using China's accession to the World Trade Organization to instrument for unemployment. The new LLM research tools developed in this thesis allowed for a far more granular analysis of American churches than previously available, at precinct, tract, and county levels.