The Geometry of Language Model Hallucination

Abstract

Hallucination, or the generation of fluent but factually incorrect text, is a central challenge in the deployment of large language models. No language model can avoid it entirely, and as these models are increasingly used in domains where factual correctness is consequential, the challenge shifts from eliminating hallucination to predicting where it will occur and reducing it where it matters most. This thesis investigates whether the geometric structure of a model's input space can predict hallucination, guide intervention, and inform permanent model improvement. We construct a benchmark spanning seven categories of knowledge failure, evaluate ten language models from three provider families, and show that geometric properties of prompt embeddings---computed from the question text alone, before generation---predict which prompts will cause hallucination. We decompose this signal and find that much of its predictive power reflects differences between question types. But within a single question type, prompts in sparser embedding neighborhoods are significantly more likely to trigger hallucination. We then design four targeted system-prompt interventions that substantially reduce hallucination while maintaining accuracy, and replicate this result at five times the original benchmark scale. The same geometric features that predict hallucination also predict which hallucinations resist intervention, connecting geometry to hallucination difficulty, not just occurrence. Finally, we distill the most effective prompt behavior into model weights through parameter-efficient fine-tuning, producing models that match the best intervention's accuracy without runtime engineering. Three independent generalization tests confirm that the models learned general epistemic caution rather than surface-level memorization. Ultimately, hallucination is not random; it has a structure that is detectable from the input alone, and that structure can guide its correction.