Pon-Barry, Heather RobertaShieber, Stuart2010-12-012011Pon-Barry, Heather Roberta, and Stuart M. Shieber. 2011. Recognizing uncertainty in speech. EURASIP Journal on Advances in Signal Processing 2011:251753.1687-6172http://nrs.harvard.edu/urn-3:HUL.InstRepos:4596186We address the problem of inferring a speaker’s level of certainty based on prosodic information in the speech signal, which has application in speech-based dialogue systems. We show that using phrase-level prosodic features centered around the phrases causing uncertainty, in addition to utterance-level prosodic features, improves our model’s level of certainty classification. In addition, our models can be used to predict which phrase a person is uncertain about. These results rely on a novel method for eliciting utterances of varying levels of certainty that allows us to compare the utility of contextually-based feature sets. We elicit level of certainty ratings from both the speakers themselves and a panel of listeners, finding that there is often a mismatch between speakers’ internal states and their perceived states, and highlighting the importance of this distinction.en-USJournal Article2010-12-0110.1155/2011/251753