Improving Response Diversity for Dialogue Systems

Abstract

An ideal dialogue system should return coherent and purposeful responses. Recently, interest has shifted from dialogue systems with specialized components for recognizing intent and achieving coherence to a single model that learns responses end-to-end. However, a common problem for these end-to-end dialogue systems is the "I don't know" problem where the system simply returns vague phrases. In this thesis, we examine how to improve the diversity of outputs for dialogue systems. We train a sequence-to-sequence model with attention on a dataset that contains 2.3 million conversations from an assortment of movies. To implement a baseline, we run beam search to generate responses with the highest probabilities given the conversational context. Similar to previous work, we notice vague responses that could apply to a broad set of contexts. In order to make replies more interesting, we explore modifications to beam search that optimize for responses with higher contextual relevance. Ultimately, we find that a combination of these techniques results in more diverse responses than the baseline approach according to commonly used metrics. Further analysis of sampled responses suggests that these replies are generally contextually relevant. Past work in dialogue response diversity has been sprawled across different datasets and inconsistent metrics, which has made progress difficult to monitor. Consequently, we hope that our results can provide more clarity for future work on dialogue response diversity.