Coarse-to-Fine Attention Models for Document Summarization

Abstract

While humans are naturally able to produce high-level summaries upon reading paragraphs of text, computers still find such a task enormously difficult. Despite progress over the years, the general problem of document summarization remains mostly unsolved, and even simple models prove to be hard to beat. Inspired by recent work in deep learning, we apply the sequence-to-sequence model with attention to the summarization problem. While sequence-to-sequence models are successful in a variety of natural language processing tasks, the computation does not scale well to problems with long sequences such as documents. To address this, we propose a novel coarse-to-fine attention model to reduce the computational complexity of the standard attention model. We experiment with our model on the CNN/Dailymail document summarization dataset. We find that while coarse-to-fine attention models lag behind state-of-the-art baselines, our method learns the desired behavior of attending to subsets of the document for generation. Therefore, we are optimistic that the general approach is viable as an approximation to state-of-the-art models. We believe that our method can be applied to a broad variety of NLP tasks to reduce the cost of training expensive deep models.