Mining Wikipedia's Article Revision History for Training Computational Linguistics Algorithms
View/ Open
Published Version
http://www.aaai.org/Library/Workshops/2008/ws08-15-006.phpMetadata
Show full item recordCitation
Nelken, Rani and Elif Yamangil. 2008. "Mining Wikipedia's article revision history for training computational linguistics algorithms." In Wikipedia and Artificial Intelligence: An Evolving Synergy: Papers from the 2008 AAAI Workshop (Chicago, Illinois, 13 July 13, 2008), ed. Razvan Bunescu. Technical Report WS-08-15, pp 31-36. Menlo Park, CA: AAAI Press.Abstract
We present a novel paradigm for obtaining large amounts of training data for computational linguistics tasks by mining Wikipedia's article revision history. By comparing adjacent versions of the same article, we extract voluminous training data for tasks for which data is usually scarce or costly to obtain. We illustrate this paradigm by applying it to three separate text processing tasks at various levels of linguistic granularity. We first apply this approach to the collection of textual errors and their correction, focusing on the specific type of lexical errors known as "eggcorns''. Second, moving up to the sentential level, we show how to mine Wikipedia revisions for training sentence compression algorithms. By dramatically increasing the size of the available training data, we are able to create more discerning lexicalized models, providing improved compression results. Finally, moving up to the document level, we present some preliminary ideas on how to use the Wikipedia data to bootstrap text summarization systems. We propose to use a sentence's persistence throughout a document's evolution as an indicator of its fitness as part of an extractive summary.Terms of Use
This article is made available under the terms and conditions applicable to Other Posted Material, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAACitable link to this page
http://nrs.harvard.edu/urn-3:HUL.InstRepos:11326227
Collections
- FAS Scholarly Articles [18292]
Contact administrator regarding this item (to report mistakes or request changes)