Mining Wikipedia's Article Revision History for Training Computational Linguistics Algorithms

DSpace/Manakin Repository

Mining Wikipedia's Article Revision History for Training Computational Linguistics Algorithms

Citable link to this page

 

 
Title: Mining Wikipedia's Article Revision History for Training Computational Linguistics Algorithms
Author: Nelken, Rani; Yamangil, Elif

Note: Order does not necessarily reflect citation order of authors.

Citation: Nelken, Rani and Elif Yamangil. 2008. "Mining Wikipedia's article revision history for training computational linguistics algorithms." In Wikipedia and Artificial Intelligence: An Evolving Synergy: Papers from the 2008 AAAI Workshop (Chicago, Illinois, 13 July 13, 2008), ed. Razvan Bunescu. Technical Report WS-08-15, pp 31-36. Menlo Park, CA: AAAI Press.
Full Text & Related Files:
Abstract: We present a novel paradigm for obtaining large amounts of training data for computational linguistics tasks by mining Wikipedia's article revision history. By comparing adjacent versions of the same article, we extract voluminous training data for tasks for which data is usually scarce or costly to obtain. We illustrate this paradigm by applying it to three separate text processing tasks at various levels of linguistic granularity. We first apply this approach to the collection of textual errors and their correction, focusing on the specific type of lexical errors known as "eggcorns''. Second, moving up to the sentential level, we show how to mine Wikipedia revisions for training sentence compression algorithms. By dramatically increasing the size of the available training data, we are able to create more discerning lexicalized models, providing improved compression results. Finally, moving up to the document level, we present some preliminary ideas on how to use the Wikipedia data to bootstrap text summarization systems. We propose to use a sentence's persistence throughout a document's evolution as an indicator of its fitness as part of an extractive summary.
Published Version: http://www.aaai.org/Library/Workshops/2008/ws08-15-006.php
Terms of Use: This article is made available under the terms and conditions applicable to Other Posted Material, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAA
Citable link to this page: http://nrs.harvard.edu/urn-3:HUL.InstRepos:11326227
Downloads of this work:

Show full Dublin Core record

This item appears in the following Collection(s)

 
 

Search DASH


Advanced Search
 
 

Submitters