Contrastive Learning Using Spectral Methods

DSpace/Manakin Repository

Contrastive Learning Using Spectral Methods

Citable link to this page


Title: Contrastive Learning Using Spectral Methods
Author: Zou, James Yang; Hsu, Daniel; Parkes, David C.; Adams, Ryan Prescott

Note: Order does not necessarily reflect citation order of authors.

Citation: Zou, James, Daniel Hsu, David Parkes and Ryan Adams. 2013. Contrastive learning Using spectral methods. In Proceedings of Advances in Neural Information Processing Systems 26 (NIPS 26), ed. C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger, Lake Tahoe, UT, December 5-10, 2013.
Full Text & Related Files:
Abstract: In many natural settings, the analysis goal is not to characterize a single data set in isolation, but rather to understand the difference between one set of observations and another. For example, given a background corpus of news articles together with writings of a particular author, one may want a topic model that explains word patterns and themes specific to the author. Another example comes from genomics, in which biological signals may be collected from different regions of a genome, and one wants a model that captures the differential statistics observed in these regions. This paper formalizes this notion of contrastive learning for mixture models, and develops spectral algorithms for inferring mixture components specific to a foreground data set when contrasted with a background data set. The method builds on recent moment-based estimators and tensor decompositions for latent variable models, and has the intuitive feature of using background data statistics to appropriately modify moments estimated from foreground data. A key advantage of the method is that the background data need only be coarsely modeled, which is important when the background is too complex, noisy, or not of interest. The method is demonstrated on applications in contrastive topic modeling and genomic sequence analysis.
Published Version:
Terms of Use: This article is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at
Citable link to this page:
Downloads of this work:

Show full Dublin Core record

This item appears in the following Collection(s)


Search DASH

Advanced Search