Stable and Efficient Representation Learning with Nonnegativity Constraints

DSpace/Manakin Repository

Stable and Efficient Representation Learning with Nonnegativity Constraints

Citable link to this page


Title: Stable and Efficient Representation Learning with Nonnegativity Constraints
Author: Lin, Tsung-Han; Kung, H. T.

Note: Order does not necessarily reflect citation order of authors.

Citation: Lin, T. H., and H. T. Kung. 2014. "Stable and Efficient Representation Learning with Nonnegativity Constraints." In Proceedings of the 31st International Conference on Machine Learning (ICML 2014), Beijing, China, June 22-24, 2014. Journal of Machine Learning Research: W&CP 32: 1323-1331.
Full Text & Related Files:
Abstract: Orthogonal matching pursuit (OMP) is an efficient approximation algorithm for computing sparse representations. However, prior research has shown that the representations computed by OMP may be of inferior quality, as they deliver suboptimal classification accuracy on several im- age datasets. We have found that this problem is caused by OMP’s relatively weak stability under data variations, which leads to unreliability in supervised classifier training. We show that by imposing a simple nonnegativity constraint, this nonnegative variant of OMP (NOMP) can mitigate OMP’s stability issue and is resistant to noise overfitting. In this work, we provide extensive analysis and experimental results to examine and validate the stability advantage of NOMP. In our experiments, we use a multi-layer deep architecture for representation learning, where we use K-means for feature learning and NOMP for representation encoding. The resulting learning framework is not only efficient and scalable to large feature dictionaries, but also is robust against input noise. This framework achieves the state-of-the-art accuracy on the STL-10 dataset.
Published Version:
Other Sources:
Terms of Use: This article is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at
Citable link to this page:
Downloads of this work:

Show full Dublin Core record

This item appears in the following Collection(s)


Search DASH

Advanced Search