Input Warping for Bayesian Optimization of Non-Stationary Functions

DSpace/Manakin Repository

Input Warping for Bayesian Optimization of Non-Stationary Functions

Citable link to this page


Title: Input Warping for Bayesian Optimization of Non-Stationary Functions
Author: Snoek, Jasper; Swersky, Kevin; Zemel, Rich; Adams, Ryan Prescott

Note: Order does not necessarily reflect citation order of authors.

Citation: Snoek, Jasper, Kevin Swersky, Richard S. Zemel, and Ryan P. Adams. 2014. "Input Warping for Bayesian Optimization of Non-stationary Functions." In Proceedings of The 31st International Conference on Machine Learning, Beijing, China, June 22-24, 2014. Journal of Machine Learning Research: W&CP 32: 1674–1682.
Full Text & Related Files:
Abstract: Bayesian optimization has proven to be a highly effective methodology for the global optimization of unknown, expensive and multimodal functions. The ability to accurately model distributions over functions is critical to the effectiveness of Bayesian optimization. Although Gaussian processes provide a flexible prior over functions, there are various classes of functions that remain difficult to model. One of the most frequently occurring of these is the class of non-stationary functions. The optimization of the hyperparameters of machine learning algorithms is a problem domain in which parameters are often manually transformed a priori, for example by optimizing in “log-space”, to mitigate the effects of spatially-varying length scale. We develop a methodology for automatically learning a wide family of bijective transformations or warpings of the input space using the Beta cumulative distribution function. We further extend the warping framework to multi-task Bayesian optimization so that multiple tasks can be warped into a jointly stationary space. On a set of challenging benchmark optimization tasks, we observe that the inclusion of warping greatly improves on the state-of-the-art, producing better results faster and more reliably.
Published Version:
Other Sources:
Terms of Use: This article is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at
Citable link to this page:
Downloads of this work:

Show full Dublin Core record

This item appears in the following Collection(s)


Search DASH

Advanced Search