Person:
Swersky, Kevin

Profile Picture

Email Address

AA Acceptance Date

Birth Date

Research Projects

Organizational Units

Job Title

Last Name

Swersky

First Name

Kevin

Name

Swersky, Kevin
Author's Publications: search.filters.isAuthorOfPublication.1760440a-c6be-4cf3-9fd9-b277b81496f3

Search Results

Now showing 1 - 2 of 2
  • Thumbnail Image
    Publication
    Multi-Task Bayesian Optimization
    (Curran Associates, Inc., 2013) Swersky, Kevin; Snoek, Jasper; Adams, Ryan Prescott
    Bayesian optimization has recently been proposed as a framework for automatically tuning the hyperparameters of machine learning models and has been shown to yield state-of-the-art performance with impressive ease and efficiency. In this paper, we explore whether it is possible to transfer the knowledge gained from previous optimizations to new tasks in order to find optimal hyperparameter settings more efficiently. Our approach is based on extending multi-task Gaussian processes to the framework of Bayesian optimization. We show that this method significantly speeds up the optimization process when compared to the standard single-task approach. We further propose a straightforward extension of our algorithm in order to jointly minimize the average error across multiple tasks and demonstrate how this can be used to greatly speed up \(k\)-fold cross-validation. Lastly, our most significant contribution is an adaptation of a recently proposed acquisition function, entropy search, to the cost-sensitive and multi-task settings. We demonstrate the utility of this new acquisition function by utilizing a small dataset in order to explore hyperparameter settings for a large dataset. Our algorithm dynamically chooses which dataset to query in order to yield the most information per unit cost.
  • Thumbnail Image
    Publication
    Taking the Human Out of the Loop: A Review of Bayesian Optimization
    (Institute of Electrical & Electronics Engineers (IEEE), 2016) Shahriari, Bobak; Swersky, Kevin; Wang, Ziyu; Adams, Ryan Prescott; de Freitas, Nando
    Big Data applications are typically associated with systems involving large numbers of users, massive complex software systems, and large-scale heterogeneous computing and storage architectures. The construction of such systems involves many distributed design choices. The end products (e.g., recommendation systems, medical analysis tools, real-time game engines, speech recognizers) thus involve many tunable configuration parameters. These parameters are often specified and hard-coded into the software by various developers or teams. If optimized jointly, these parameters can result in significant improvements. Bayesian optimization is a powerful tool for the joint optimization of design choices that is gaining great popularity in recent years. It promises greater automation so as to increase both product quality and human productivity. This review paper introduces Bayesian optimization, highlights some of its methodological aspects, and showcases a wide range of applications.