Bayesian Optimisation over Multiple Continuous and Categorical Inputs (2019-06-20T00:00:00.000000Z)

TL;DR

This work proposes a new approach, Continuous and Categorical Bayesian Optimisation (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimisation to select values for both categorical and continuous inputs, and model this mixed-type space using a Gaussian Process kernel.

Abstract

Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges. We propose a new approach, Continuous and Categorical Bayesian Optimisation (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimisation to select values for both categorical and continuous inputs. We model this mixed-type space using a Gaussian Process kernel, designed to allow sharing of information across multiple categorical variables, each with multiple possible values; this allows CoCaBO to leverage all available data efficiently. We extend our method to the batch setting and propose an efficient selection procedure that dynamically balances exploration and exploitation whilst encouraging batch diversity. We demonstrate empirically that our method outperforms existing approaches on both synthetic and real-world optimisation tasks with continuous and categorical inputs.

Authors

S. Roberts

2 papers

Binxin Ru

2 papers

A. Alvi

1 papers

TL;DR

Abstract

Authors

References42 items

Bayesian Optimization for Categorical and Category-Specific Continuous Inputs

Knowing The What But Not The Where in Bayesian Optimization

NAS-Bench-101: Towards Reproducible Neural Architecture Search

Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation

A Tutorial on Bayesian Optimization

Fast Information-theoretic Bayesian Optimisation

A Tutorial on Thompson Sampling

Dealing with Categorical and Integer-valued Variables in Bayesian Optimization with Gaussian Processes

The non-stationary stochastic multi-armed bandit problem

The Parallel Knowledge Gradient Method for Batch Bayesian Optimization

XGBoost: A Scalable Tree Boosting System

Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Batch Bayesian Optimization via Local Penalization

High Dimensional Bayesian Optimisation and Bandits via Additive Models

Predictive Entropy Search for Bayesian Optimization with Unknown Constraints

Parallel Gaussian Process Optimization with Upper Confidence Bound and Pure Exploration

Structure Discovery in Nonparametric Regression through Compositional Kernel Search

Practical Bayesian Optimization of Machine Learning Algorithms

Algorithms for Hyper-Parameter Optimization

Entropy Search for Information-Efficient Global Optimization

Revisiting k-means: New Algorithms via Bayesian Nonparametrics

Scikit-learn: Machine Learning in Python

Sequential Model-Based Optimization for General Algorithm Configuration

A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

Algorithms for Adversarial Bandit Problems with Multiple Plays

Finite-time Analysis of the Multiarmed Bandit Problem

ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES

Overview and Comparison of Gaussian Process-Based Surrogate Models for Mixed Continuous and Discrete Variables: Application on Aerospace Design Problems

MANAS}: Multi-agent neural architecture

Mixed-Variable Bayesian Optimization

Algorithmic Assurance: An Active Approach to Algorithmic Testing using Bayesian Optimisation

High Dimensional Bayesian Optimization with Elastic Gaussian Process

UCI machine learning repository, 2017

Taking the Human Out of the Loop: A Review of Bayesian Optimization

Surrogate Models for Mixed Discrete-Continuous Variables

Kriging is well-suited to parallelize optimization

MNIST handwritten digit database

Gaussian process optimization in the bandit setting: No regret and experimental design

The Nonstochastic Multiarmed Bandit Problem

Random Forests

Unlike the original objective function f (x), the acquisition function α t (x) is cheap to compute and can be optimised using standard techniques

Field of Study

Venue Information

Name

Type

URL

Alternate Names