High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning (2021-06-07T00:00:00.000000Z)

TL;DR

A method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces is introduced, using label guidance from the blackbox function to structure the VAE latent space, facilitating the Gaussian process fit and yielding improved BO performance.

Abstract

We introduce a method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces. By adapting ideas from deep metric learning, we use label guidance from the blackbox function to structure the VAE latent space, facilitating the Gaussian process fit and yielding improved BO performance. Importantly for BO problem settings, our method operates in semi-supervised regimes where only few labelled data points are available. We run experiments on three real-world tasks, achieving state-of-the-art results on the penalised logP molecule generation benchmark using just 3% of the labelled data required by previous approaches. As a theoretical contribution, we present a proof of vanishing regret for VAE BO.

Authors

Jan Peters

5 papers

Lin Yang

2 papers

Ryan-Rhys Griffiths

4 papers

TL;DR

Abstract

Authors

References127 items

Constrained Efficient Global Optimization of Expensive Black-box Functions

Learning a Discriminant Latent Space with Neural Discriminant Analysis

Improving black-box optimization in VAE latent space using decoder uncertainty

Pitch-Timbre Disentanglement Of Musical Instrument Sounds Based On Vae-Based Metric Learning

Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020

Bayesian reaction optimization as a tool for chemical synthesis

Good practices for Bayesian Optimization of high dimensional structured spaces

CLEAR: Contrastive Learning for Sentence Representation

Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?

Mercer Features for Efficient Combinatorial Bayesian Optimization

Multi-task Bayesian Optimization of Chemical Reactions

Embedding of Molecular Structure Using Molecular Hypergraph Variational Autoencoder with Metric Learning

Generative model‐enhanced human motion prediction

Gaussian Process Molecule Property Prediction with FlowMO

BOSS: Bayesian Optimization over String Spaces

Mapping Materials and Molecules.

The Photoswitch Dataset: A Molecular Machine Learning Benchmark for the Advancement of Synthetic Chemistry

Learning Invariant Representations for Reinforcement Learning without Reconstruction

Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining

Neural Architecture Search using Bayesian Optimisation with Weisfeiler-Lehman Kernel

Supervised Contrastive Learning

Proxy Anchor Loss for Deep Metric Learning

Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization

The Group Loss for Deep Metric Learning

Constrained Bayesian optimization for automatic chemical design using variational autoencoders

Achieving robustness to aleatoric uncertainty with heteroscedastic Bayesian optimisation

Scalable Global Optimization via Local Bayesian Optimization

Ordinal Triplet Loss: Investigating Sleepiness Detection from Speech

Deep Metric Learning: A Survey

Learning Trajectory Dependencies for Human Motion Prediction

ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations

Bayesian Optimization in Variational Latent Spaces with Dynamic Compression

Bayesian Optimization on Large Graphs via a Graph Convolutional Generative Model: Application in Cardiac Model Personalization

Self-driving laboratory for accelerated discovery of thin-film materials

All SMILES Variational Autoencoder

Adaptive Sensor Placement for Continuous Spaces

BayesNAS: A Bayesian Approach for Neural Architecture Search

Deep Metric Learning Beyond Binary Supervision

Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly

High-dimensional Bayesian optimization using low-dimensional feature spaces

Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey

Ordinal Deep Learning for Facial Age Estimation

A tutorial on distance metric learning: Mathematical foundations, algorithms, experimental analysis, prospects and challenges

Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models

GuacaMol: Benchmarking Models for De Novo Molecular Design

Dataset Bias in the Natural Sciences: A Case Study in Chemical Reaction Prediction and Synthesis Design

Optimizing Photonic Nanostructures via Multi-fidelity Gaussian Processes

Efficient search of compositional space for hybrid organic–inorganic perovskites via Bayesian optimization

Deep Metric Learning with Hierarchical Triplet Loss

Phoenics: A Bayesian Optimizer for Chemistry

Revised Contrastive Loss for Robust Age Estimation from Face

A Tutorial on Bayesian Optimization

BOHB: Robust and Efficient Hyperparameter Optimization at Scale

Structured Variationally Auto-encoded Optimization

BOCK : Bayesian Optimization with Cylindrical Kernels

Deep Discriminative Latent Space for Clustering

Nonlinear Metric Learning through Geodesic Interpolation within Lie Groups

Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance

High Dimensional Bayesian Optimization using Dropout

Junction Tree Variational Autoencoder for Molecular Graph Generation

TVAE: Triplet-Based Variational Autoencoder using Metric Learning

Neural Architecture Search with Bayesian Optimisation and Optimal Transport

Neural networks for topology optimization

Deep Metric Learning with Angular Loss

On the choice of the low-dimensional domain for global optimization via random embeddings

Discovering and Exploiting Additive Structure for Bayesian Optimization

Supervised distance metric learning through maximization of the Jeffrey divergence

Grammar Variational Autoencoder

Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules

Deep Kernel Learning

ZINC 15 – Ligand Discovery for Everyone

Bayesian optimization for learning gaits under uncertainty

When crowds hold privileges: Bayesian unsupervised representation learning with oracle constraints

Nonlinear Metric Learning with Kernel Density Estimation

FaceNet: A unified embedding for face recognition and clustering

High Dimensional Bayesian Optimisation and Bandits via Additive Models