Dataset2Vec: learning dataset meta-features (2019-05-27T00:00:00.000000Z)

TL;DR

In an experiment on a large-scale hyperparameter optimization task for 120 UCI datasets with varying schemas as a meta-learning task, it is shown that the meta-features of Dataset2Vec outperform the expert engineered meta- Features and thus demonstrate the usefulness of learned meta- features for datasets with varies schemas for the first time.

Abstract

Meta-learning, or learning to learn, is a machine learning approach that utilizes prior learning experiences to expedite the learning process on unseen tasks. As a data-driven approach, meta-learning requires meta-features that represent the primary learning tasks or datasets, and are estimated traditonally as engineered dataset statistics that require expert domain knowledge tailored for every meta-task. In this paper, first, we propose a meta-feature extractor called Dataset2Vec that combines the versatility of engineered dataset meta-features with the expressivity of meta-features learned by deep neural networks. Primary learning tasks or datasets are represented as hierarchical sets, i.e., as a set of sets, esp. as a set of predictor/target pairs, and then a DeepSet architecture is employed to regress meta-features on them. Second, we propose a novel auxiliary meta-learning task with abundant data called dataset similarity learning that aims to predict if two batches stem from the same dataset or different ones. In an experiment on a large-scale hyperparameter optimization task for 120 UCI datasets with varying schemas as a meta-learning task, we show that the meta-features of Dataset2Vec outperform the expert engineered meta-features and thus demonstrate the usefulness of learned meta-features for datasets with varying schemas for the first time.

Authors

L. Schmidt-Thieme

3 papers

H. Jomaa

2 papers

Josif Grabocka

3 papers

TL;DR

Abstract

Authors

References76 items

Constrained Efficient Global Optimization of Expensive Black-box Functions

Optimal Transport in Reproducing Kernel Hilbert Spaces: Theory and Applications

Chameleon: Learning Model Initializations Across Tasks With Different Schemas

Task2Vec: Task Embedding for Meta-Learning

Meta-Learning: A Survey

Towards Reproducible Empirical Research in Meta-Learning

Image Captioning with Affective Guiding and Selective Attention

Meta-Learning with Latent Embedding Optimization

BOHB: Robust and Efficient Hyperparameter Optimization at Scale

Bayesian Model-Agnostic Meta-Learning

Probabilistic Model-Agnostic Meta-Learning

Domain Generalization via Conditional Invariant Representations

The Variational Homoencoder: Learning to learn high capacity generative models from few examples

Scalable Meta-Learning for Bayesian Optimization

Class-balanced siamese neural networks

Learning to Generalize: Meta-Learning for Domain Generalization

Warmstarting of Model-based Algorithm Configuration

Self-Normalizing Neural Networks

Prototypical Networks for Few-shot Learning

Deep Sets

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

A Gaussian Process Regression Model for Distribution Inputs

Multiple instance learning: A survey of problem characteristics and applications

Bayesian Optimization with Robust Bayesian Neural Networks

A Discriminatively Learned CNN Embedding for Person Reidentification

Two-Stage Transfer Surrogate Model for Automatic Hyperparameter Optimization

Towards a Neural Statistician

Wide Residual Networks

Deep Metric Learning via Lifted Structured Feature Embedding

Sequential Model-Free Hyperparameter Tuning

Datasets meta-feature description for recommending feature selection algorithm

Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Initializing Bayesian Hyperparameter Optimization via Meta-Learning

Adam: A Method for Stochastic Optimization

OpenML: networked science in machine learning

Efficient Transfer Learning Method for Automatic Hyperparameter Tuning

Automatic classifier selection for non-experts

Auto-Encoding Variational Bayes

Collaborative hyperparameter tuning

Distribution to Distribution Regression

Distribution-Free Distribution Regression

Domain Generalization via Invariant Feature Representation

Practical Bayesian Optimization of Machine Learning Algorithms

Random Search for Hyper-Parameter Optimization

Algorithms for Hyper-Parameter Optimization

Scikit-learn: Machine Learning in Python

Sequential Model-Based Optimization for General Algorithm Configuration

Cross-disciplinary perspectives on meta-learning for algorithm selection

Information-Theoretic Measures for Meta-learning

Statistical Comparisons of Classifiers over Multiple Data Sets

Predicting relative performance of classifiers from samples

Meta-data: Characterization of Input Features for Meta-learning

Modern Multidimensional Scaling: Theory and Applications

Kolmogorov's Theorem Is Relevant

Scalable Hyperparameter Transfer Learning

UCI machine learning repository, 2017

Scalable Gaussian process-based transfer surrogates for hyperparameter optimization

StorkeyAJ (2017a) Towards a neural statistician https://openreview.net/forum?id=HJDBUF5le

This approach relies on Bayesian Neural Networks to model the surrogate

Zagoruyko S, Komodakis N (2016

Siamese Neural Networks for One-Shot Image Recognition

Artificial intelligence and natural language and information extraction, social media and web search FRUCT conference (AINL-ISMW FRUCT

Dropout: a simple way to prevent neural networks from overfitting

Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude

A tree-based approach that constructs a density estimate over good and bad instantiations of each hyperpa-rameter

Instead of a Gaussian process, the surrogate is modeled as a random-forest (Breiman 2001) that yields uncertainty estimates. 4

Large-Scale Machine Learning with Stochastic Gradient Descent

The surrogate is modeled by a Gaussian process with a Matérn 3/2 kernel

Random Forests

This Paper Is Included in the Proceedings of the 12th Usenix Symposium on Operating Systems Design and Implementation (osdi '16). Tensorflow: a System for Large-scale Machine Learning Tensorflow: a System for Large-scale Machine Learning

Dataset2Vec: learning dataset meta-features

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

We design a novel meta-task called dataset similarity learning that has abundant data and is therefore well-suited as an auxiliary meta-task to train the meta-feature

i) the meta-feature extractor ˆ φ : D → R K , that extracts from a dataset a ﬁxed number K of meta-features

We design and investigate a meta-feature extractor, Dataset2Vec, based on a representation of datasets as hierarchical sets of predictor/target pairs