Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples (2019-03-07T00:00:00.000000Z)

TL;DR

This work proposes Meta-Dataset: a new benchmark for training and evaluating models that is large-scale, consists of diverse datasets, and presents more realistic tasks, and proposes a new set of baselines for quantifying the benefit of meta-learning in Meta- Dataset.

Abstract

Few-shot classification refers to learning a classifier for new classes given only a few examples. While a plethora of models have emerged to tackle it, we find the procedure and datasets that are used to assess their progress lacking. To address this limitation, we propose Meta-Dataset: a new benchmark for training and evaluating models that is large-scale, consists of diverse datasets, and presents more realistic tasks. We experiment with popular baselines and meta-learners on Meta-Dataset, along with a competitive method that we propose. We analyze performance as a function of various characteristics of test tasks and examine the models' ability to leverage diverse training sources for improving their generalization. We also propose a new set of baselines for quantifying the benefit of meta-learning in Meta-Dataset. Our extensive experimentation has uncovered important research challenges and we hope to inspire work in these directions.

Authors

H. Larochelle

18 papers

Vincent Dumoulin

7 papers

Pierre-Antoine Manzagol

2 papers

TL;DR

Abstract

Authors

References40 items

A Baseline for Few-Shot Image Classification

A Closer Look at Few-shot Classification

How to train your MAML

Meta-Learning with Latent Embedding Optimization

Bayesian Model-Agnostic Meta-Learning

TADAM: Task dependent adaptive metric for improved few-shot learning

Meta-learning with differentiable closed-form solvers

Dynamic Few-Shot Visual Learning Without Forgetting

Meta-Learning for Semi-Supervised Few-Shot Classification

Low-Shot Learning from Imaginary Data

Low-Shot Learning with Imprinted Weights

Learning to Compare: Relation Network for Few-Shot Learning

Few-Shot Learning with Graph Neural Networks

A Simple Neural Attentive Meta-Learner

A Neural Representation of Sketch Drawings

Prototypical Networks for Few-shot Learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Meta Networks

Optimization as a Model for Few-Shot Learning

Learning to Learn: Model Regression Networks for Easy Small Sample Learning

Meta-Learning with Memory-Augmented Neural Networks

Matching Networks for One Shot Learning

Low-Shot Visual Recognition by Shrinking and Hallucinating Features

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

Human-level concept learning through probabilistic program induction

ImageNet Large Scale Visual Recognition Challenge

Microsoft COCO: Common Objects in Context

Describing Textures in the Wild

Detection of traffic signs in real-world images: The German traffic sign detection benchmark

Fine-Grained Visual Classification of Aircraft

The German Traffic Sign Recognition Benchmark: A multi-class classification competition

The Caltech-UCSD Birds-200-2011 Dataset

Automated Flower Classification over a Large Number of Classes

FGVCx fungi classification challenge 2018

Learning to Model the Tail

Jongmin Kim, and Nick Fox-Gieg. The Quick, Draw! -A.I. experiment. quickdraw.withgoogle.com

The Quick, Draw! – A.I. experiment. quickdraw

For the baselines, we tuned whether the cosine classifier of Baseline++ will be used, as opposed to a standard forward pass through a linear classification layer

Caltech-ucsd birds

Learning Multiple Layers of Features from Tiny Images

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names