SHIFT15M: Fashion-specific dataset for set-to-set matching with several distribution shifts (2021-08-30T00:00:00.000000Z)

TL;DR

This paper proposes SHIFT15M, a dataset that can be used to properly evaluate set-to-set matching models in situations where the distribution of data changes between training and testing.

Abstract

Set-to-set matching is the problem of matching two different sets of items based on some criteria. Especially when each item in the set is high-dimensional, such as an image, set-to-set matching is treated as one of the applied problems to be solved by utilizing neural networks. Most machine learning-based set-to-set matching generally assumes that the training and test data follow the same distribution. However, such assumptions are often violated in real-world machine learning problems. In this paper, we propose SHIFT15M, a dataset that can be used to properly evaluate set-to-set matching models in situations where the distribution of data changes between training and testing. Some benchmark experiments show that the performance of naive methods drops due to the effects of the distribution shift. In addition, we provide software to handle SHIFT15M dataset in a very simple way: https://github.com/st-tech/zozo-shift15m. The URL for the software will appear after this manuscript is published.

Authors

Masanari Kimura

1 papers

Takuma Nakamura

2 papers

Yuki Saito

2 papers

TL;DR

Abstract

Authors

References169 items

Generalization Bounds for Set-to-Set Matching with Negative Sampling

Information Geometrically Generalized Covariate Shift Adaptation

Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift

Out-of-Distribution Generalization on Graphs: A Survey

Generalized Out-of-Distribution Detection: A Survey

Towards Out-Of-Distribution Generalization: A Survey

Accuracy on the Line: on the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization

Universal Approximation of Functions on Sets

Towards a Theoretical Framework of Out-of-Distribution Generalization

Toward Causal Representation Learning

Deep Stable Learning for Out-Of-Distribution Generalization

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Domain Generalization with MixStyle

SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data

Domain Generalization: A Survey

Out of Distribution Generalization in Machine Learning

On Calibration and Out-of-domain Generalization

CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation

Transformers in Vision: A Survey

WILDS: A Benchmark of in-the-Wild Distribution Shifts

Exchangeable Deep Neural Networks for Set-to-Set Matching and Learning

Batch Prioritization of Data Labeling Tasks for Training Classifiers

Efficient Transformers: A Survey

Robust and Generalizable Visual Representation Learning via Random Convolutions

Stable Learning via Differentiated Variable Decorrelation

Curriculum Manager for Source Selection in Multi-Source Domain Adaptation

Measuring Robustness to Natural Distribution Shifts in Image Classification

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

Model Adaptation: Unsupervised Domain Adaptation Without Source Data

Using publicly available satellite imagery and deep learning to understand economic well-being in Africa

Open Graph Benchmark: Datasets for Machine Learning on Graphs

Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset

Global Wheat Head Detection (GWHD) Dataset: A Large and Diverse Dataset of High-Resolution RGB-Labelled Images to Develop and Benchmark Wheat Head Detection Methods

The iWildCam 2020 Competition Dataset

CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models

Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition From a Domain Adaptation Perspective

A Simple Way to Make Neural Networks Robust Against Diverse Image Corruptions

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Learning under Concept Drift: A Review

Multi-source Distilling Domain Adaptation

A Comprehensive Survey on Transfer Learning

Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects

Multi-source Domain Adaptation for Semantic Segmentation

Advances in Domain Adaptation Theory

Distributionally Robust Optimization: A Review

Natural Adversarial Examples

Deep Set Prediction Networks

Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

MNIST-C: A Robustness Benchmark for Computer Vision

The Fashion IQ Dataset: Retrieving Images by Combining Side Information and Relative Natural Language Feedback

Semi-Supervised Domain Adaptation via Minimax Entropy

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

On Deep Set Learning and the Choice of Aggregations

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

On Evaluating Adversarial Robustness

Context-Aware Visual Compatibility Prediction

From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge

On the Limitations of Representing Functions on Sets

Contrastive Adaptation Network for Unsupervised Domain Adaptation

Moment Matching for Multi-Source Domain Adaptation

Convolutional Set Matching for Graph Similarity

Generalisation in humans and deep neural networks

Hard-Aware Point-to-Set Deep Metric for Person Re-identification

Domain Generalization with Adversarial Feature Learning

Learning Type-Aware Embeddings for Fashion Compatibility

Datasheets for datasets

Continual Lifelong Learning with Neural Networks: A Review

Disentangling by Factorising

Deep Visual Domain Adaptation: A Survey

Anchor regression: Heterogeneous data meet causality

Generating Adversarial Examples with Adversarial Networks

Exploring the Landscape of Spatial Robustness

Functional Map of the World

Three-layer concept drifting detection in text data streams

Deeper, Broader and Artier Domain Generalization

Unified Deep Supervised Domain Adaptation and Generalization