XIMAGENET-12: An Explainable AI Benchmark Dataset for Model Robustness Evaluation (2023-10-12T00:00:00.000000Z)

TL;DR

This work proposes an explainable visual dataset, XIMAGENET-12, to evaluate the robustness of visual models, and develops a quantitative criterion for robustness assessment, allowing for a nuanced understanding of how visual models perform under varying conditions.

Abstract

Despite the promising performance of existing visual models on public benchmarks, the critical assessment of their robustness for real-world applications remains an ongoing challenge. To bridge this gap, we propose an explainable visual dataset, XIMAGENET-12, to evaluate the robustness of visual models. XIMAGENET-12 consists of over 200K images with 15,410 manual semantic annotations. Specifically, we deliberately selected 12 categories from ImageNet, representing objects commonly encountered in practical life. To simulate real-world situations, we incorporated six diverse scenarios, such as overexposure, blurring, and color changes, etc. We further develop a quantitative criterion for robustness assessment, allowing for a nuanced understanding of how visual models perform under varying conditions, notably in relation to the background. We make the XIMAGENET-12 dataset and its corresponding code openly accessible at \url{https://sites.google.com/view/ximagenet-12/home}. We expect the introduction of the XIMAGENET-12 dataset will empower researchers to thoroughly evaluate the robustness of their visual models under challenging conditions.

Authors

Qiang Li

1 papers

Dan Zhang

1 papers

Shengzhao Lei

1 papers

TL;DR

Abstract

Authors

References61 items

Unleashing the Power of Self-Supervised Image Denoising: A Comprehensive Review

Network Pruning and Fine-tuning for Few-shot Industrial Image Anomaly Detection

Self-supervised Learning with Temporary Exact Solutions: Linear Projection

SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

Exploiting Interactivity and Heterogeneity for Sleep Stage Classification Via Heterogeneous Graph Neural Network

High Dynamic Range Imaging with Context-aware Transformer

MM-BSN: Self-Supervised Image Denoising for Real-World with Multi-Mask based on Blind-Spot Network

SSGD: A Smartphone Screen Glass Dataset for Defect Detection

Continual learning on deployment pipelines for Machine Learning Systems

Automatic Labeling to Generate Training Data for Online LiDAR-Based Moving Object Segmentation

High-Resolution Image Synthesis with Latent Diffusion Models

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

GPT-GNN: Generative Pre-Training of Graph Neural Networks

Noise or Signal: The Role of Image Backgrounds in Object Recognition

Interactive Object Segmentation With Inside-Outside Guidance

Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization

Self-Training With Noisy Student Improves ImageNet Classification

MMDetection: Open MMLab Detection Toolbox and Benchmark

MNIST-C: A Robustness Benchmark for Computer Vision

MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Searching for MobileNetV3

Unified Perceptual Parsing for Scene Understanding

Learning to Separate Object Sounds by Watching Unlabeled Video

The Intuitive Appeal of Explainable Machines

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Deep Learning with Python

Automatic differentiation in PyTorch

Rethinking Atrous Convolution for Semantic Image Segmentation

Feature Pyramid Networks for Object Detection

Pyramid Scene Parsing Network

Object Recognition with and without Objects

Densely Connected Convolutional Networks

R-FCN: Object Detection via Region-based Fully Convolutional Networks

Understanding how image quality affects deep neural networks

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

“Why Should I Trust You?”: Explaining the Predictions of Any Classifier

Deep Residual Learning for Image Recognition

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

VITRO - Model based vision testing for robustness

ImageNet classification with deep convolutional neural networks

Undoing the Damage of Dataset Bias

Unbiased look at dataset bias

ImageNet: A large-scale hierarchical image database

Why should I?

Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study

Contextual Priming for Object Detection

On the Theory of Correlation

Weakly Supervised Medical Image Segmentation Through Dense Combinations of Dense Pseudo-Labels

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

All you need is cell attention: A cell annotation tool for single-cell morphology data

MMSegmentation Contributors

Not us-ing the car to see the sidewalk - quantifying and controlling the effects of context in classification and segmentation

A method for stochastic optimization

Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks

Learning Multiple Layers of Features from Tiny Images

Statistical Methods for Research Workers

PlaygroundAI

: Extend large vision-language model for task oriented object

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names