DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data

Published in

IEEE Transactions on Neural Networks and Learni...(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

Deep synthetic minority oversampling technique (SMOTE) is proposed, a novel oversamplings algorithm for deep learning models that leverages the properties of the successful SMOTE algorithm and is capable of generating high-quality, artificial images that can enhance minority classes and balance the training set.

Abstract

Despite over two decades of progress, imbalanced data is still considered a significant challenge for contemporary machine learning models. Modern advances in deep learning have further magnified the importance of the imbalanced data problem, especially when learning from images. Therefore, there is a need for an oversampling method that is specifically tailored to deep learning models, can work on raw images while preserving their properties, and is capable of generating high-quality, artificial images that can enhance minority classes and balance the training set. We propose Deep synthetic minority oversampling technique (SMOTE), a novel oversampling algorithm for deep learning models that leverages the properties of the successful SMOTE algorithm. It is simple, yet effective in its design. It consists of three major components: 1) an encoder/decoder framework; 2) SMOTE-based oversampling; and 3) a dedicated loss function that is enhanced with a penalty term. An important advantage of DeepSMOTE over generative adversarial network (GAN)-based oversampling is that DeepSMOTE does not require a discriminator, and it generates high-quality artificial images that are both information-rich and suitable for visual inspection. DeepSMOTE code is publicly available at https://github.com/dd1github/DeepSMOTE.

Authors

Damien Dablain

1 papers

B. Krawczyk

2 papers

N. Chawla

5 papers

References119 items

On oversampling imbalanced data with deep conditional generative models

Class-Incremental Experience Replay for Continual Learning under Concept Drift

Concept Drift Detection from Multi-Class Imbalanced Data Streams

How to design the fair experimental classifier evaluation

Preprocessed dynamic classifier ensemble selection for highly imbalanced drifted data streams

DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data

Published in

IEEE Transactions on Neural Networks and Learni...(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Damien Dablain

1 papers

B. Krawczyk

2 papers

N. Chawla

5 papers

References119 items

On oversampling imbalanced data with deep conditional generative models

Class-Incremental Experience Replay for Continual Learning under Concept Drift

Concept Drift Detection from Multi-Class Imbalanced Data Streams

How to design the fair experimental classifier evaluation

Preprocessed dynamic classifier ensemble selection for highly imbalanced drifted data streams

Multiset Feature Learning for Highly Imbalanced Data Classification

ReMix: Calibrated Resampling for Class Imbalance in Deep learning

Radial-Based Oversampling for Multiclass Imbalanced Data Classification

Learning Deep Landmarks for Imbalanced Classification

Deep Neural Architectures for Highly Imbalanced Data in Bioinformatics

LR-SMOTE - An improved unbalanced data set oversampling based on K-means and SVM

Combined Cleaning and Resampling Algorithm for Multi-Class Imbalanced Data with Label Noise

Long-Tailed Recognition Using Class-Balanced Experts

Long-tail Visual Relationship Recognition with a Visiolinguistic Hubless Loss

Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition From a Domain Adaptation Perspective

Equalization Loss for Long-Tailed Object Recognition

Neighbourhood-based undersampling approach for handling imbalanced and overlapped data

Improvement of Bagging performance for classification of imbalanced datasets using evolutionary multi-objective optimization

BBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition

Analyzing and Improving the Image Quality of StyleGAN

A new adaptive weighted imbalanced data classifier via improved support vector machines with high-dimension nature

Geometric SMOTE a geometrically enhanced drop-in replacement for SMOTE

Deep class-skewed learning for face recognition

Conditional Transferring Features: Scaling GANs to Thousands of Classes with 30% Less High-Quality Data for Training

LOGAN: Latent Optimisation for Generative Adversarial Networks

Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations

On evaluating the online local pool generation method for imbalance learning

Radial-Based Undersampling for Imbalanced Data Classification

Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification

Multiobjective Support Vector Machines: Handling Class Imbalance With Pareto Optimality

Radial-Based oversampling for noisy imbalanced data classification

Generative Adversarial Minority Oversampling

Survey on deep learning with class imbalance

Class-Balanced Loss Based on Effective Number of Samples

Learning in presence of class imbalance and class overlapping by using one-class SVM and undersampling technique

Deep Discriminative Features Learning and Sampling for Imbalanced Data Problem

Learning from Imbalanced Data Sets

Generative Adversarial Network in Medical Imaging: A Review

AMDO: An Over-Sampling Technique for Multi-Class Imbalanced Problems

Deep Imbalanced Learning for Face Recognition and Attribute Prediction

IMCStacking: Cost-sensitive stacking learning with feature inverse mapping for imbalanced problems

Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels

Imbalanced Deep Learning by Minority Class Incremental Rectification

A Cost-Sensitive Deep Belief Network for Imbalanced Classification

RSDNE: Exploring Relaxed Similarity and Dissimilarity from Completely-Imbalanced Labels for Network Embedding

A study on combining dynamic selection and data preprocessing for imbalance learning

BAGAN: Data Augmentation with Balancing GAN

Spectral Normalization for Generative Adversarial Networks

Synthesizing realistic neural population activity patterns using Generative Adversarial Networks

CCR: A combined cleaning and resampling algorithm for imbalanced data classification

Are GANs Created Equal? A Large-Scale Study

Wasserstein Auto-Encoders

Manifold-based synthetic oversampling with manifold conformance estimation

Controlling Imbalanced Error in Deep Learning with the Log Bilinear Loss

Clustering-based undersampling in class-imbalanced data

Boosting with Lexicographic Programming: Addressing Class Imbalance without Cost Tuning

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

Focal Loss for Dense Object Detection

Wasserstein Generative Adversarial Networks

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

On Unifying Deep Generative Models

Deep Over-sampling Framework for Classifying Imbalanced Data

Improved Training of Wasserstein GANs

Range Loss for Deep Face Recognition with Long-Tailed Training Data

Connecting Generative Adversarial Networks and Actor-Critic Methods

Training deep neural networks on imbalanced data sets

Tutorial on Variational Autoencoders

Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Improved Techniques for Training GANs

Learning from imbalanced data: open challenges and future directions

Cost-sensitive feature selection using random forest: Selecting low-cost subsets of informative features

Decision tree induction based on minority entropy for the class imbalance problem

Diversity techniques improve the performance of the best imbalance learning ensembles

Deep Residual Learning for Image Recognition

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Random Balance: Ensembles of variable priors classifiers for imbalanced data

Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data

Selecting target concept in one-class classification for handling class imbalance problem

Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

Neighbourhood sampling in bagging for imbalanced data

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Weighted one-class classification for different types of minority class examples in imbalanced data

Deep Learning Face Attributes in the Wild

A survey of multiple classifier systems as hybrid systems

Auto-Encoding Variational Bayes

Rectified Linear Units Improve Restricted Boltzmann Machines

Roughly balanced bagging for imbalanced data

A systematic analysis of performance measures for classification tasks

ADASYN: Adaptive synthetic sampling approach for imbalanced learning

Low-Dimensional Representation Learning from Imbalanced Data Streams

Research on Privacy Disclosure Detection Method in Social Networks Based on Multi-Dimensional Deep Learning

Oversampling methods combined clustering and data cleaning for imbalanced network data

SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary

Cost-sensitive and hybrid-attribute measure multi-decision tree over imbalanced data sets

Transfer Boosting With Synthetic Instances for Class Imbalanced Object Recognition

[ more than 100 contributions to conferences. He has coauthored the book Learning from Imbalanced Datasets

Entropy-based fuzzy support vector machine for imbalanced datasets

100

“Unpaired image-to-image translation using cycle-consistent adversarial networks,”

101

Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy

102

Cost-sensitive decision tree ensembles for effective imbalanced classification

103

Rectifier Nonlinearities Improve Neural Network Acoustic Models

104

Hellinger distance decision trees are robust and skew-insensitive

105

Reading Digits in Natural Images with Unsupervised Feature Learning

106

Learning Multiple Layers of Features from Tiny Images

107

Visualizing Data using t-SNE

108

SMOTE: Synthetic Minority Over-sampling Technique

109

Gradient-based learning applied to document recognition

110

learning, data streams, class imbalance, continual learning, and explainable artiﬁcial intelligence

111

High quality of generated images

112

RQ6: Can DeepSMOTE produce stable models under extreme class imbalance?

113

Superiority over pixel-based and GAN-based algorithms

114

member of the editorial board for Applied Soft Computing

115

RQ3: What is the impact of the test set distribution on DeepSMOTE performance?

116

He is an Assistant Professor with the Department of Computer Science, Virginia Commonwealth University, Richmond, VA, USA, where he heads the Machine Learning and Stream Mining Laboratory

117

RQ4: What is the visual quality of artiﬁcial images generated by DeepSMOTE?

118

RQ5: Is DeepSMOTE robust to increasing class imbalance ratios?

119

RQ2: Is DeepSMOTE capable of outperforming state-of-the-art GAN-based resampling algorithms designed to work with complex and imbalanced data representations?

Field of Study

Computer ScienceMedicine

Journal Information

Name

IEEE Transactions on Neural Networks and Learning Systems

Page

6390-6404

Volume

Venue Information

Name

IEEE Transactions on Neural Networks and Learning Systems

Type

URL

http://ieeexplore.ieee.org/servlet/opac?punumber=5962385

Alternate Names

IEEE Trans Neural Netw Learn Syst