A Survey on ensemble learning under the era of deep learning (2021-01-21T00:00:00.000000Z)

TL;DR

This article presents discussions focusing on data analyses of published works, methodologies, recent advances and unattainability of traditional ensemble learning and ensemble deep learning to realize the intrinsic problems and technical challenges faced by future developments of ensemble learning under the era of deep learning.

Abstract

Due to the dominant position of deep learning (mostly deep neural networks) in various artificial intelligence applications, recently, ensemble learning based on deep neural networks (ensemble deep learning) has shown significant performances in improving the generalization of learning system. However, since modern deep neural networks usually have millions to billions of parameters, the time and space overheads for training multiple base deep learners and testing with the ensemble deep learner are far greater than that of traditional ensemble learning. Though several algorithms of fast ensemble deep learning have been proposed to promote the deployment of ensemble deep learning in some applications, further advances still need to be made for many applications in specific fields, where the developing time and computing resources are usually restricted or the data to be processed is of large dimensionality. An urgent problem needs to be solved is how to take the significant advantages of ensemble deep learning while reduce the required expenses so that many more applications in specific fields can benefit from it. For the alleviation of this problem, it is essential to know about how ensemble learning has developed under the era of deep learning. Thus, in this article, we present discussions focusing on data analyses of published works, methodologies, recent advances and unattainability of traditional ensemble learning and ensemble deep learning. We hope this article will be helpful to realize the intrinsic problems and technical challenges faced by future developments of ensemble learning under the era of deep learning.

Authors

Yongquan Yang

1 papers

Haijun Lv

1 papers

Ning Chen

1 papers

TL;DR

Abstract

Authors

References200 items

Design of deep ensemble classifier with fuzzy decision method for biomedical image classification

An effective ensemble deep learning framework for text classification

Design of Gradient Boosting Ensemble Classifier with Variation of Learning Rate for Automated Cardiac Data Classification

Epileptic Seizure Detection using Deep Ensemble Network with Empirical Wavelet Transform

EfficientNetV2: Smaller Models and Faster Training

GAN Ensemble for Anomaly Detection

Handling noisy labels via one-step abductive multi-target learning and its application to helicobacter pylori segmentation

Handling Noisy Labels via One-Step Abductive Multi-Target Learning

Ensemble deep learning in bioinformatics

Detecting helicobacter pylori in whole slide images via weakly supervised multi-task learning

Ensemble Learning for Hyperspectral Image Classification Using Tangent Collaborative Representation

FTBME: feature transferring based multi-model ensemble

Short-Term Load Forecasting with Improved CEEMDAN and GWO-Based Multiple Kernel ELM

Flood Detection and Susceptibility Mapping Using Sentinel-1 Remote Sensing Data and a Machine Learning Approach: Hybrid Intelligence of Bagging Ensemble Based on K-Nearest Neighbor Classifier

Improved landslide assessment using support vector machine with bagging, boosting, and stacking ensemble machine learning framework in a mountainous watershed, Japan

Ensemble Learning

A Deep Learning Ensemble Approach for Diabetic Retinopathy Detection

Nonlinear Regression via Deep Negative Correlation Learning

A survey on ensemble learning

Survey on Deep Neural Networks in Speech and Vision Systems

Ensemble Methods

Detection of Ocular Artifacts Using Bagged Tree Ensemble Model

Bearing Fault Diagnosis Method Based on Deep Convolutional Neural Network and Random Forest Ensemble Learning

A Simple Baseline for Bayesian Uncertainty in Deep Learning

Ensemble deep learning-based fault diagnosis of rotor bearing systems

A survey of the recent architectures of deep convolutional neural networks

Automatic detection of lung cancer from biomedical data set using discrete AdaBoost optimized ensemble learning generalized neural networks

MEAL: Multi-Model Ensemble via Adversarial Learning

A novel deep learning-based multi-model ensemble method for the prediction of neuromuscular disorders

An empirical comparison on state-of-the-art multi-class imbalance learning algorithms and a new diversified ensemble learning scheme

M6APred-EL: A Sequence-Based Predictor for Identifying N6-methyladenosine Sites Using Ensemble Learning

Explorations on visual localization from active to passive

Ensemble learning: A survey

Wind speed forecasting using nonlinear-learning ensemble of deep learning time series prediction and extremal optimization

Soil moisture forecasting by a hybrid machine learning technique: ELM integrated with ensemble empirical mode decomposition

Ensemble deep kernel learning with application to quality prediction in industrial polymerization processes

Averaging Weights Leads to Wider Optima and Better Generalization

Collaborative strategy for visual object tracking

Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Prediction of Protein‐Protein Interactions

Distilling a Neural Network Into a Soft Decision Tree

Landslide spatial modeling: Introducing new ensembles of ANN, MaxEnt, and SVM machine learning techniques

An Ensemble Deep Learning Method for Vehicle Type Classification on Visual Traffic Surveillance Sensors

Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review

A deep learning ensemble approach for crude oil price forecasting

Learning Transferable Architectures for Scalable Image Recognition

An ensemble deep learning based approach for red lesion detection in fundus images

Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting

Short-term electric load forecasting based on singular spectrum analysis and support vector machine optimized by Cuckoo search algorithm

A survey of deep neural network architectures and their applications

Multi-agent Diverse Generative Adversarial Networks

Deep ensemble learning of sparse regression models for brain disease diagnosis

Snapshot Ensembles: Train 1, get M for free

A novel ensemble classifier of hyperspectral and LiDAR data using morphological features

Deep learning based ensemble approach for probabilistic wind power forecasting

Hybrid integration of Multilayer Perceptron Neural Networks and machine learning ensembles for landslide susceptibility assessment at Himalayan area (India) using GIS

Aggregated Residual Transformations for Deep Neural Networks

Generative Multi-Adversarial Networks

Deep learning ensembles for melanoma recognition in dermoscopy images

Xception: Deep Learning with Depthwise Separable Convolutions

Identifying N6-methyladenosine sites in the Arabidopsis thaliana transcriptome

Densely Connected Convolutional Networks

iDHS-EL: identifying DNase I hypersensitive sites by fusing three different modes of pseudo nucleotide composition into an ensemble learning framework

SGDR: Stochastic Gradient Descent with Warm Restarts

The distance function effect on k-nearest neighbor classification for medical datasets

Fault analysis of High Speed Train with DBN hierarchical ensemble

Tangent Distance-Based Collaborative Representation for Hyperspectral Image Classification

Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC)

Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data

Estimating and forecasting the real prices of crude oil: A data rich model using a dynamic model averaging (DMA) approach

Hyperspectral Image Classification by Fusing Collaborative and Sparse Representations

Red Lesion Detection Using Dynamic Shape Features for Diabetic Retinopathy Screening

pRNAm-PC: Predicting N(6)-methyladenosine sites in RNA sequences via physical-chemical properties.

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Flood susceptibility mapping using frequency ratio and weights-of-evidence models in the Golastan Province, Iran