Selecting critical features for data classification based on machine learning methods (2020-07-23T00:00:00.000000Z)

TL;DR

This paper adopts Random Forest to select the important feature in classification and compares the result of the dataset with and without essential features selection by RF methods varImp(), Boruta, and Recursive Feature Elimination to get the best percentage accuracy and kappa.

Abstract

Feature selection becomes prominent, especially in the data sets with many variables and features. It will eliminate unimportant variables and improve the accuracy as well as the performance of classification. Random Forest has emerged as a quite useful algorithm that can handle the feature selection issue even with a higher number of variables. In this paper, we use three popular datasets with a higher number of variables (Bank Marketing, Car Evaluation Database, Human Activity Recognition Using Smartphones) to conduct the experiment. There are four main reasons why feature selection is essential. First, to simplify the model by reducing the number of parameters, next to decrease the training time, to reduce overfilling by enhancing generalization, and to avoid the curse of dimensionality. Besides, we evaluate and compare each accuracy and performance of the classification model, such as Random Forest (RF), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), and Linear Discriminant Analysis (LDA). The highest accuracy of the model is the best classifier. Practically, this paper adopts Random Forest to select the important feature in classification. Our experiments clearly show the comparative study of the RF algorithm from different perspectives. Furthermore, we compare the result of the dataset with and without essential features selection by RF methods varImp(), Boruta, and Recursive Feature Elimination (RFE) to get the best percentage accuracy and kappa. Experimental results demonstrate that Random Forest achieves a better performance in all experiment groups.

Authors

R. Chen

1 papers

Christine Dewi

1 papers

Su-Wen Huang

1 papers

TL;DR

Abstract

Authors

References134 items

Confusion Matrix

A New Feature Selection Method for Text Classification Based on Independent Feature Space Search

Leveraging feature selection to detect potential tax fraudsters

Feature importance of the aortic anatomy on endovascular aneurysm repair (EVAR) using Boruta and Bayesian MCMC

Optimization of risk control in financial markets based on particle swarm optimization algorithm

Use of a K-nearest neighbors model to predict the development of type 2 diabetes within 2 years in an obese, hypertensive population

Class-imbalanced dynamic financial distress prediction based on Adaboost-SVM ensemble combined with SMOTE and time weighting

A global database for metacommunity ecology, integrating species, traits, environment and space

Enhanced Binary Moth Flame Optimization as a Feature Selection Algorithm to Predict Software Fault Prediction

The GenTree Dendroecological Collection, tree-ring and wood density data from seven tree species across Europe

Optimal feature selection through a cluster-based DT learning (CDTL) in heart disease prediction

A combined strategy of feature selection and machine learning to identify predictors of prediabetes

Flash-flood hazard assessment using ensembles and Bayesian-based machine learning models: Application of the simulated annealing feature selection method.

Unsupervised feature selection algorithm for multiclass cancer classification of gene expression RNA-Seq data.

Prediction of Status Particulate Matter 2.5 Using State Markov Chain Stochastic Process and HYBRID VAR-NN-PSO

An application of machine learning with feature selection to improve diagnosis and classification of neurodegenerative disorders

Human Activity Recognition Based on Evolution of Features Selection and Random Forest

Multi-sink distributed power control algorithm for Cyber-physical-systems in coal mine tunnels

Improved grey wolf optimization-based feature subset selection with fuzzy neural classifier for financial crisis prediction

A regional adaptive variational PDE model for computed tomography image reconstruction

Neurocomputing fundamental climate analysis

RSLIME: An Efficient Feature Importance Analysis Approach for Industrial Recommendation Systems

Flash flood susceptibility modeling using an optimized fuzzy rule based feature selection technique and tree based ensemble methods.

Fast Adaptive K-Means Subspace Clustering for High-Dimensional Data

Machine Learning Approach for Bottom 40 Percent Households (B40) Poverty Classification

Feature Extraction and Selection of Sentinel-1 Dual-Pol Data for Global-Scale Local Climate Zone Classification

Fractal Intelligent Privacy Protection in Online Social Network Using Attribute-Based Encryption Schemes

VAR and GSTAR-Based Feature Selection in Support Vector Regression for Multivariate Spatio-Temporal Forecasting

Research on classification method of high-dimensional class-imbalanced datasets based on SVM

Hybrid support vector regression in electric load during national holiday season

Understanding the Data Science behind Business Analytics

Gradient-driven parking navigation using a continuous information potential field based on wireless sensor network

Shahabad Stone Misuse Scheme in India

A novel approach for optimizing climate features and network parameters in rainfall forecasting

Gender recognition using PCA and LDA with improve preprocessing and classification technique

Research on Classification Method of High-Dimensional Class-Imbalanced Data Sets Based on SVM

Applications of Hadoop Ecosystems Tools

CDMA-based anti-collision algorithm for EPC global C1 Gen2 systems

Linear discriminant analysis: A detailed tutorial

NoSQLDatabase for Storage and Retrieval of Data in Cloud

Optimization of Distributions Differences for Classification

Application of Random Forest Algorithm on Feature Subset Selection and Classification and Regression

A Comparative Study of Random Forest & K – Nearest Neighbors on HAR dataset Using Caret

Learning Change from Synthetic Aperture Radar Images: Performance Evaluation of a Support Vector Machine to Detect Earthquake and Tsunami-Induced Changes

Principal component analysis: a review and recent developments

An efficient framework for heart disease classification using feature extraction and feature selection technique in data mining

Machine learning predictive models for mineral prospectivity: an evaluation of neural networks, random forest, regression trees and support vector machines

So Many Variables: Joint Modeling in Community Ecology.

Important variable assessment and electricity price forecasting based on regression tree models: classification and regression trees, Bagging and Random Forests

A general procedure to generate models for urban environmental-noise pollution using feature selection and machine learning methods.

BOOST: A supervised approach for multiple sclerosis lesion segmentation

Inactivation of Human Salivary Glutathione Transferase P1-1 by Hypothiocyanite: A Post-Translational Control System in Search of a Role

GI/Geom/1 queue based on communication model for mesh networks

Application of Random Forests Methods to Diabetic Retinopathy Classification Analyses

Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik

ROC curves for regression

Feature selection for medical diagnosis : Evaluation for cardiovascular diseases

Earthquake Damage Detection in Urban Areas Using Curvilinear Features

A Bijection between Lattice-Valued Filters and Lattice-Valued Congruences in Residuated Lattices

Holes Detection in Anisotropic Sensornets: Topological Methods

Stochastic Modeling of Power Demand Due to EVs Using Copula

Classification of Local Climate Zones Based on Multiple Earth Observation Data

Digital modulation classification based on characteristic features and GentleBoost algorithm

Receiver operating characteristic curves

Hybrid feature selection by combining filters and wrappers

A review of robust clustering methods

CO, NO2 and NOx urban pollution monitoring with on-field calibrated electronic nose by automatic bayesian regularization

Variable Importance Assessment in Regression: Linear Regression versus Random Forest

K-nearest neighbor

Methods to quantify variable importance: implications for the analysis of noisy ecological data.

Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers

High Accuracy Human Activity Monitoring Using Neural Network

FAST: a roc-based feature selection metric for small samples and imbalanced data classification problems

Learning on the border: active learning in imbalanced data classification

An Empirical Study of Learning from Imbalanced Data Using Random Forest

A New Feature Transformation Method Based on Rotation for Speaker Identification