Mitigating the Multicollinearity Problem and Its Machine Learning Approach: A Review (2022-04-12T00:00:00.000000Z)

TL;DR

This study details the chronological developments to mitigate the effects of multicollinearity and up-to-date recommendations to better mitigate multicoll inearity.

Abstract

Technologies have driven big data collection across many fields, such as genomics and business intelligence. This results in a significant increase in variables and data points (observations) collected and stored. Although this presents opportunities to better model the relationship between predictors and the response variables, this also causes serious problems during data analysis, one of which is the multicollinearity problem. The two main approaches used to mitigate multicollinearity are variable selection methods and modified estimator methods. However, variable selection methods may negate efforts to collect more data as new data may eventually be dropped from modeling, while recent studies suggest that optimization approaches via machine learning handle data with multicollinearity better than statistical estimators. Therefore, this study details the chronological developments to mitigate the effects of multicollinearity and up-to-date recommendations to better mitigate multicollinearity.

Authors

Jireh Yi-Le Chan

1 papers

Steven Mun Hong Leow

1 papers

Khean Thye Bea

1 papers

TL;DR

Abstract

Authors

References82 items

A Recursive Least-Squares with a Time-Varying Regularization Parameter

Improved high-dimensional regression models with matrix approximations applied to the comparative case studies with support vector machines

A High-Dimensional Counterpart for the Ridge Estimator in Multicollinear Situations

A nonlinear mixed–integer programming approach for variable selection in linear regression model

Outlier Detection Based Feature Selection Exploiting Bio-Inspired Optimization Algorithms

Stock Price Movement Prediction Based on a Deep Factorization Machine and the Attention Mechanism

Dealing with Multicollinearity Problem in Analysis of Side Friction Characteristics Under Urban Heterogeneous Traffic Conditions

An Efficient Traffic Classification Scheme Using Embedded Feature Selection and LightGBM

A New Ridge-Type Estimator for the Linear Regression Model: Simulations and Applications

Ensemble feature selection in medical datasets: Combining filter, wrapper, and embedded feature selection results

Generalized Cross-Validation for Simultaneous Optimization of Tuning Parameters in Ridge Regression

Residual Control Chart for Binary Response with Multicollinearity Covariates by Neural Network Model

Multicollinearity Effect in Regression Analysis: A Feed Forward Artificial Neural Network Approach

High‐dimensional variable screening under multicollinearity

CNNpred: CNN-based stock market prediction using a diverse set of variables

HATS: A Hierarchical Graph Attention Network for Stock Movement Prediction

Variable Selection

Variable selection under multicollinearity using modified log penalty

A sequential learning method with Kalman filter and extreme learning machine for regression and time series forecasting

Diagnosing and correcting the effects of multicollinearity: Bayesian implications of ridge regression

Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data

Multicollinearity problem of CPM communication signals and its suppression method with PLS algorithm

Mixed integer quadratic optimization formulations for eliminating multicollinearity based on variance inflation factor

Biased estimators in Poisson regression model in the presence of multicollinearity: A subject review

Deep learning for sentiment analysis: A survey

Recent Trends in Deep Learning Based Natural Language Processing

BEST SUBSET SELECTION FOR ELIMINATING MULTICOLLINEARITY

A new maximum relevance-minimum multicollinearity (MRmMC) method for feature selection and ranking

Comprehensive study of feature selection methods to solve multicollinearity problem according to evaluation criteria

A Comparative Study on Regression Methods in the presence of Multicollinearity

A new modified Jackknifed estimator for the Poisson regression model

Partial ridge regression under multicollinearity

Ridge Regression for Solving the Multicollinearity Problem: Review of Methods and Models

Neural Machine Translation by Jointly Learning to Align and Translate

An incremental extreme learning machine for online sequential learning problems

TOSELM: Timeliness Online Sequential Extreme Learning Machine

A two-parameter estimator in the negative binomial regression model

Online sequential extreme learning machine in nonstationary environments

Comparison of statistical and machine learning methods in modelling of data with multicollinearity

Liu-Type Logistic Estimator

Comparison of regression analysis, Artificial Neural Network and genetic programming in Handling the multicollinearity problem

Model selection and psychological theory: a discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).

Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks

Introduction to Linear Regression Analysis

Sure independence screening for ultrahigh dimensional feature space

A Genetic Algorithm-Based, Hybrid Machine Learning Approach to Model Selection

Applied Linear Regression

Performance of some variable selection methods when multicollinearity is present

Symbolic regression in multicollinearity problems

Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy

Using Liu-Type Estimator to Combat Collinearity

A MORE GENERAL CRITERION FOR SUBSET SELECTION IN MULTIPLE LINEAR REGRESSION

PLS-regression: a basic tool of chemometrics

Using genetic programming to predict financial data

Systems modelling using genetic programming

An explanation of the use of principal-components analysis to detect and correct for multicollinearity

Collinearity, Power, and Interpretation of Multiple Regression Analysis

Diagnosing and Dealing with Multicollinearity

Finding Structure in Time

Multicollinearity in regression: Review and examples

Regression Diagnostics: Identifying Influential Data and Sources of Collinearity

Applied Linear Statistical Models

Selection of the Best Subset in Regression Analysis

Selection of Variables for Fitting Equations to Data

On multiple regression analysis

Mathematical Methods for Digital Computers

I. Problems and Designs of Cross-Validation 1

A Hybrid Feature Selection Optimization Model for High Dimension Data Classification

Improving the Classification Accuracy using Recursive Feature Elimination with Cross-Validation

Stresstest procedures for feature selection algorithms ∗

Principle Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression

Multicollinearity

Correlation-based Feature Selection for Machine Learning

Gradient-based learning applied to document recognition

Regression Shrinkage and Selection via the Lasso

A new class of blased estimate in linear regression