On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice (2020-07-30T00:00:00.000000Z)

TL;DR

This survey paper will help industrial users, data analysts, and researchers to better develop machine learning models by identifying the proper hyper-parameter configurations effectively and introducing several state-of-the-art optimization techniques.

Authors

Li Yang

2 papers

A. Shami

2 papers

TL;DR

Authors

References131 items

Multi-split optimized bagging ensemble model selection for multi-class educational data mining

Ensemble-based Feature Selection and Classification Model for DNS Typo-squatting Detection

Systematic Ensemble Model Selection Approach for Educational Data Mining

Student Engagement Level in an e-Learning Environment: Clustering Using K-means

Machine Learning for Performance-Aware Virtual Network Function Placement

Tree-Based Intelligent Intrusion Detection System in Internet of Vehicles

Automated Architecture Design for Deep Neural Networks

A Survey of Optimization Methods From a Machine Learning Perspective

Automated Machine Learning: State-of-The-Art and Open Challenges

Benchmark and Survey of Automated Machine Learning Frameworks

Effect of the Sampling of a Dataset in the Hyperparameter Optimization Phase over the Efficiency of a Machine Learning Algorithm

Clustering Enabled Classification using Ensemble Feature Selection for Intrusion Detection

Optimization Models for Machine Learning: A Survey

DNS Typo-Squatting Domain Detection: A Data Analytics &amp; Machine Learning Based Approach

Bayesian Optimization with Machine Learning Algorithms Towards Anomaly Detection

Image-Based Visibility Estimation Algorithm for Intelligent Transportation Systems

Automated Machine Learning: From Principles to Practices

Sherpa: Hyperparameter Optimization for Machine Learning Models

Numerical Optimization

Multiobjective Dynamic-Guiding PSO for Optimizing Work Shift Schedules

Hyper‐parameters optimisation of deep CNN architecture for vehicle logo recognition

BOHB: Robust and Efficient Hyperparameter Optimization at Scale

Extending MLP ANN hyper-parameters Optimization by using Genetic Algorithm

Comprehensive Visibility Indicator Algorithm for Adaptable Speed Limit Control in Intelligent Transportation Systems

Genetic algorithms for hyperparameter optimization in predictive business process monitoring

A new image classification method using CNN transfer learning and web data augmentation

Combination of Hyperband and Bayesian Optimization for Hyperparameter Optimization in Deep Learning

Effective hyperparameter optimization using Nelder-Mead method in deep learning

Informing the Use of Hyperparameter Optimization Through Metalearning

GPflowOpt: A Bayesian Optimization Library using TensorFlow

A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring

Particle swarm optimization for hyper-parameter selection in deep neural networks

Deep-learning: investigating deep neural networks hyper-parameters and comparison of performance to shallow methods for modeling bioactivity data

Hyperparameter Optimization: A Spectral Approach

An effective algorithm for hyperparameter optimization of neural networks

A Novel Hardware/Software Partitioning Method Based on Position Disturbed Particle Swarm Optimization with Invasive Weed Optimization

Forward and Reverse Gradient-Based Hyperparameter Optimization

Comparative Study of CNN and RNN for Natural Language Processing

TPOT: A Tree-based Pipeline Optimization Tool for Automating Machine Learning

Osprey: Hyperparameter Optimization for Machine Learning

SystemML: Declarative Machine Learning on Spark

A new optimal sampling rule for multi-fidelity optimization via ordinal transformation

Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates

A review of automatic selection methods for machine learning algorithms and hyper-parameter values

COCO: a platform for comparing continuous optimizers in a black-box setting

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

XGBoost: A Scalable Tree Boosting System

The impact of particles initialization in PSO: Parameter estimation as a case in point

Automating model search for large scale machine learning

Unbounded Bayesian Optimization via Regularization

Hyperopt: a Python library for model selection and hyperparameter optimization

Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

Machine learning: Trends, perspectives, and prospects

MLlib: Machine Learning in Apache Spark

Gradient-based Hyperparameter Optimization through Reversible Learning

Hyperparameter Search in Machine Learning

Efficient Benchmarking of Hyperparameter Optimizers via Surrogates

Easy Hyperparameter Search Using Optunity

A review of population initialization techniques for evolutionary algorithms

BayesOpt: a Bayesian optimization library for nonlinear optimization, experimental design and bandits

Convex Optimization: Algorithms and Complexity

A Tutorial on Principal Component Analysis

Metaheuristics: review and application

Gaussian Processes for Big Data

Almost Optimal Exploration in Multi-Armed Bandits

Fast and Accurate Sentiment Classification Using an Enhanced Naive Bayes Model

Applied Predictive Modeling

ImageNet classification with deep convolutional neural networks

Practical Bayesian Optimization of Machine Learning Algorithms

Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions

A classification system for remote sensing satellite images using support vector machine with non-linear kernel functions

Random Search for Hyper-Parameter Optimization

Algorithms for Hyper-Parameter Optimization

Scikit-learn: Machine Learning in Python

Sequential Model-Based Optimization for General Algorithm Configuration

On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation

DNS Typo-Squatting Domain Detection: A Data Analytics & Machine Learning Based Approach