methodology-8
L2 Regularization
3260 papers • 126 benchmarks • 313 datasets
See Weight Decay. $L_{2}$ Regularization or Weight Decay, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L_{2}$ Norm of the weights: $$L_{new}\left(w\right) = L_{original}\left(w\right) + \lambda{w^{T}w}$$ where $\lambda$ is a value determining the strength of the penalty (encouraging smaller weights). Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).
(Image credit: Papersgraph)
Benchmarks
These leaderboards are used to track progress in l2-regularization-8
Trend
Dataset
Best Model
Actions
No benchmarks available.
Libraries
iUse these libraries to find l2-regularization-8 models and implementations
Datasets
No datasets available.
Subtasks
No subtasks available.
Most implemented papers
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
Z. Kira, Yen-Cheng Liu, Yen-Chang Hsu•Mon Oct 29 2018
The results provide an understanding of the relative difficulty of the scenarios and that simple baselines (Adagrad, L2 regularization, and naive rehearsal strategies) can surprisingly achieve similar performance to current mainstream methods.
386