Keyword

implicit regularization

Description: A tendency of an optimization method or model architecture to favor particular solutions without an explicit regularization penalty, especially in machine learning.

Primary topic: Stochastic Gradient Optimization Techniques (30%)

OpenAlex ID: keywords/implicit-regularization

Albumentations: Fast and Flexible Image Augmentations

2020 · Alexander Buslaev, Vladimir I. Iglovikov, et al. · Information · 2,696

PDF

Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data

2017 · Dziugaite, Gintare Karolina, Roy, Daniel M. · arXiv (Cornell University) · 320

PDF

Role of the calibration process in reducing model predictive error

2005 · Catherine Moore, John Doherty · Water Resources Research · 267

Improvement of Generalization Ability of Deep CNN via Implicit Regularization in Two-Stage Training Process

2018 · Qinghe Zheng, Mingqiang Yang, et al. · IEEE Access · 259

Stochastic Gradient Descent Performs Variational Inference, Converges to Limit Cycles for Deep Networks

2018 · Pratik Chaudhari, Stefano Soatto · Information Theory and Applications Workshop (ITA) · 221

PDF

Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution

2019 · Cong Ma, Kaizheng Wang, et al. · Foundations of Computational Mathematics · 205

PDF

Deep learning: a statistical viewpoint

2021 · Peter L. Bartlett, Andrea Montanari, et al. · Acta Numerica · 203

Just interpolate: Kernel “Ridgeless” regression can generalize

2020 · Liang, Tengyuan, Rakhlin, Alexander · DSpace@MIT (Massachusetts Institute of Technology) · 178

Kohn-Sham Equations as Regularizer: Building Prior Knowledge into Machine-Learned Physics

2021 · Li Li, Stephan Hoyer, et al. · Physical Review Letters · 170

PDF

Theoretical issues in deep networks

2020 · Tomaso Poggio, Andrzej Banburski, et al. · Proceedings of the National Academy of Sciences · 162

PDF

In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning

2014 · Neyshabur, Behnam, Tomioka, Ryota, et al. · arXiv (Cornell University) · 146

PDF

Implicit Regularization in Matrix Factorization

2018 · Suriya Gunasekar, Blake Woodworth, et al. · Information Theory and Applications Workshop (ITA) · 145

PDF

Implicit Regularization in Deep Matrix Factorization

2019 · Arora, Sanjeev, Cohen, Nadav, et al. · arXiv (Cornell University) · 143

PDF

Towards Understanding Regularization in Batch Normalization

2018 · Luo, Ping, Wang, Xinjiang, et al. · arXiv (Cornell University) · 140

PDF

Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations

2017 · Li, Yuanzhi, Ma, Tengyu, et al. · arXiv (Cornell University) · 140

PDF

From Variational to Deterministic Autoencoders

2019 · Ghosh, Partha, Sajjadi, Mehdi S. M., et al. · arXiv (Cornell University) · 139

PDF

Further Advantages of Data Augmentation on Convolutional Neural Networks

2018 · Alex Hernández-García, Peter König · Lecture notes in computer science · 129

PDF

Feature selection for support vector machines

2002 · L. Hermes, J.M. Buhmann · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · 112

On a PCA-based lung motion model

2011 · Ruijiang Li, John H Lewis, et al. · Physics in Medicine and Biology · 109

PDF

Gradient Descent Maximizes the Margin of Homogeneous Neural Networks

2019 · Lyu, Kaifeng, Li, Jian · arXiv (Cornell University) · 108

PDF

Action Robust Reinforcement Learning and Applications in Continuous Control

2019 · Tessler, Chen, Efroni, Yonathan, et al. · arXiv (Cornell University) · 108

PDF

Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Data

2021 · Cai, T. Tony, Ma, Rong · arXiv (Cornell University) · 107

PDF