Constraint Unlearnability in Overparameterized Neural Networks
Year: 2026
Type: article
Abstract: We study whether overparameterized neural networks can learn to satisfy pointwise constraints without inductive bias. One might expect that a sufficiently expressive network would learn to respect such constraints. We prove that this is not the case. For overparameterized feedforward neural networks with real-analytic activation functions, assuming the manifold of global minima is connected, the set of parameters for which the network satisfies a real-analytic pointwise constraint has measure zero on this manifold. Since gradient descent converges to this manifold, it almost surely finds solutions that violate the constraint. The proof applies the identity theorem for real-analytic functions twice: first, to show that a network either satisfies the constraint everywhere or violates it almost everywhere; second, to show that overparameterization excludes the everywhere-satisfied case. We call this phenomenon constraint unlearnability—despite interpolating the training data, the network cannot satisfy the constraint exactly on the entire input domain.
Source: Mathematics
DOI: 10.3390/math14173039
Authors: Hongyu Qi, Zhen Tan
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7170284971