Year: 2026
Type: preprint
Abstract: We study several aspects of the semi-algebraicity of the neuromanifold $\mathcal{M}_{\mathbf{d}}$ of a feedforward ReLU neural network. We prove that $\mathcal{M}_{\mathbf{d}}$ in general is not a semi-algebraic quotient of the space of parameters of the network, but it can be approximated (in a precise sense) by semi-algebraic spaces. We then consider the notion of identifiable subsets of the space of parameters, where the network does not have any hidden symmetries. We prove that the maximal identifiable subset is semi-algebraic for any neural network with semi-algebraic activation functions. Lastly, we compute explicitly the identifiable subset of a shallow ReLU network and prove that for input dimensions $\ge 3$ and width $\ge 6$ it is not euclidean open.
Source: arXiv (Cornell University)
Institution: Umeå University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7163370601