Backpropagation as Dirac-bracket dynamics of dissipative training
Year: 2026
Type: preprint
Abstract: Gradient-based training of a parametric model is the motion of a damped particle in the potential set by the loss. Using the Bateman–Galley doubling of the degrees of freedom, which gives dissipative systems an action principle, we write gradient flow, the heavy-ball method and Nesterov's accelerated gradient method as Hamiltonian systems. We show that, within this Hamiltonian description, backpropagation is Dirac evolution. The activations of a feed-forward network are treated as coordinates of phase space, and the layer equations as constraints. These constraints are second class for every architecture, and Hamilton's equations with the resulting Dirac bracket reproduce training with the exact gradient of the loss, although the Hamiltonian contains only the cost of the network output. The Dirac bracket of an activation with the momenta of the parameters is its Jacobian, the inverse of the constraint matrix contains both the forward and the reverse modes of automatic differentiation, and the errors propagated by backpropagation are the Lagrange multipliers of the layer constraints: the constraint forces of a kinematic chain under the load of the cost. These results hold in the full doubled phase space, not only in the physical limit. The network's part of the constraint algebra is the same for every optimizer, Nesterov's look-ahead adds one layer of momentum-dependent constraints that couple the two copies of the doubled system, and gradient flow is obtained by constructing the Dirac bracket in stages. We confirm these statements numerically to machine precision and train a network by integrating the Dirac flow of its constraints, with no backward pass in the computation.
Institution: Universidade do Estado de Santa Catarina
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Keywords: backpropagation, Dirac brackets, Hamiltonian mechanics, dissipative systems, gradient flow, heavy ball method, Nesterov acceleration, neural network training, phase space constraints, second-class constraints, automatic differentiation, Lagrange multipliers, feedforward neural networks, Hamiltonian neural networks
Open Access status: green
OpenAlex ID: W7236133894
