Year: 2026
Type: preprint
Abstract: We systematically investigate finite-difference (FD) derivative computation in Physics-Informed Neural Networks (PINNs) as an alternative to automatic differentiation (AD). On three benchmark PDEs we show that, with a properly calibrated step size, FD matches AD in accuracy on every problem while running faster across the full tested batch-size range and using substantially less GPU memory, and that a stochastic variant we propose outperforms AD on a stationary problem. We further show that for neural architectures with inter-sample dependencies (e.g. BatchNorm, self-attention) the standard PyTorch autograd idiom is silently incorrect; the correct per-sample alternative is computationally infeasible at PINN-relevant batch sizes, while FD provides a forward-only approximation that is empirically an order of magnitude closer to the true per-sample derivative.
Source: arXiv (Cornell University)
Authors: Mikulski, Maciej J., Uhl, Tadeusz
Institution: AGH University of Krakow
Subfield: Statistical and Nonlinear Physics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7202277386
Cites:
Cited by:
Related to: