Year: 2024
Type: preprint
Abstract: Training Neural ODEs requires backpropagating through an ODE solve. The state-of-the-art backpropagation method is recursive checkpointing that balances recomputation with memory cost. Here, we introduce a class of algebraically reversible ODE solvers that significantly improve upon both the time and memory cost of recursive checkpointing. The reversible solvers presented calculate exact gradients, are high-order and numerically stable -- strictly improving on previous reversible architectures.
Source: arXiv (Cornell University)
Authors: McCallum, Sam, Foster, James
Institution: University of Bath
Language: English
Subfield: Numerical Analysis
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4403574996