Year: 2020
Type: preprint
Abstract: In this paper we present a C++ implementation of the analytic derivative of a feed-forward neural network with respect to its free parameters for an arbitrary architecture, known as back-propagation. We dubbed this code NNAD (Neural Network Analytic Derivatives) and interfaced it with the widely-used ceres-solver minimiser to fit neural networks to pseudodata in two different least-squares problems. The first is a direct fit of Legendre polynomials. The second is a somewhat more involved minimisation problem where the function to be fitted takes part in an integral. Finally, using a consistent framework, we assess the efficiency of our analytic derivative formula as compared to numerical and automatic differentiation as provided by ceres-solver. We thus demonstrate the advantage of using NNAD in problems involving both deep or shallow neural networks.
Source: arXiv (Cornell University)
Authors: Khalek, Rabah Abdul, Bertone, Valerio
Institutions: National Institute for Subatomic Physics, Vrije Universiteit Amsterdam, University of Pavia, Commissariat à l'Énergie Atomique et aux Énergies Alternatives, Université Paris-Saclay, CEA Paris-Saclay - Etablissement de Saclay, Istituto Nazionale di Fisica Nucleare, Sezione di Pavia, Institut de Recherche sur les Lois Fondamentales de l'Univers
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3025643614