Work

On batch normalization for SPDnet

Year: 2026

Type: article

Abstract: This paper deals with batch normalization for SPDnet, a deep learning architecture specifically designed to handle covariance matrices. Current SPDnet batch normalization relies on geometric means, which require expensive iterative computations and lacks formal backpropagation derivations. These limitations are addressed through two contributions: (i) formal backpropagation derivation for SPDnet batch normalization, and (ii) computationally efficient closed-form alternatives to the geometric mean: log-Euclidean, arithmetic and harmonic means, and the geometric mean of arithmetic and harmonic means. Simulation results on the batch normalization layer with different means demonstrate significant reductions in memory consumption and comparable training time by avoiding automatic differentiation, demonstrating the effectiveness of our approach. Numerical experiments on three real datasets show that simpler means can outperform the geometric mean with up to 5% accuracy improvements while reducing training time by a factor of 5.

Source: Signal Processing

DOI: 10.1016/j.sigpro.2026.110793

Authors: Matthieu Gallet, Ammar Mian, Florent Bouchard, Guillaume Ginolhac

Institutions: Université Savoie Mont Blanc, Centre National de la Recherche Scientifique, Université Paris-Saclay, CentraleSupélec

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $3,190

Funder: Agence Nationale de la Recherche