Work

Batch Normalization is a Cause of Adversarial Vulnerability

Year: 2019

Type: preprint

Abstract: Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets. Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension. Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.1905.02161

Authors: Galloway, Angus, Golubeva, Anna, Tanay, Thomas, Moussa, Medhat, Taylor, Graham W.

Institutions: Vector Institute, University of Guelph, Huawei Technologies (China)

Language: English

Topic: Adversarial Robustness in Machine Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Nvidia, Canadian Institute for Advanced Research, Compute Canada, Industry Canada, Institut Périmètre de physique théorique, Natural Sciences and Engineering Research Council of Canada, Engineering and Physical Sciences Research Council

OpenAlex ID: W2942801205

Cites: 33

Cited by: 58

Related to: 20