Year: 2019
Type: preprint
Abstract: Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, the high cost of generating strong adversarial examples makes standard adversarial training impractical on large-scale problems like ImageNet. We present an algorithm that eliminates the overhead cost of generating adversarial examples by recycling the gradient information computed when updating model parameters. Our "free" adversarial training algorithm achieves comparable robustness to PGD adversarial training on the CIFAR-10 and CIFAR-100 datasets at negligible additional cost compared to natural training, and can be 7 to 30 times faster than other strong adversarial training methods. Using a single workstation with 4 P100 GPUs and 2 days of runtime, we can train a robust model for the large-scale ImageNet classification task that maintains 40% accuracy against PGD attacks. The code is available at https://github.com/ashafahi/free_adv_train.
Source: arXiv (Cornell University)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funders: National Science Foundation, U.S. Department of Defense, Office of the Director of National Intelligence, Defense Advanced Research Projects Agency, Intelligence Advanced Research Projects Activity, Multidisciplinary University Research Initiative, Office of Naval Research, Division of Electrical, Communications and Cyber Systems, Air Force Office of Scientific Research
Awards: SpecEES: Spatio-Spectral Sensing with Wideband Feature Extraction Arrays, AitF: EXPL: Collaborative Research: Approximate Discrete Programming for Real-Time Systems, Collaborative Research: BAMM: Baseband Accelerators for Massive Multiple-Input Multiple-Output (MIMO) Technology, CAREER: Hardware Accelerated Bayesian Inference via Approximate Message Passing: A Bottom-Up Approach, G4383626004, NeTS: Small: Collaborative Research: BRICK: Breaking the I/O and Computation Bottlenecks in Massive MIMO Base Stations, G73875660
OpenAlex ID: W2940648411