Depthwise Separable Convolutions Allow for Fast and Memory-Efficient Spectral Normalization
Year: 2021
Type: preprint
Abstract: An increasing number of models require the control of the spectral norm of convolutional layers of a neural network. While there is an abundance of methods for estimating and enforcing upper bounds on those during training, they are typically costly in either memory or time. In this work, we introduce a very simple method for spectral normalization of depthwise separable convolutions, which introduces negligible computational and memory overhead. We demonstrate the effectiveness of our method on image classification tasks using standard architectures like MobileNetV2.
Source: arXiv (Cornell University)
Institutions: University of Siegen, University of Cambridge
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: Nonlocal Methods for Arbitrary Data Sources, Robust and Efficient Analysis Approaches of Remote Imagery for Assessing Population and Forest Health in India, Cambridge Mathematics of Information in Healthcare (CMIH), EPSRC Centre for Mathematical and Statistical Analysis of Multimodal Clinical Imaging, PET++: Improving Localisation, Diagnosis and Quantification in Clinical and Medical PET Imaging with Randomised Optimisation
OpenAlex ID: W3133479497