Work

SkipW: Resource Adaptable RNN with Strict Upper Computational Limit

Year: 2021

Type: article

Abstract: We introduce Skip-Window, a method to allow recurrent neural networks (RNNs) to trade off accuracy for computational cost during the analysis of a sequence. Similarly to existing approaches, Skip-Window extends existing RNN cells by adding a mechanism to encourage the model to process fewer inputs. Unlike existing approaches, Skip-Window is able to respect a strict computational budget, making this model more suitable for limited hardware like edge devices. We evaluate this approach on four datasets: a human activity recognition task, sequential MNIST, IMDB and adding task. Our results show that Skip-Window is often able to exceed the accuracy of existing approaches for a lower computational cost while strictly limiting said cost.

Source: Zenodo (CERN European Organization for Nuclear Research)

DOI: 10.5281/zenodo.4911370

Authors: Mayet, Tsiry, Lambert, Anne, Le Guyadec, Pascal, Le Bolzer, Francoise, Schnitzler, Francois

Institution: InterDigital (United States)

Language: English

Topic: Advanced Neural Network Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: European Commission

Award: A European Excellence Centre for Media, Society and Democracy