Work

Intermittent Inference: Trading a 1% Accuracy Loss for a 1.9x Throughput Speedup

Year: 2024

Type: conference-paper

Abstract: We present INTERCEPT, a compile-time toolchain enabling manifold throughput improvements when running intermittent DNN inference on IoT devices, in exchange of a maximum 1% accuracy loss. Intermittently-computing IoT devices rely on ambient energy harvesting and compute opportunistically, as energy is available. They use NVM to persist intermediate results in anticipation of energy failures. Without requiring changes to existing models and by exploiting the features of STT-MRAM as NVM, INTERCEPT optimizes the placement and configuration of state persistence operations when executing the inference process. This happens off-line with no user intervention, while enforcing a maximum 1% accuracy loss. Our results, obtained across three platforms and six diverse neural networks, indicate that INTERCEPT provides a 40% energy gain in a single inference process, on average. With the same energy budget, this yields a 1.9x throughput speedup.

Source: ACM Conference on Embedded Networked Sensor Systems (SenSys)

DOI: 10.1145/3666025.3699364

Authors: Rei Barjami, Antonio Miele, Luca Mottola

Institutions: Politecnico di Milano, Uppsala University

Language: English

Topic: Parallel Computing and Optimization Techniques

Subfield: Hardware and Architecture

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: hybrid

Funders: European Commission, Ministero dell'Università e della Ricerca