Year: 2023
Type: dissertation
Abstract: Methods and Applications for Low-Power Deep Neural Networks on Edge Devices by Luciano PRONO Machine Learning (ML) techniques introduce the possibility of solving many complex tasks requiring humongous amounts of data.In particular, Deep Neural Networks (DNNs) are structures that provide excellent results with the use of a massive number of trainable parameters.At the same time, the implementation of DNNs on cheap and low-power devices with high resource constraints, referred to as tinyML, would unlock a large number of entirely new applications possible in the fields of Internet of Things (IoT) and Edge Computing.In this dissertation, we focus on the design of low-power, low-memory DNN structures and we implement them with novel Edge Computing approaches such as an innovative Compressed Sensing (CS) DNN-based decoder.In the first part, we focus on the methods and architectures for the design of low-power and tiny DNN structures.Firstly, we introduce parameters quantization and interconnection pruning methods.Secondly, we present two novel map-reduce alternatives to Multiply and Accumulate (MAC) paradigm, namely Sum and Max (SAM) and Multiply and Max-Min (MAM²), and we use them to build DNN layers that are highly resistant to the pruning process.Additionally, we present an on-device implementation of a MAM² layer, where the implementation with standard MAC-based layers would not be possible.Finally, we present an on-device implementation of a Recurrent Spiking Neural Network (RSNN), intrinsically low-power due to its sparsity properties.In the second part, we introduce a novel CS decoder based on DNNs.Firstly, we describe the novel encoding-decoding CS framework, with the decoding phase split in two distinct working blocks, namely support retrieval and signal reconstruction.This novel approach outperforms state-of-the-art techniques in terms of quality of reconstruction.Finally, we apply some of the methods of the first part of this dissertation to study how this structure can be compressed to reduce the hardware requirements, showing that this novel approach to CS needs very few resources.
Source: PORTO@Iris
Author: PRONO, LUCIANO
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7217675147