Adapting deep neural networks to a low-power environment
Year: 2017
Type: dissertation
Abstract: These days, working with deep neural networks goes hand in hand with the use of\nGPUs. Once a deep neural network has been trained for hours, days, or even weeks on\na desktop GPU, it is deployed in the field where it runs inference computations, which\nare far less expensive than training. This fact, in conjunction with the mobile nature of\ndeep learning applications, makes very interesting the possibility of running inference\nlocally on a mobile device.\nThere already exist mobile applications that can perform tasks involving deep neural\nnetworks, but they rely on remote servers to run the most expensive computations.\nThis is not ideal because the user’s privacy may be compromised or the algorithm\nperformance may be damaged due to latency issues on a poor network connection. In\nthis project, the possibility of running inference natively on a mobile GPU is explored.\nOne of the main applications of deep learning is object recognition, which encompasses\ndifferent problems such as classification, identification and detection. In this\nproject, we select a very deep neural network called Faster R-CNN, which solves a\ndetection problem, and optimize it to run natively on a mobile platform. This innovation\nprovides to a mobile device, such as a smartphone or a tablet, the capability of\nidentifying objects and its location in images, potentially improving the performance\nof applications that currently employ the device camera combined with deep neural\nnetworks.\nHowever, mobile devices have limitations in power, memory, and compute capability.\nThis makes power and memory hungry applications such as deep neural networks\nhard to deploy, requiring smart software design. As a result, mobile presents both an\nopportunity and challenge for machine learning systems.\nA preliminary profiling of the network on the Nvidia Jetson TX1 module, a stateof-the-art\nplatform used in modern smartphones and handheld consoles, shows that the\nconvolutional and fully-connected layers take most of the forward pass execution time,\nup to 88.16% of the total. The network parameters take 548.3 MBytes of space, an\n87.2% of which belong to fully-connected layers. Hence, the main performance and\nenergy bottlenecks are on the convolutional and fully-connected layers.\nIn order to overcome these bottlenecks, two optimizations are proposed. In first\nplace, we use half-precision floating points instead of single-precision; this reduces by\nhalf the memory bandwidth, improving performance and providing energy savings. In\nsecond place, we implement a neuron-pruning technique to remove up to 80% of the\nneurons in the fully-connected layers; pruning reduces the memory footprint of the\nnetwork model and the amount of FP operations, reducing both energy consumption\nand execution time.\nTo evaluate the aforementioned optimizations, thorough experimentation is carried\nout on a Nvidia Jetson TX1 module. Results show that, combining all the optimizations,\nwe obtain, on average, a speedup of 1.55x, an energy reduction of 31.3%, an improvement\nin energy-delay of 2.26x and a memory footprint reduction of 86%.
Author: Oscar Mañas Sánchez
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2737493334
