Evaluation of Machine Learning Inference Workloads on Heterogeneous Edge Devices
Year: 2020
Type: dissertation
Abstract: Machine learning has traditionally been a thing of powerful computers and even supercomputers. Recent increases in computational power and the development of specialized architecture led to the possibility to perform machine learning, especially inference, on the edge. Those edge devices include small computers like the Nvidia Jetson Nano and accelerators like the Intel Neural Compute Sticks. In this thesis these devices and their respective frameworks are compared amongst each other and to an Apple MacBook Pro which represents a more traditional machine learning device. The comparisons are done by using an almost identical inference script across all platforms. The maximum possible throughput is also tested with tools specialized on the respective framework. Four models with a different number of convolutional layers were used to put a special emphasis on this kind of layer. The precision of the models dropped after the conversion to OpenVINO for the Neural Compute Sticks, but remained equally high on the other frameworks. Regarding performance, the MacBook was still the fastest device overall, but lost more performance proportionally to edge devices when used with an increasing number of convolutional layers. Amongst the edge devices, the Jetson Nano was the fastest, followed by the 2nd gen Neural Compute Stick and the 1st gen Neural Compute Stick.
Author: Rubin, Marco
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6981761789