Year: 2025
Type: article
Abstract: Inference of large machine learning models can quickly exceed the capabilities of edge devices in terms of performance, memory or energy consumption. When offloading computations to a cloud server is not possible or feasible, for instance, due to data sovereignty concerns or latency constraints, a solution can be to distribute the inference load across multiple devices in a local edge network. We propose an approach which is capable of orchestrating multi-stage inference tasks in a mobile ad-hoc network consisting of heterogeneous devices in a self-organized and fully distributed manner. As individual edge devices may be battery-powered and volatile, the framework ensures a high degree of reliability even in dynamic environments. In particular, new nodes are automatically and seamlessly integrated into the ensemble, rendering the approach highly scalable. Moreover, resilience against spontaneous node dropouts or connection failures is implemented through adaptive task rerouting. Finally, by enabling complex inference tasks to be processed in small segments on the most suitable hardware available in the network, the ensemble is able to attain considerable pipelining performance and energy efficiency.
Source: Applied Sciences
DOI: 10.3390/app152312615
Institution: Fraunhofer Institute for Integrated Circuits
Language: English
Topic: IoT and Edge/Fog Computing
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
Award: G6497260524
OpenAlex ID: W4416850444