Work

Latency-aware DNN Inference Acceleration with Dynamic Model Partitioning in Vehicular Edge Computing

Year: 2025

Type: article

Abstract: Distributed embedded inference architectures play a critical role in realizing intelligent decision-making for vehicles. Deep Neural Networks (DNNs) are a pivotal technology for driver assistance. During the operation of autonomous vehicles, sensors generate a massive amount of data, necessitating the rapid computation of DNN tasks. However, the computational resources and energy available locally on the vehicle are limited, making it challenging to respond swiftly to these tasks. Edge collaborative inference integrates edge computing with DNN task inference, harnessing the powerful computational capabilities of edge servers to complete the inference within tolerable latency. Scholars have employed model partitioning to accelerate edge collaborative inference. Nevertheless, research on model partitioning under local energy consumption constraints in vehicle networks is scarce, and few studies consider local vehicle constraints. Addressing the high computational demands of local DNN tasks, and the latency sensitivity that complicates DNN inference, this article proposes a method for collaborative DNN task inference based on model partitioning. In contrast to existing work, this study considers vehicle’s computational energy consumption and memory constraints, formulating a DNN model partitioning problem to minimize latency. The problem of DNN model partitioning under constraints is transformed into a maximum flow problem in a constrained Directed Acyclic Graph (DAG) network flow graph, and a DNN Model Partitioning Algorithm Based on Dinic’s Algorithm Under Constraints(MPAUC) is proposed for the solution. Compared to the DADs method, our approach achieves approximately a 1.5× reduction in latency and a 1.2× increase in throughput on the InceptionV3 network. These results demonstrate the superiority of the MPAUC algorithm in optimizing the deployment of deep learning models.

Source: ACM Transactions on Architecture and Code Optimization

DOI: 10.1145/3786345

Authors: Chunlin Li, Mengjie Yang, Zhihao Zhang, Xiaoheng Deng, Shaohua Wan

Institutions: Wuhan University of Technology, Central South University, University of Electronic Science and Technology of China

Language: English

Topic: IoT and Edge/Fog Computing

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: diamond

Funder: National Natural Science Foundation of China