Work

Neural Architecture Search and Automatic Code Optimization: Techniques, trends, and challenges

Year: 2025

Type: article

Abstract: Deep Learning models have experienced exponential growth in complexity and resource demands in recent years. Accelerating these models for efficient execution on resource-constrained devices has become more crucial than ever. Two notable techniques used to achieve this goal are Hardware-Aware Neural Architecture Search (HW-NAS) and Automatic Code Optimization (ACO). HW-NAS automatically designs accurate yet hardware-friendly neural networks, while ACO involves searching for the best code optimizations to apply on neural networks for efficient mapping and inference on the target hardware. This review explores recent work that combines these two techniques within a single framework. We present the fundamental principles of both domains and demonstrate their suboptimality when performed independently. We then investigate their integration into a joint optimization process that we call Hardware Aware- N eural A rchitecture and C ompiler O ptimizations co- S earch (NACOS).

Source: Journal of Systems Architecture

DOI: 10.1016/j.sysarc.2025.103645

Authors: Inas Bachiri, Smaïl Niar, Riyadh Baghdadi, Hamza Ouarnoughi