Work

Delta-NAS: Difference of Architecture Encoding for Predictor-Based Evolutionary Neural Architecture Search

Year: 2025

Type: conference-paper

Abstract: Neural Architecture Search (NAS) continues to serve a key roll in the design and development of neural networks for task specific deployment. Modern NAS techniques struggle to deal with ever increasing search space complexity and compute cost constraints. Existing approaches can be categorized into two buckets: fine-grained computational expensive NAS and coarse-grained low cost NAS. Our ob-jective is to craft an algorithm with the capability to per-form fine-grain NAS at a low cost. We propose projecting the problem to a lower dimensional space through pre-dicting the difference in accuracy of a pair of similar net-works. This paradigm shift allows for reducing computational complexity from exponential down to linear with re-spect to the size of the search space. We present a strong mathematical foundation for our algorithm in addition to extensive experimental results across a host of common NAS Benchmarks. Our methods significantly out performs ex-isting works achieving better performance coupled with a significantly higher sample efficiency.

Source: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

DOI: 10.1109/wacv61041.2025.00763

Authors: Arjun Sridhar, Yiran Chen

Institution: Duke University

Language: English

Topic: Advanced Neural Network Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Award: AI Institute for Edge Computing Leveraging Next Generation Networks (Athena)