Spiking the transformer: NeuViT with single-step attention for efficient high-resolution UAV vision
Year: 2026
Type: article
Abstract: • Real-time high-resolution drone vision under a ten watts power limit • Brain-inspired, event-driven design cuts energy use by 89 percent • Delivers 22 FPS at 3.54W on 1500 × 2000 UAV images under real-time constraints • Keeps 79 percent of detection accuracy within tight power limits • Scales to megapixel images without runaway computing costs High-resolution aerial imagery from unmanned aerial vehicles (UAVs) provides the fine-grained spatial detail required for reliable perception during field missions, yet onboard processing of such data is constrained by high computational, latency, and energy demands. We present Neuromorphic Vision Transformer (NeuViT), a spiking vision transformer tailored for real-time high-resolution object detection under the strict power and latency constraints of UAV platforms. NeuViT integrates a single-step stateless spiking paradigm that removes temporal dependencies and membrane state tracking, and a spike-driven multi-head self-attention mechanism that replaces dense attention with binary spike coincidences, eliminating all multiply-accumulate operations. Evaluated on the VisDrone2019 dataset at 1500 × 2000 resolution, NeuViT achieves 88.8 percent lower energy consumption and 31 percent lower latency than Swin Transformer, while retaining 79.4 percent of its detection accuracy. NeuViT remains fully operational within a 10 W power budget and sustains more than 22 frames per second, whereas baseline vision transformers, efficient convolutional neural networks, and spiking neural networks fail to meet UAV deployment requirements. Through per-class precision analysis, confusion matrix characterization, and spike distribution correlation, we identify that accuracy loss concentrates on small, low-contrast objects where spike thresholds suppress weak activations, informing an enhancement roadmap. Energy projections across 45 nm, 16 nm, and 7 nm technology nodes confirm that NeuViT maintains sub-10 W operation regardless of fabrication process. These results demonstrate that neuromorphic principles, when pragmatically adapted to silicon constraints, can enable high-resolution vision capabilities in resource-constrained aerial platforms.
Source: Results in Engineering
Institution: Monash University Malaysia
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,940
OpenAlex ID: W7118732028