A Hybrid CNN-Transformer with Window-Based Attention for Vegetation-Soil Semantic Segmentation in Agricultural Vision Systems
Year: 2026
Type: article
Abstract: Accurate vegetation-soil segmentation is essential for agricultural vision systems operating under variable field conditions. This study evaluates a hybrid CNN-Transformer architecture combining a convolutional pre-encoder, hierarchical shifted-window attention, and a U-Net-like decoder for segmentation in structured crop-row environments. A 13-channel input representation incorporating ten RGB-derived vegetation descriptors was investigated without requiring additional spectral sensors. Model development systematically examined input representation, optimizer, Transformer depth, loss function, and patch size. The final configuration was evaluated across five independent training runs and compared with conventional CNN, lightweight, Transformer-based, and hybrid segmentation architectures. On the internal test set, Hybrid-13Ch achieved 95.9 ± 0.22% Global Accuracy, 96.3 ± 0.76% Mean Accuracy, 89.1 ± 0.37% mean Intersection over Union (mIoU), and 96.6 ± 0.53% Mean BFScore. On an external test set comprising 300 annotated frames from two previously unseen video sequences, it achieved 87.3 ± 0.39% mIoU and 94.3 ± 0.58% Mean BFScore. Incremental and subtractive ablation experiments assessed the contributions of the convolutional pre-encoder, shifted-window attention, and MLP processing. These findings support the proposed architecture and RGB-derived representation for agricultural semantic segmentation.
Source: Applied Sciences
DOI: 10.3390/app162211486
Language: English
Topic: Smart Agriculture and AI
Subfield: Plant Science
Domain: Life Sciences
SDG: Zero hunger
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W7236567442
