Year: 2023
Type: article
Abstract: Vision transformers have challenged the decade-long dominance of convolutional neural networks in image recognition,demonstrating that the self-attention mechanism can capture visual features as effectively as convolutions when trainedwith sufficient data and appropriate regularisation. This study presents a controlled comparison of six vision architectures-- standard ViT, DeiT (data-efficient ViT with distillation), Swin Transformer (hierarchical shifted windows), ConvNeXt(modernised CNN), hybrid CoAtNet (convolution + attention), and EfficientNetV2 (CNN baseline) -- across four imagerecognition tasks: ImageNet-1K classification, fine-grained recognition (CUB-200 birds, Stanford Cars), out-of-distributionrobustness (ImageNet-C, ImageNet-R, ImageNet-Sketch), and transfer learning to small datasets (CIFAR-100, OxfordFlowers). All models were evaluated at three computational scales (small ~22M, base ~86M, large ~300M parameters)with matched training recipes to isolate architectural effects. A total of 2,520 experiments were conducted. SwinTransformer achieved the highest ImageNet-1K accuracy at all scales (84.5 +- 0.2% at base), outperforming ViT (81.8%),DeiT (83.1%), ConvNeXt (84.1%), CoAtNet (84.3%), and EfficientNetV2 (83.6%). On out-of-distribution robustness, ViTvariants showed 18.4% smaller effective robustness gap than CNNs of equivalent accuracy, confirming thatattention-based architectures develop more shape-biased representations. ConvNeXt matched Swin's ImageNetaccuracy while retaining CNN deployment simplicity (no shifting window logic), suggesting that architectural innovationsfrom transformers can be back-ported to CNNs. On fine-grained recognition, attention-based models outperformed CNNsby 2.4-4.8 points, with attention maps revealing learned part-based decomposition of objects. A practical architectureselection guide mapping dataset size, robustness requirements, and deployment constraints to recommendedconfigurations is proposed.
Source: Bio-QI Journal
Authors: Andreas Muller, Laura Hansen, Laura Ivanov
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7203693746