Get started
Work

Hierarchical multi-model ensemble with auxiliary CLIP fusion for fine-grained art style recognition

Year: 2026
Type: article
Abstract: Fine-grained art style recognition remains difficult because adjacent styles often share subjects, color structures, brushwork, and historical transitions. This study presents a controlled evaluation-and-fusion framework that combines heterogeneous visual classifiers, a low-weight fine-tuned CLIP component, and post-hoc hierarchical diagnostics for ArtBench-10. The resulting Hierarchical Multi-Model Ensemble with Auxiliary CLIP Fusion (HMME-ACF) uses validation-selected probability averaging across five ImageNet-pretrained visual backbones and adds CLIP ViT-B/32 only as a minor auxiliary component. Experiments use 45,000 training images, 5000 validation images, and 10,000 test images under a fixed protocol. The full configuration achieves 86.15% top-1 accuracy, 85.93% Macro-F1, and 86.15% balanced accuracy on the 10-class task. Deterministic post-hoc aggregation yields 92.07%, 89.76%, and 88.03% top-1 accuracy at the period_3, family_4, and family_5 levels, respectively; an explicit hierarchical multitask loss provides only modest coarse-level gains while slightly reducing fine-grained accuracy. The auxiliary CLIP component improves top-1 accuracy by 0.43 percentage points and mainly benefits the most confusable Western style pairs. Accuracy-efficiency analysis shows that the full 84-pass configuration is best suited to offline cataloguing, whereas a five-model no-TTA variant retains 84.27% accuracy at 80 ms per image. These findings position HMME-ACF as a controlled ensemble and diagnostic framework rather than a new hierarchical learning architecture.
DOI: 10.1038/s41598-026-66795-5
Language: English
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7204098263
FWCI:
Cites:
Cited by:
Related to: