Improving ViT Performance for Colon Cancer Histopathological Image Classification Using Transfer Learning
Year: 2025
Type: conference-paper
Abstract: Deep learning has revolutionized medical image analysis, particularly the automatic classification of histopathology images. Vision Transformers (ViTs) now surpass convolutional neural networks (CNNs) in modeling long-range dependencies and capturing global features. However, when initialized with random weights, ViTs often struggle to adapt to the varying scales and morphologies of tumor tissues, resulting in suboptimal performance. Transfer learning addresses this challenge by pretraining the model on large datasets to obtain a stronger weight initialization before fine-tuning on the target domain. This strategy has proven especially effective for histology image classification. In this study, we fine-tuned a ViT-B/16 model pretrained on ImageNet-21k by first training it on 75,000 Renal Cell Carcinoma (RCC) patches and then applying it to classify colon tumor subtypes in the Chaoyang dataset. Our proposed method achieves an accuracy of 87.14 %, a precision of 83.59 %, a recall of 82.08 % and an F1 score of 82.77 %, outperforming both a model trained from scratch and one pretrained only on ImageNet. These results underscore the value of domain-specific pretraining for enhancing ViT performance on diverse tumor morphologies in histopathology classification tasks.
Institutions: University of Abou Bekr Belkaïd, Université de ain Témouchent, Centre National de la Recherche Scientifique, Institut national de recherche en sciences et technologies du numérique, Laboratoire d'Informatique, Signaux et Systèmes de Sophia Antipolis, Université Nice Sophia Antipolis, Centre Hospitalier Universitaire de Nice, Hôpital Pasteur
Language: English
Topic: AI in cancer detection
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7124944006