Work

Benchmarking of closed vision-language models for Ki-67 index prediction in breast cancer histopathology images

Year: 2025

Type: article

Abstract: Quantifying the Ki-67 proliferation index is crucial for the prognosis of breast cancer. However, manual scoring remains subjective and labor-intensive. This study presents the first benchmarking of proprietary vision–language models (VLMs), including those from OpenAI, Google, xAI, and Anthropic, applied to breast cancer histopathology. Eight commercial models were evaluated in a strict zero-shot setting, using a unified, guideline-based prompt across two available, annotated datasets (BCData, n = 402 ; SHIDC-B-Ki-67, n = 700 ). Results show substantial differences between providers: on BCData, OpenAI’s GPT-4.5 achieved the best concordance with expert annotations ( R 2 = 0.8570 , RMSE=7.9708) and reached a macro-F1 of 74.56 % for three-class cut-off classification (Low: Ki-67 < 16 % (less proliferative), Medium: 16 % ≤ Ki-67 < 30 % (intermediate proliferation), High: Ki-67 ≥ 30 % (highly proliferative)); on SHIDC-B-Ki-67, GPT-4.1 mini led with R 2 = 0.5465 , RMSE=16.8202, and macro-F1 of 66.86 % across the same ranges. Inference times ranged from 1.67 to 6.84 s per image. Our findings highlight that closed VLMs can deliver an accurate process for index calculation and rapidly deployable Ki-67 quantification, while revealing significant vendor-specific variability and potential use as a complementary tool for medical professionals. This work establishes a new zero-shot benchmarking framework, using vision–language models guided by human instructions, as a reference baseline for future Ki-67 quantification studies.

Source: Expert Systems with Applications

DOI: 10.1016/j.eswa.2025.130196

Authors: Harold Brayan Arteaga-Arteaga, Juan Andrés Giraldo-Arias, Mario Alejandro Bravo-Ortíz, Cristina Juárez Landín, Pablo Guillen-Rondon, Reinel Tabares-Soto