1221 Deciphering cellular features: StyleGAN for enhancing H&E cell classification
Year: 2024
Type: conference-abstract
Abstract: Background A critical challenge in immunotherapy research is the distinction between tumor-specific CD39+CD8+ T cells and non-specific ‘bystander’ cells.1 Deep classifiers for identifying cells in hematoxylin and eosin (H&E)-stained images are effective but lack interpretability. These models may rely on shortcut patterns instead of generalizable features. Biases in cell datasets, such as those from patients and equipment, combined with low human expert classification accuracy, reduce the trustworthiness of these black-box models and limit the broader applicability of cell classifiers. We propose using StyleGAN to improve classifier interpretability. By synthesizing diverse cell images, we can observe how a CD39CD8 classifier’s confidence varies. This approach helps identify both meaningful image patterns and spurious shortcuts that influence high-confidence classifications, improving the trustworthiness of deep cell classifiers. Methods Tissue sections from five patients with colorectal cancer first underwent multiplex immunohistochemistry (mIHC) staining to target CD39 and CD8 markers and imaged with Axioscan 7 (Zeiss, Germany). The same sections were then stained with H&E followed by brightfield imaging on the same system. Cell labels from mIHC images were registered to H&E images. Individual 100x100-pixel cell tiles were obtained from H&E images, followed by a train-validation-test split. We trained a DenseNet-121 CD39+CD8+ cell classifier using cross entropy, Adam optimizer, batch size 128, learning rate 1e-4, with TensorFlow 2.10. The decision threshold was optimized for the highest F1 score. Subsequently, we trained StyleGAN32 with hyperparameters from their official github (figure 1). During synthesis, we linearly interpolate 100 points between two distinct style vectors to produce cell images, and analyze how the cell classifier confidence varies between tumor-specific and bystander CD8+ T cell states (figure 2). Results Analysis of synthesized images reveals distinct cellular patterns that correspond to high classifier confidence: small, dense nuclei for CD39+CD8+, larger, irregular, and translucent nuclei for CD39-CD8-, and nuclei clusters for CD39-CD8+. Principal component analysis showed that CD8+ T-cells, regardless of tumor specificity, exhibit similar style vectors, indicating analogous semantic features. However, tumor-specific CD8+ T-cells appear morphologically closer to CD39-CD8-cells. Conclusions Our findings highlight StyleGAN’s potential to identify both robust and shortcut patterns learnt by a cell classifier. Integrating our approach into cell prediction studies helps identify inherent classifier biases obscured by high classification scores. Visually confirming learnt patterns helps improve interpretability and trustworthiness, paving the way for broader applicability of effective deep-learning cell classifiers. Acknowledgements We would like to thank our institutions for supporting our work - BII, SIgN and IMCB (A*STAR). This work was also funded by OFYIRG23jan-0021 and I2001E0065. References Simoni Y, Becht E, Fehlings M, Loh CY, Koo SL, Teng KWW, Yeong JPS, Nahar R, Zhang T, Kared H, Duan K, Ang N, Poidinger M, Lee YY, Larbi A, Khng AJ, Tan E, Fu C, Mathew R, Teo M, Lim WT, Toh CK, Ong BH, Koh T, Hillmer AM, Takano A, Lim TKH, Tan EH, Zhai W, Tan DSW, Tan IB, Newell EW. Bystander CD8+ T cells are abundant and phenotypically distinct in human tumour infiltrates. Nature. 2018 May;557(7706):575–579. doi: 10.1038/s41586-018-0130-2. Karras T, Aittala M, Laine S, Härkönen E, Hellsten J, Lehtinen J, Aila T. Alias-free generative adversarial networks. Adv Neural Inf Process Syst. 2021;34:852–863. Ethics Approval The study was approved by the Agency for Science, Technology and Research (A*STAR) Human Biomedical Research Office (A*STAR IRB: 2021-188). Consent De-identified patient data was used in our work. Samples were collected with consent from patients.
Language: English
Topic: AI in cancer detection
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4404065330