Work

ProtoMedAgent : Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows

Year: 2026

Type: book-chapter

Abstract: While interpretable prototype networks offer compelling case-based reasoning for clinical diagnostics, their raw continuous outputs lack the semantic structure required for medical documentation.Bridging this gap via standard Retrieval-Augmented Generation (RAG) routinely triggers "retrieval sycophancy," where Large Language Models (LLMs) hallucinate post-hoc rationalizations to align with visual predictions.We introduce ProtoMedAgent, a framework that formalizes multimodal clinical reporting as an iterative, zero-gradient test-time optimization problem over a strict neuro-symbolic bottleneck.Operating on a frozen prototype backbone, we distill latent visual and tabular features into a discrete semantic memory.Online generation is strictly constrained by exact set-theoretic differentials and a reflective Scribe-Critic loop, mathematically precluding unsupported narrative claims.To safely bound data disclosure, we introduce a semantic privacy gate governed by k-anonymity and ℓ-diversity.Evaluated on a 4,160-patient clinical cohort, ProtoMedAgent achieves 91.2%Comparison Set Faithfulness where it fundamentally outperforms standard RAG (46.2%).ProtoMedAgent additionally leverages a binding ℓ-diversity phase transition to systematically reduce artifact-level membership inference risks by an absolute 9.8%.

Source: Lancaster EPrints (Lancaster University)