Enhancing cross-encoders using knowledge graph hierarchy for medical entity linking in zero- and few-shot scenarios
Year: 2025
Type: article
Abstract: Medical Entity Linking (MEL) is a common task in natural language processing, focusing on the normalization of recognized entities from clinical texts using large knowledge bases (KBs). This task presents significant challenges, especially when working with electronic health records that often lack annotated clinical notes, even in languages like English. The difficulty increases in few-shot or zero-shot scenarios, where models must operate with minimal or no training data, a common issue when dealing with less-documented languages such as Spanish. Existing solutions that combine contrastive learning with external sources, like the Unified Medical Language System (UMLS), have shown competitive results. However, most of these methods focus on individual concepts from the KBs, ignoring relationships such as synonymy or hierarchical links between concepts. In this paper, we propose leveraging these relationships to enrich the training triplets used for contrastive learning, improving performance in MEL tasks. Specifically, we fine-tune several BERT-based cross-encoders using enriched triplets on three clinical corpora in Spanish : DisTEMIST, MedProcNER, and SympTEMIST. Our approach addresses the complexity of real-world data, where unseen mentions and concepts are frequent. The results show a notable improvement in lower top- k accuracies, surpassing the state-of-the-art by up to 5.5 percentage points for unseen mentions and by up to 5.9 points for unseen concepts. This improvement reduces the number of candidate concepts required for cross-encoders, enabling more efficient semi-automatic annotation and decreasing human effort. Additionally, our findings underscore the importance of leveraging not only the concept-level information in KBs but also the relationships between those concepts. • Our strategy significantly improves medical entity linking (MEL) performance in zero-shot and few-shot scenarios, achieving up to 5.5% and 5.9% increases in unseen mentions and concepts,respectively. • The enriched approach we propose achieves a top-25 accuracy of around 75% and even higher in the most challenging entity normalization scenarios. • The robustness and efficacy of the triplets maintain stable performance across the three main Spanish medical corpora (DisTEMIST, MedProcNER, SympTEMIST), showing consistent behavior in different medical domains. • We propose a new triplet generation strategy for training cross-encoders using both parent–child and bidirectional hierarchical relationships in knowledge graphs. • The approach enables efficient candidate re-ranking, reducing the manual effort in medical annotation tasks by improving top-k accuracies and minimizing the candidates needed for normalization.
Source: Knowledge-Based Systems
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: hybrid
Estimated APC paid: $3,430
OpenAlex ID: W4408076475