Methoden van diep leren voor de extractie van informatie uit tekst
Year: 2025
Type: dissertation
Abstract: This dissertation explores advancements in representation learning and knowledge extraction, particularly for biomedical and healthcare applications, by leveraging structured and unstructured data with and without supervision. First, a novel contrastive learning framework is presented, which encodes relational information within sentence embeddings, enhancing their capacity to reflect entity relationships for tasks such as relation extraction. The resulting relation-aware language model (LM) embeddings achieve competitive results on the relation extraction task using a simple k-nearest neighbors (KNN) classifier, thereby demonstrating the success of the proposed method. We show that we can learn a different space for named entity recognition, again using a contrastive learning objective, and demonstrate how to successfully combine both representation spaces in an end-to-end information extraction (IE) task. The influence of tokenization remains understudied in sequence tagging tasks. We examine how tokenization impacts IE from documents and offer a comparative study of subword-based and character-based models. In particular, we investigate IE from biomedical texts and perform a hyphenation analysis to identify unique linguistic characteristics. Our primary findings are twofold: tokenization patterns can incorporate an inductive bias that achieves state-of-the-art performance, and character-based models yield promising outcomes, making the shift towards token-free IE models viable. Despite advances in knowledge graphs (KGs), representing knowledge remains a non-trivial task across industries and it is especially challenging in the biomedical and healthcare domains due to complex interdependent relations between entities, heterogeneity, lack of standardization, and sparseness of data. This research addresses challenges in knowledge representation from heterogeneous biomedical data sources and investigates entity-centric KGs to better capture and utilize semantic relationships. An end-to-end representation learning framework is introduced to extract entity-centric KGs from structured and unstructured data, utilizing a star-shaped ontology to represent the multiple facets of a person and use it to guide KG creation. Compact representations of the graphs are created leveraging graph neural networks (GNNs) and experiments are conducted using different levels of heterogeneity or explicitness. An intensive care unit (ICU) readmission prediction task is used to evaluate the results of the proposed framework, showing a stable system, robust to missing data, that outperforms a range of baseline machine learning classifiers. Additionally, a systematic approach is presented to perform ablation studies on different clinical, demographic, and social data and draw insights regarding the robustness in the identification of predictive features in entity-centric KGs for the task of ICU readmission prediction. The growing volume of biomedical literature poses a considerable challenge to effective knowledge discovery. To tackle this, we present an open-source, comprehensive framework that automatically extracts and structures disease-specific information from unstructured text. To support research in disease-focused knowledge discovery, we develop two annotated datasets centered on Rett syndrome (RS) and Alzheimer's disease (AD), facilitating the identification of semantic relationships among various biomedical entities. We conduct extensive benchmarking to assess various methods for representing relations and entities, offering insights into optimal strategies for semantic relation detection and demonstrating the capabilities of LMs in knowledge discovery. Additionally, we perform probing experiments to better understand transformer models, analyzing layer representations and attention mechanisms to assess transformers' ability to capture semantic relationships within the text. To address the scarcity of annotated biomedical data and sidestep the resource-intensive task of the gold dataset creation, we focus on semantic relation identification among biomedical entities, reducing reliance on supervision and facilitating scalable knowledge discovery. We present a set of unsupervised algorithms and evaluate pointwise binary classification methods to transition from weakly supervised to fully unsupervised approaches and evaluate their capacity to learn from data with silver labels. The evaluation on four biomedical benchmark datasets explores the effectiveness of the methods, demonstrating their potential to enable scalable knowledge discovery systems less reliant on annotated datasets.
Source: Lirias (KU Leuven)
Author: Theodoropoulos, Christos
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7112792819
Cites:
Cited by:
Related to: