Multi-step Transfer Learning in Natural Language Processing for the Health Domain
Year: 2024
Type: article
Abstract: As a result of the restricted access to data in healthcare facilities due to patient privacy and confidentiality policies, the application of general natural language processing (NLP) techniques has advanced relatively slowly in the health domain. Additionally, because clinical data is unique to various institutions and laboratories, there aren't enough standards and standards for data annotation. In places without robust death registration systems, the cause of death (COD) is determined through a verbal autopsy (VA) report. A non-clinician field agent completes a VA report using a set of predefined questions as a guidance in order to identify the symptoms of a COD. The narrative text of the VA report is used as a case study to examine the difficulties of applying NLP techniques to the health care domain.Motivated by multi-step learning where a final learning task is realized via a sequence of intermediate learning tasks, this paper presents a framework to leverage knowledge across the two domain adaptation settings of feature extraction and fine-tuning to improve VA narrative representations for COD classification task in the health domain. We use the Bidirectional Encoder Representations from Transformers (BERT) and Embeddings from Language Models (ELMo) models pretrained on the general English and health domains to extract features from the VA narratives. Our results demonstrate increased performance when the two embeddings from the English and health domains—learned through the domain adaptation tasks of feature extraction and fine-tuning are combined. The advantage of using character embeddings in conjunction with word embeddings is also apparent.
Source: Neural Processing Letters
Institutions: University of the Witwatersrand, University of Johannesburg
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,990
OpenAlex ID: W4398141723