Year: 2024
Type: dissertation
Abstract: The emergence of Pre-trained Language Models (PLMs) has revolutionized natural language processing, yielding remarkable improvements across diverse tasks. While PLMs excel at discerning language patterns and crafting coherent narratives, they face challenges when adapting to specialized topics, such as medical concepts, cross-lingual conversational systems, and content from diverse sociodemographic backgrounds. These challenges, collectively referred to as the "adaptation barrier," stem from the difficulty of tailoring PLMs to fields requiring expert knowledge, often due to insufficient topic-relevant information within the model’s training data. Recent research has explored techniques like domain adaptation, cross-lingual transfer learning, and the integration of external knowledge to address these limitations. However, existing approaches often focus narrowly on either domain-specific, language-specific, or social dimensions, limiting a comprehensive understanding of the effectiveness of proposed adaptation approaches. In this thesis, we systematically investigate the adaptation barrier by conducting experiments across multiple dimensions. We focus on three key challenges: effectiveness, efficiency, and interpretability in adapting PLMs to diverse domains, languages, and social dimensions. Through our findings, we hope to contribute to the development of more adaptable language models across multiple dimensions, enhancing their applicability in addressing real-world challenges, thereby mitigating the adaptation barrier inherent to PLMs.
Source: MADOC (University of Mannheim)
Author: Hung, Chia-Chien
Institution: University of Mannheim
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6993040765
