An ontology-guided, language model-assisted approach for theme-specific information extraction
Year: 2025
Type: dissertation
Abstract: In an era of information explosion, text data are generated every second so fast that digesting raw texts becomes increasingly time-consuming.The research field of text mining aims at automatically extracting structured knowledge from massive unstructured text data.An essential step in the process of converting text to knowledge is information extraction (IE), which aims at extracting information about entities and relations from text.In recent years, the development of large language models (LLMs) has shed light on new possibilities of performing better IE under label-sparse settings that are hard, if not impossible, for traditional methods due to the lack of human supervision that is necessary for model training.On the other hand, LLM-assisted domain-specific IE applications often face long-tail challenges, which refer to suboptimal performance due to the fact that many important signals (e.g., entities and relations) in the target domain are sparse in the pre-training corpora and are thus not handled well by pre-trained LLMs.Such long-tail problems become more severe when we move one step further from domains into highly specialized themes.My dissertation research explores the utilization of theme-specific entity type ontologies to improve the long-tail robustness of LLMs for IE tasks.I utilize such ontologies to 1) retrieve theme-relevant documents from large corpora based on the distribution of query entities and relations on the ontology, 2) find entities from large corpora that have fine-grained entity types relevant to the query entities, 3) retrieve relevant external, non-parametric knowledge in a structured manner to augment LLM queries, and 4) organize the internal, parametric knowledge of LLMs into relation structures for robust open relation extraction.While the 1st and 3rd components focus more on information retrieval (IR), all the components together form an end-to-end framework for corpus-level theme-specific IE.Extensive experiments demonstrate that my solutions achieve state-of-the-art performance with remarkable long-tail robustness.One more notable advantage of my approach is that it works purely at the LLM inference stage.No extra pre-training or post-training is required.This means my approach works without additional annotation efforts or heavy resource consumption.All my methods are easy to deploy on a single graphics processing unit (GPU) with any recent LLMs, and thus, my solutions are highly accessible to ordinary people.In addition, my design essentially separates data from models, which ensures there is technically zero risk of leaking private user data.Thus, my approach is highly suitable for sensitive domain applications such as medical care and legal advising.First and foremost, I would like to express my deepest gratitude to my advisor, Professor Jiawei Han, for his continued guidance and unwavering support throughout my journey at the University of Illinois at Urbana-Champaign (UIUC).It has been a long and challenging path, and I cannot imagine reaching my current accomplishments without his steadfast encouragement and mentorship.I still vividly remember the first project that brought Jiawei and me together: we won a challenge on prostate cancer as part of our effort to build a knowledge engine for genomics.Even though I was very junior and not yet his student then, he felt comfortable letting me lead a team that included four of his talented students, two of whom became successful faculty members later.This collaboration with Jiawei and his students went very well, which significantly boosted my confidence in research and fostered my interest in later joining his group.Jiawei's passion for research, dedication to education, and exceptional accessibility have always inspired me to aim higher.During my Ph.D. years, he has always been practicing high standards for research, meeting me and other students every week, making himself available almost every day, staying up late for paper deadlines, and replying to emails within hours.It is amazing that Jiawei consistently invests so much time in me and every other student while keeping a considerable lab size.Despite being very senior and successful, Jiawei remains so enthusiastic and energetic that he sets a career and life role model for me and many others.Pursuing a Ph.D. degree and subsequent career opportunities as an international student in the United States is never easy.It is Jiawei's steadfast support that has helped me persevere through difficult times.One of my darkest moments happened right prior to my planned graduation when my job offer was unexpectedly rescinded due to changes in international policies.This was shocking because Jiawei and I had years of smooth and fruitful research collaboration with that company, and all my colleagues there believed my graduation would bring me right into their team.Feeling extremely upset and lost, I turned to Jiawei for advice.He met me almost immediately, comforted me that it was not my fault, told me stories about more stringent international policies when he was young, shared his thoughts on how to deal with such complications, advised me on possible research efforts I could consider to make both my dissertation and future job interviews more impressive, and ensured me continued support if I need to extend my Ph.D. program.I was deeply touched and managed to pull myself together quickly.After spending one more year in school, I published three more
Author: Xiao, Jinfeng
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7220817269