Work

Structure-enhanced text mining for science

Year: 2024
Type: dissertation
Abstract: Language models pre-trained on large scientific text corpora (e.g., research papers, electronic health records, and encyclopedia articles) have achieved remarkable success in many scientific text mining tasks.Meanwhile, text is usually accompanied by various types of structural signals, such as paper metadata, concept ontologies, and citation networks, that can potentially benefit scientific literature understanding.To enhance the effectiveness of scientific text mining methods, my doctoral research focuses on teaching language models to exploit structural information for fundamental and advanced domain-specific applications in science, with an emphasis on understanding and augmenting scientific discovery.This thesis summarizes my endeavors spanning the following three key subtopics:• Structure-Aware Fine-Grained Scientific Paper Classification (Chapter 2).Automatically indexing scientific papers in a multi-dimensional and multi-granularity topic space not only facilitates flexible bibliographic exploration and analysis but also benefits a wide range of scientific applications.I have developed effective and efficient approaches that perform multi-label paper classification in an extremely large and multi-faceted label space (e.g., with 10,000-100,000 fields-of-study) by utilizing paper metadata [272], label taxonomy [276], heterogeneous information networks [277], and structures within full-text articles [271].• Structure-Aware Scientific Topic Discovery (Chapter 3).Mining textual and structural topic-indicative signals (e.g., keywords, entities, metadata, and their combinations) has crucial applications in scientific discovery, such as finding biomarker proteins for different disease categories.I have proposed performant approaches for extracting labelindicative keywords [274] and structural information [270], which enables the retrieval and synthesis of pseudo-labeled training samples and significantly enriches supervision in zero-shot and weakly supervised text mining.• Structure-Enhanced Language Model Pre-training for Scientific Applications (Chapter 4).Scientific pre-trained language models can work alongside humans throughout the scientific discovery process by detecting and explaining relevant literature, unveiling knowledge structures, and evaluating research outcomes.I have systematically explored how to integrate fundamental scientific text mining tasks (e.g., paper classification, citation prediction, and literature retrieval) into language model pre-training to facilitate sophisticated applications, such as patient-to-article matching [269] and peer review assignment [275].First and foremost, I would like to express my deepest gratitude to my advisor, Professor Jiawei Han, for his steadfast support, guidance, and encouragement throughout my doctoral journey.I still remember when I was a senior undergraduate student and submitted my first paper to the WWW conference, my submission got unanimously negative scores.Now, I graduate with seven WWW papers published under Professor Han's mentorship.This transformation process is absolutely impossible without his continual and immense support.Professor Han has always encouraged me to pursue compelling research questions, tackle real-world challenges, and keep the broader vision in mind.As I delve deeper into research, I come to realize just how much support he has provided at every step of the way, from guiding me through paper writing to polishing my conference presentations to helping me connect with other researchers.Professor Han has always been there whenever I would like to discuss research ideas or seek guidance on my career.I vividly recall that during my job interview at Texas A&M University, one interview question was to describe my role model, and I spoke about Professor Han without any hesitation.His passion for research, dedication to high academic standards, and kindness toward young scholars have shown me what it means to be an exemplary researcher.I am so proud to introduce myself as a student of one of the most renowned researchers in the field.I would like to extend my heartfelt gratitude to my esteemed thesis committee members:
Author: Zhang, Yu
Language: English
Open Access status: green
OpenAlex ID: W7220761354
Cites: 259
Cited by: 1
Related to: 10