Work

Consistent and efficient long document understanding

Year: 2023
Type: article
Abstract: In the age of information overload, people's information needs from long documents are rapidly emerging, while people's patience for careful reading and reasoning is gradually vanishing.While people are inundated with large amounts of long textual documents covering topics in various domains, such as news, healthcare, legal service, and finance, they struggle to gain quick, concise, and accurate insights from these long and tedious documents.The development of automatic document understanding systems promises the possibility of assisting humans in gaining insights from those long documents.Automatic systems capture and analyze the information contained in a collection of news and scientific reports in a concise and machine-understandable way.Automatic systems parse unstructured text by identifying the relations between events and entities from long complex reading for structured data usage.Automatic systems provide reliable digests by factually and consistently summarizing recent papers, reports, news, and reviews.However, automatically understanding long documents remains a challenge because recent state-of-the-art document understanding systems are mostly built upon transformer structures and are mostly motivated, designed, implemented, and evaluated under the short-input setting.To adapt those short-input systems to long sequences, documents have to be truncated, chunked using a sliding window, or processed in parallel on multiple machines.These additional operations usually cause the loss of long-range interdependency and introduce additional costs.Therefore, this thesis focuses on developing principled and scalable methods for more consistent and efficient long document understanding.In particular, we investigate four research problems from the perspectives of consistency and efficiency:1) Consistent Meta-review Generation.Current work on Opinion Summarization extracts and selects representing opinions on aspects of interest under the assumption that input opinions are non-controversial.Opinions in the scientific domain can be divergent, leading to controversy or consensus among reviewers, while the scientific meta-review should be consistent with the synthesized opinions from individual reviews.Therefore, we propose to benchmark scientific opinion summarization by collecting paper meta-reviews from Open-Review, proposing a Checklist-guided Iterative Introspection approach, and constructing a comprehensive evaluation framework.2) Consistent Document Summarization.Current abstractive summarization models often generate inconsistent content, i.e. texts that are not directly inferable from the source document, are not consistent with respect to world knowledge, or are self-contradictory.
Author: Zeng, Qi
Language: English
Open Access status: green
OpenAlex ID: W7220823991
FWCI: 0.459
Cites: 169
Cited by: 8
Related to: 10