Work

T5-Based Model for Abstractive Summarization: A Semi-Supervised Learning Approach with Consistency Loss Functions

Year: 2023

Type: article

Abstract: Text summarization is a prominent task in natural language processing (NLP) that condenses lengthy texts into concise summaries. Despite the success of existing supervised models, they often rely on datasets of well-constructed text pairs, which can be insufficient for languages with limited annotated data, such as Chinese. To address this issue, we propose a semi-supervised learning method for text summarization. Our method is inspired by the cycle-consistent adversarial network (CycleGAN) and considers text summarization as a style transfer task. The model is trained by using a similar procedure and loss function to those of CycleGAN and learns to transfer the style of a document to its summary and vice versa. Our method can be applied to multiple languages, but this paper focuses on its performance on Chinese documents. We trained a T5-based model and evaluated it on two datasets, CSL and LCSTS, and the results demonstrate the effectiveness of the proposed method.

Source: Applied Sciences

DOI: 10.3390/app13127111

Authors: Mingye Wang, Pan Xie, Yao Du, Xiaohui Hu

Institutions: Beihang University, Chinese Academy of Sciences, Institute of Software

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,558

Funder: Ministry of Science and Technology of the People's Republic of China