Work

DiffusionSL: Sequence Labeling via Tag Diffusion Process

Year: 2023

Type: conference-paper

Abstract: Sequence Labeling (SL) is long-standing in Natural Language Processing (NLP).Traditionally, discriminative models have been widely used to capture the conditional distribution of sequence tags, rather than generative models.In this paper, we present DiffusionSL, a framework that utilizes a conditional discrete diffusion model for generating discrete tag data, resulting in a Tag Diffusion Process.We treat the natural language sequence as the conditional signal and the sequence tags as the generation target, iteratively refining the noisy tags to obtain clean ones.To address the discreteness issue, we propose the Bit-Tag Converter (BTConverter) to model the target in continuous data space.Furthermore, we introduce the Bit Diffusion Transformer (BitDiT) to model the process of noise elimination.Leveraging the powerful iterative refinement capability of the diffusion model, DiffusionSL achieves superior performance against previous state-ofthe-art (SOTA) baselines and outperforms gpt-3.5-turbosignificantly across multiple benchmark datasets and various tasks 1 .

Source: Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)

DOI: 10.18653/v1/2023.findings-emnlp.860

Authors: Ziyang Huang, Pengfei Cao, Jun Zhao, Kang Liu

Institutions: Chinese Academy of Sciences, Institute of Automation, University of Chinese Academy of Sciences

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Natural Science Foundation of China, Chinese Academy of Sciences, Youth Innovation Promotion Association of the Chinese Academy of Sciences, Youth Innovation Promotion Association

Award: G2682406033