Year: 2025
Type: article
Abstract: ABSTRACT Lexical analysis is a fundamental task in natural language processing, which involves several subtasks, such as word segmentation (WS), part‐of‐speech (POS) tagging, and named entity recognition (NER). Recent works have shown that taking advantage of relatedness between these subtasks can be beneficial. This paper proposes a unified neural framework to address these subtasks simultaneously. Apart from the sequence tagging paradigm, the proposed method tackles the multitask lexical analysis via two‐stage sequence span classification. Firstly, the model detects the word and named entity boundaries by multi‐label classification over character spans in a sentence. Then, the authors assign POS labels and entity labels for words and named entities by multi‐class classification, respectively. Furthermore, a Gated Task Transformation (GTT) is proposed to encourage the model to share valuable features between tasks. The performance of the proposed model was evaluated on Chinese and Thai public datasets, demonstrating state‐of‐the‐art results.
DOI: 10.1049/cit2.70015
Institution: Kunming University of Science and Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,760
OpenAlex ID: W4410257934