SentiLR: Linguistic Knowledge Enhanced Language Representation for Sentiment Analysis.
Year: 2019
Type: preprint
Abstract: Most of the existing pre-trained language representation models neglect toconsider the linguistic knowledge of texts, whereas we argue that such
knowledge can promote language understanding in various NLP tasks. In this
paper, we propose a novel language representation model called SentiLR, which
introduces word-level linguistic knowledge including part-of-speech tag and
prior sentiment polarity from SentiWordNet to benefit the downstream tasks in
sentiment analysis. During pre-training, we first acquire the prior sentiment
polarity of each word by querying the SentiWordNet dictionary with its
part-of-speech tag. Then, we devise a new pre-training task called label-aware
masked language model (LA-MLM) consisting of two subtasks: 1) word knowledge
recovering given the sentence-level label; 2) sentence-level label prediction
with linguistic knowledge enhanced context. Experiments show that SentiLR
achieves state-of-the-art performance on several sentence-level / aspect-level
sentiment analysis tasks by fine-tuning, and also obtain comparative results on
general language understanding tasks.
Source: arXiv (Cornell University)
Authors: Pei Ke, Haozhe Ji, Siyang Liu, Xiaoyan Zhu, Minlie Huang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2983772459