Work

SsciBERT: A Pre-trained Language Model for Social Science Texts

Year: 2022

Type: preprint

Abstract: The academic literature of social sciences records human civilization and studies human social problems. With its large-scale growth, the ways to quickly find existing research on relevant issues have become an urgent demand for researchers. Previous studies, such as SciBERT, have shown that pre-training using domain-specific texts can improve the performance of natural language processing tasks. However, the pre-trained language model for social sciences is not available so far. In light of this, the present research proposes a pre-trained model based on the abstracts published in the Social Science Citation Index (SSCI) journals. The models, which are available on GitHub (https://github.com/S-T-Full-Text-Knowledge-Mining/SSCI-BERT), show excellent performance on discipline classification, abstract structure-function recognition, and named entity recognition tasks with the social sciences literature.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2206.04510

Authors: Shen, Si, Liu, Jiangfeng, Lin, Litao, Huang, Ying, Zhang, Lin, Liu, Chang, Feng, Yutong, Wang, Dongbo

Institutions: Nanjing Agricultural University, Wuhan University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Natural Science Foundation of China, Nanjing University of Science and Technology, Nanjing University, Nanjing Agricultural University

Awards: 基于深度学习的学术全文本知识图谱构建及检索研究, G6674815636

OpenAlex ID: W4320169933

Cites:

Cited by:

Related to: 10