Work

A Method for Discovering Ancient Chinese Neologisms Based on the Combination of Information Entropy - Mutual Information and BI-LSTM-CRF Model

Year: 2025

Type: conference-paper

Abstract: Traditional new word discovery often relies on the information entropy-mutual information (IM) algorithm. In this study, we propose a joint IM algorithm and bidirectional long short-term memory conditional random field (BI-LSTM-CRF) neural network model, referred to as IM-BI-LSTM-CRF. This model retains the advantages of the IM algorithm while leveraging the BI-LSTM model to capture contextual dependencies and the CRF layer to model label dependencies. Using this hybrid approach, we conducted new word discovery experiments on a self-annotated dataset, utilizing word embedding vectors and different word position annotation sets. The results demonstrate that the IM-BI-LSTM-CRF model surpasses traditional LSTM and BI-LSTM models in new word discovery and exhibits strong generalization capabilities.

Source: International Conference on Generative Artificial Intelligence and Information Security (GAIIS)

DOI: 10.1145/3728725.3728745

Author: Lin Fu

Institution: Xihua University

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W4411016350

FWCI:

Cites: 3

Cited by:

Related to: 10