OBI-BERT: A Pre-trained Language Representation Model for Ancient Script Character Prediction
Year: 2024
Type: conference-paper
Abstract: This paper presents a computer-aided method for interpreting Oracle Bone Inscriptions (OBI) utilizing a domain-specific BERT pre-training model. To tackle critical challenges in current OBI studies, such as incomplete character recognition and ambiguous character identification, we introduced OBI-BERT, a tailored pre-trained language model. This model employs a deep-shallow feature fusion strategy, significantly improving its understanding of OBI texts by initializing the embedding layer with pre-trained OBI CBOW and optimizing learning rates differentially. During training, we implemented a dynamic random masking strategy (with a mask rate of 20%) and utilized a six-layer encoder structure to achieve advanced semantic modeling of OBI texts. The experimental results indicate that our method attains an accuracy of 66.26% in OBI character prediction tasks, showcasing not only precise recognition of common OBI characters but also robust performance in managing incomplete and ambiguous characters. This approach offers a novel technical pathway for OBI research, potentially overcoming the limitations of traditional manual interpretation and expediting the compilation and study of historical documents from the Shang Dynasty.
Authors: Xiaojie Wang, Jing Xiong, Zhaoan Dong, Zhibin Liu, Yang Song
Institution: Qufu Normal University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4413080378