Year: 2025
Type: article
Abstract: Error confusion sets refer to collections of words or characters that, within the same group, are similar in spelling and grammar to the corresponding words or characters in the correct text, thereby easily causing confusion. Such confusion sets are commonly used in automatic text correction systems, where the system replaces potential errors with other candidates from the confusion set to enhance the accuracy of the correction. This paper introduces a method for constructing Tibetan syllable confusion sets aimed at text correction, and validates the confusion sets in experiments on Tibetan text correction. The Tibetan syllable confusion sets presented in this paper are composed of three distinct subsets based on the characteristics of Tibetan text: a confusion subset based on Tibetan syllable component information and edit distance, a confusion subset based on input key position distance, and a confusion subset based on Tibetan grammatical knowledge. These subsets are merged sequentially to form a comprehensive Tibetan syllable confusion set for text correction, totaling 59.2M in size. Experiments on public data with this confusion set have shown that the accuracy of the Bert+ confusion set correction method is 10.63% higher than that of a standalone Bert model, demonstrating the effectiveness of the Tibetan syl-lable confusion set in Tibetan text correction tasks.
Source: Data Intelligence
Authors: Dingguo Gao, Jinyi Li, Liyuan Zhou
Institution: Tibet University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4416330795
FWCI:
Cites:
Cited by:
Related to: