Year: 2023
Type: dissertation
Abstract: A character sequence tends to comprise segmentation alternatives, leading to segmentation ambiguity.Properly handling this ambiguity using multi-granularity linguistic units, such as character clusters, subwords, and words, can improve word segmentation performance and lessen ambiguous boundary decisions.We conduct a study to investigate the potential of using various linguistic units and leveraging segmentation alternatives for character-based word segmentation.Our experimental results demonstrated improvements in segmentation performance, outperforming previous work on the BCCWJ, CTB6, and BEST2010 datasets in Japanese, Chinese, and Thai, respectively.
Authors: CHAY-INTR, Thodsaporn, Chay-intr, Thodsaporn
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7047850498