Work

Modeling Orthographic Variation in Occitan’s Dialects

Year: 2024
Type: conference-paper
Abstract: Effectively normalizing textual data poses a considerable challenge, especially for lowresource languages lacking standardized writing systems.In this study, we fine-tuned a multilingual model with data from several Occitan dialects and conducted a series of experiments to assess the model's representations of these dialects.For evaluation purposes, we compiled a parallel lexicon encompassing four Occitan dialects.Intrinsic evaluations of the model's embeddings revealed that surface similarity between the dialects strengthened representations.When the model was further fine-tuned for partof-speech tagging and Universal Dependency parsing, its performance was robust to dialectical variation, even when trained solely on part-of-speech data from a single dialect.Our findings suggest that large multilingual models minimize the need for spelling normalization during pre-processing.
Language: English
Open Access status: hybrid
OpenAlex ID: W4401042004
FWCI: 0.347
Cites: 33
Cited by: 4
Related to: 10