Get started
Work

Approches neuronales de l'extraction automatique de terminologie

Year: 2024
Type: dissertation
Abstract: Automatic term extraction (ATE) is a natural language processing (NLP) task that identifies specialized terminology from the domain-specific corpora. By easing the time and effort needed to extract the terms manually, ATE is not only widely used for terminographical tasks (e.g., specialized dictionary creation) but also contributes to several complex downstream tasks (e.g., machine translation, and information retrieval). Over the past forty years, much progress has been made in automatically providing a ranked list of the candidate terms from specialized corpora; however, ATE tasks remain a notoriously challenging problem. Our dissertation focused on the following aspects. Regarding scenarios where the well-annotated data is adequate for fully supervised settings, we investigated the improvement of the neural approaches by introducing the task as a token classification (so-called sequence labeling) problem using Transformers as the base model with additional representation (e.g., label semantics) and modified layers (e.g., Mixture of Expert or MoE, RNN). Furthermore, we elaborated on the current systems by introducing NOBI, a novel annotation regime to better capture the nested terms. Regarding scenarios where the well-annotated data from the same languages is limited but those from other languages are adequate for fully supervised settings, we proposed cross-lingual and multilingual learning to highlight the potential of transfer learning from rich-resourced languages or the combined ones to lesser-known languages in the neural systems. Regarding scenarios where the well-annotated data and computation resources are both limited, we proposed a novel pipeline that used large language models (LLMs) called LlamATE as a predictor to query the candidate terms without any additional fine-tuning steps, just by using few-shot demonstration with self-verification steps. Our study concluded with the following points. Firstly, token classification approaches (e.g., XLMR) are valid and promising methods for full-supervised learning in term extraction, which surpassed the performance of the non-sequential and binary sequence classifiers, and reduced computational and storage challenges mentioned in the benchmarks. Incorporating an MoE layer atop the deep neural model (e.g., (m)DeBERTA) consistently improved performance compared to the baseline with a dense token classification head. Using NOBI annotation regimes demonstrated a visible improvement in both single- and multi-word terms for the token classifier trained on the dataset in which the number of nested terms is significant enough. Secondly, with limited annotated data from the same languages, our results on the token classifier demonstrated the promising impact of multilingual and cross-lingual cross-domain learning when transferring from the rich- to lesser-known languages. Lastly, our pipeline named LlamATE suggested the potential of LLMs with few-shot demonstration and self-verification in learning from a few examples in the same domain even without being told the domain explicitly as well as the potential in the transfer knowledge from well-covered languages (English) to less-represented languages (French, Dutch) in LLMs. While they might not be a replacement for fully-supervised models, they can enhance efficiency and accuracy by streamlining the pre-annotation process and accelerating manual annotation efforts.
Language: English
Open Access status: green
OpenAlex ID: W7035886709
Cites:
Cited by:
Related to: