TAFSIL: Taxonomy Adaptable Fine-grained Entity Recognition through Distant Supervision for Indian Languages
Year: 2025
Type: conference-paper
Abstract: Several studies have used distant supervision to create resources for fine-grained entity recognition (FgER) to mitigate the challenges of manual annotation.However, most of these methods are primarily developed for English and cannot be efficiently adapted to many other languages, including Indian languages.Moreover, the emergence of new and unseen entity types deteriorates the performance of the supervised models trained on FgER datasets with different predefined sets of entity types.This work introduces TAFSIL, a taxonomy-adaptable FgER framework to create FgER datasets in six Indian languages.The chosen languages are spoken by more than a billion speakers across various countries.TAFSIL utilizes the high interlink between the knowledge base WikiData and linked corpora Wikipedia through multi-stage heuristics and improves annotation through fuzzy match and quality sentence selection.TAFSIL enables us to create datasets of a total size of around three million samples for six languages Hindi (Hi), Marathi (Mr), Sanskrit (Sa), Tamil (Ta), Telugu (Te), and Urdu (Ur) belonging to two language families Indo-European and Dravidian.We evaluate the robustness of TAFSIL by creating various datasets in four taxonomies FIGER, OntoNotes, HAnDS, and MultiCoNER2.Our extensive experiments suggest the sound quality of the datasets as there is a relative improvement of 83% in average F1 score over zero-shot performance across various FgER state-of-the-art models.The resource is publicly available at https://huggingface.co/datasets/prachuryyaI- ITG/TAFSIL.
Authors: Prachuryya Kaushik, Shivansh Mishra, Ashish Anand
Institution: Indian Institute of Technology Guwahati
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4412377990