Work

Reference-Free Evaluation of Taxonomies

Year: 2026

Type: conference-paper

Abstract: We introduce two reference-free metrics for quality evaluation of taxonomies in the absence of labels.The first metric evaluates robustness by calculating the correlation between semantic and taxonomic similarity, addressing error types not considered by existing metrics.The second uses Natural Language Inference to assess logical adequacy.Both metrics are tested on five taxonomies and are shown to correlate well with F1 against ground truth taxonomies.We further demonstrate that our metrics can predict downstream performance in hierarchical classification when used with label hierarchies.

Source: Findings of the Association for Computational Linguistics: ACL (Findings of ACL)

DOI: 10.18653/v1/2026.findings-acl.1273

Authors: Pascal Wullschleger, Majid Zarharan, Donnacha Daly, Marc Pouly, Jennifer Foster

Institutions: National University of Ireland, Maynooth, University of Lucerne, Dublin City University

Language: English

Topic: Text and Document Classification Technologies

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, Science Foundation Ireland, Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung, European Regional Development Fund

Awards: G2399253094, ADAPT_Phase 2