Work

Evaluating Column Type Annotation Models and Benchmarks

Year: 2025

Type: conference-paper

Abstract: Column Type Annotation (CTA) is the process of assigning semantic types to the columns of a table. Models and approaches for CTA are typically evaluated on distinct datasets, making benchmarking and comparisons not trivial. In this paper, we identify major issues in commonly-used datasets for CTA and explore their impact on the evaluation process and reported results. Drawing from our analysis, we propose changes towards more robust and unbiased benchmarking practices.

DOI: 10.1145/3701716.3715530

Authors: Amir Babamahmoudi, Davood Rafiei, Mario A. Nascimento

Institution: University of Alberta

Language: English

Topic: Semantic Web and Ontologies

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Funders: Universitas Brawijaya, Natural Sciences and Engineering Research Council of Canada

OpenAlex ID: W4410638111

FWCI: 1.059

Cites: 14

Cited by: 1

Related to: 10