Evaluating Semantic Resources in the age of Generative AI
Year: 2026
Type: article
Abstract: Ensuring the quality of semantic resources such as ontologies and knowledge graphs remains a central challenge in knowledge engineering, particularly as knowledge-driven systems are increasingly deployed in high-stakes domains. Errors, inconsistencies, and embedded biases in these resources can propagate through downstream applications, leading to significant technical, social, and ethical consequences. Historically, the field has navigated a challenging trade-off between automatic evaluation approaches, which offer high scalability, and human-centric methods, which are necessary for capturing errors that require deep contextual understanding. This keynote explores how the emergence of Generative Artificial Intelligence (GenAI) provides a transformative solution to this dilemma by supporting and scaling human-centric evaluations. We will exam the evolving landscape of GenAI supported evaluation methods, with a specific emphasis on the complexity of knowledge these systems are capable of assessing. Furthermore, we will present frameworks for the optimal combination of GenAI and human contributors, highlighting how this synergy can enhance the scalability of semantic quality assurance in next-generation knowledge-driven systems.
Author: Sabou, Marta
Institution: Vienna University of Economics and Business
Language: English
Topic: Semantic Web and Ontologies
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Austrian Science Fund
OpenAlex ID: W7225378502