Evaluation of Effectiveness of Large Language Models in Ontology and Knowledge Graph Creation
Year: 2025
Type: article
Abstract: This study investigates the effectiveness of selected Large Language Models (LLMs), namely ChatGPT 3.5, Semantic Web Senior (GPT-4), and Perplexity AI, in automating the creation of ontologies and knowledge graphs from unstructured, paragraph-style text using prompt engineering techniques. The research follows a five-phase methodology, including identifying suitable LLMs, developing tailored prompts for interaction, and assessing their performance based on specific parameters. The findings reveal that Perplexity AI outperforms other LLMs regarding comprehensiveness, query handling, and defining data properties. While ChatGPT models demonstrate the ability to generate ontologies, they exhibit limitations in defining subclass relationships and managing domain and range specifications of properties. The study emphasises the critical role of prompt engineering in optimising the capabilities of LLMs for ontology and knowledge graph creation. It offers a structured evaluation methodology, shedding light on the strengths and weaknesses of each LLM. By leveraging prompt engineering, this research uniquely illustrates how systematically chosen LLMs can facilitate the development of ontologies and knowledge graphs from unstructured text and identifies the most effective models through detailed performance analysis.
Authors: Biswanath Dutta, Debanjali Bain, Nabina Khamaru
Institution: Indian Statistical Institute, Bangalore Centre
Language: English
Topic: Semantic Web and Ontologies
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4410096736