Year: 2025
Type: article
Abstract: The role of image data in knowledge extraction and representation has become increasingly significant. This study introduces a novel methodology, termed Image to Graph via Large Language Model (ImgGraph-LLM), which constructs a knowledge graph for each image in a dataset. Unlike existing methods that rely on text descriptions or multimodal data to build a comprehensive knowledge graph, our approach focuses solely on unlabeled individual image data, representing a distinct form of unsupervised knowledge graph construction. To tackle the challenge of generating a knowledge graph from individual images in an unsupervised manner, we first design two self-supervised operations to generate training data from unlabeled images. We then propose an iterative fine-tuning process that uses this self-supervised information, enabling the fine-tuned LLM to recognize the triplets needed to construct the knowledge graph. To improve the accuracy of triplet extraction, we introduce filtering strategies that effectively remove low-confidence training data. Finally, experiments on two large-scale real-world datasets demonstrate the superiority of our proposed model.
Source: Information
DOI: 10.3390/info16050335
Authors: Lei Chen, Zhenyu Chen, Wei Yang, Shi Liu, Yong Li
Institutions: Tsinghua University, State Grid Corporation of China (China)
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,930
Funder: State Grid Corporation of China
Award: G274263082
OpenAlex ID: W4409682143