Work

MMCOMET: A Large-Scale Multimodal Commonsense Knowledge Graph for Contextual Reasoning

Year: 2026

Type: preprint

Abstract: We present MMCOMET, the first multimodal commonsense knowledge graph (MMKG) that integrates physical, social, and eventive knowledge. MMCOMET extends the ATOMIC2020 knowledge graph to include a visual dimension, through an efficient image retrieval process, resulting in over 900K multimodal triples. This new resource addresses a major limitation of existing MMKGs in supporting complex reasoning tasks like image captioning and storytelling. Through a standard visual storytelling experiment, we show that our holistic approach enables the generation of richer, coherent, and contextually grounded stories than those produced using text-only knowledge. This resource establishes a new foundation for multimodal commonsense reasoning and narrative generation.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2603.01055

Authors: Wang, Eileen, Arnaout, Hiba, Pratama, Dhita, Yang, Shuo, Liu, Dangyang, Yang, Jie, Poon, Josiah, Pan, Jeff, Han, Caren

Institutions: The University of Sydney, Technische Universität Darmstadt, The University of Melbourne, University of Edinburgh

Topic: Multimodal Machine Learning Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7133336712

Cites:

Cited by:

Related to: