Towards image-based node embedding: A novel approach for enriching multimodal representation
Year: 2022
Type: dissertation
Abstract: While there have been advances in Natural Language Processing (NLP), their success is mainly gained by applying a self-attention mechanism into single or multi-modalities.While this approach has brought significant improvements in multiple downstream tasks, it fails to capture the interaction between different entities.Therefore, we propose MM-GATBT, a multimodal graph representation learning model that captures not only the relational semantics within one modality but also the interactions between different modalities.Specifically, the proposed method constructs image-based node embedding which contains relational semantics of entities.Our empirical results show that MM-GATBT achieves state-of-the-art results among all published papers on the MM-IMDb dataset.I would like to express my best gratitude to my advisor Prof. Payam Delgosha for providing me with an invaluable opportunity to initiate this project under his support and guidance.Without his proper advice, I would not have been able to learn and grow to the extent to present this work.Because this research work covers a wide range of subtopics, I also would like to thank Prof.
Author: Seo, Seung Byum
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7220773182