A Multimodal Transfer Learning-Based Method for Constructing Knowledge Graphs of Wireless Communication Devices
Year: 2026
Type: conference-paper
Abstract: Scarcity of annotated multimodal data limits entity recognition and relation extraction when building knowledge graphs for wireless communication equipment. Traditional textonly extraction methods cannot adapt to mixed heterogeneous data consisting of equipment structural drawings, parameter specification documents and circuit diagrams, which severely restricts the practical deployment of knowledge graph in communication operation and maintenance scenarios. This paper optimizes ViLBERT from three domain-specific perspectives: hierarchical parameter freezing based on equipment feature distribution, communication-customized lightweight low-rank adapter and equipment-manual-derived domain prompt learning. Text synonym replacement and image geometric transformation including rotation, flipping and translation are implemented for data augmentation to alleviate few-shot learning bottleneck. Apart from a self-built dataset containing 1023 image-text pairs covering eight mainstream communication devices, public MME-KG industrial multimodal dataset with 3279 samples is adopted for cross-domain generalization test. Apart from original ViLBERT and CLIP, BLIP-2 and UniVLP are introduced as state-of-the-art comparative baselines to verify model advancement. Experimental results on two datasets show that the optimized model achieves entity recognition F1 = 0.88 and relation extraction accuracy = 0.82 on self-built data, and F1 = 0.83, Acc = 0.78 on MME-KG. The proposed lightweight optimization reduces overall computational overhead by nearly 30%. The presented method realizes efficient multimodal knowledge extraction and automatic knowledge graph construction for wireless communication devices, which can provide technical support for intelligent fault diagnosis of communication equipment.
Authors: Xiaojian Zhang, Zhengxin Fu, Ji Yang, Xiaopeng Li
Institution: PLA Information Engineering University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7202373394