Work

MSF-Net: A Data-Driven Multimodal Transformer for Intelligent Behavior Recognition and Financial Risk Reasoning in Virtual Live-Streaming

Year: 2025

Type: article

Abstract: With the rapid advancement of virtual human technology and live-streaming e-commerce, virtual anchors have increasingly become key interactive entities in the digital economy. However, emerging issues such as fake reviews, abnormal tipping, and illegal transactions pose significant threats to platform financial security and user privacy. To address these challenges, a multimodal emotion–finance fusion security recognition framework (MSF-Net) is proposed, which integrates visual, audio, textual, and financial transaction signals to achieve cross-modal feature alignment and multi-signal risk modeling. The framework consists of three core modules: the multimodal alignment transformer (MAT), the fake review detection (FRD) module, and the multi-signal fusion decision module (MSFDM), enabling deep integration of semantic consistency modeling and emotion–behavior collaborative recognition. Experimental results demonstrate that MSF-Net achieves superior performance in virtual live-streaming financial security detection, reaching a precision of 0.932, a recall of 0.924, an F1-score of 0.928, an accuracy of 0.931, and an area under curve (AUC) of 0.956, while maintaining a real-time inference speed of 60.7 FPS, indicating outstanding precision and responsiveness. The ablation experiments further verify the necessity of each module, as the removal of any component leads to an F1-score decrease exceeding 4%, confirming the structural validity of the model’s hierarchical fusion design. In addition, a lightweight version of MSF-Net was developed through parameter distillation and quantization pruning techniques, achieving real-time deployment on mobile devices with an average latency of only 19.4 milliseconds while maintaining an F1-score of 0.923 and an AUC of 0.947. The results indicate that MSF-Net exhibits both innovation and practicality in multimodal deep fusion and security risk recognition, offering a scalable solution for intelligent risk control in data-driven artificial intelligence applications across financial and virtual interaction domains.

Source: Electronics

DOI: 10.3390/electronics14234769