MMHFN: a multimodal deep learning framework for intelligent classification and management of government documents
Year: 2025
Type: article
Abstract: Government document intelligence faces significant challenges from dense text, pervasive visual noise (e.g., stamps, low-resolution scans), and high OCR error rates (>25%), hindering automated classification in e-governance. To address this, we propose MMHFN: a multimodal deep learning framework integrating: 1) a Spatial Attention-enhanced MobileNetV3 for noise-robust visual feature extraction; 2) a dual-path text encoder (FastText subword embeddings + domain-adapted BERT) with gated fusion to mitigate OCR errors; 3) lightweight differentiable optimal transport for cross-modal alignment; 4) a Seq2Seq OCR-correction module. Experimental results on RVL-CDIP, Tobacco3482, and GOV-DOCBench datasets show MMHFN achieves 92.7% accuracy (+6.2% over unimodal baselines) and 90.3% F1-score, with only 3.1% accuracy drop under severe OCR noise and real-time edge deployment (190 ms/page). This work contributes an efficient, domain-adapted solution for intelligent document management, publicly releasing the 12,000-document GOV-DOCBench benchmark.
Authors: Qizhong Luo, Tiantian Huang, Xianli Zeng
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4414437408