A hybrid deep learning framework with domain-adapted embeddings for regulatory violation classification
Year: 2026
Type: article
Abstract: Regulatory violation classification encounters fundamental challenges arising from the lack of domain-specific knowledge in general-purpose language models, insufficient multi-granular feature representation in single-architecture frameworks, and severe class imbalance characteristic of real-world regulatory datasets. This study introduces a hybrid deep learning framework integrating three synergistic innovations to address these limitations through domain-adapted embedding fusion, hierarchical feature extraction, and class-aware contrastive learning mechanisms. The domain-adapted dual-layer embedding fusion (DA-DLEF) dynamically combines general linguistic knowledge, specialized legal understanding, and regulatory-specific terminology through attention-based integration, while the Hierarchical Hybrid Feature Extraction Network (H2-FEN) captures discriminative patterns across local n-gram sequences, bidirectional temporal dependencies, and global contextual relationships through multi-scale convolutional processing, recurrent modeling, and regulatory-biased self-attention mechanisms. The class-aware contrastive fusion classifier (CA-CFC) optimizes inter-class separability and intra-class compactness through supervised contrastive learning combined with focal loss and multi-expert fusion strategies. Comprehensive evaluation on a constructed dataset of Chinese administrative penalty decisions demonstrates substantial improvements over state-of-the-art baselines, particularly for minority violation categories, validating the effectiveness of domain adaptation, hierarchical architecture design, and contrastive learning in enhancing classification performance. The framework provides practical solutions for governmental regulatory agencies requiring automated violation categorization with enhanced accuracy and equitable treatment of underrepresented categories.
Source: Scientific Reports
Author: Shuai Li
Institution: Beijing Foreign Studies University
Language: English
Subfield: Law
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7225798090