MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification
Year: 2026
Type: conference-paper
Abstract: Tabular data forms the backbone of high-stakes decision systems in finance, healthcare, and beyond.Yet industrial tabular datasets are inherently difficult: high-dimensional, riddled with missing entries, and rarely labeled at scale.While foundation models have revolutionized vision and language, tabular learning still leans on handcrafted features and lacks a general self-supervised framework.We present MaskTab, a unified pre-training framework designed specifically for industrialscale tabular data.MaskTab encodes missing values via dedicated learnable tokens, enabling the model to distinguish structural absence from random dropout.It jointly optimizes a hybrid supervised pre-training scheme-utilizing a twin-path architecture to reconcile masked reconstruction with taskspecific supervision-and an MoE-augmented loss that adaptively routes features through specialized subnetworks.On industrial-scale benchmarks, it achieves +5.04% AUC and +8.28% KS over prior art under rigorous scaling.Moreover, its representations distill effectively into lightweight models, yielding +2.55% AUC and +4.85% KS under strict latency and interpretability constraints, while improving robustness to distribution shifts.Our work demonstrates that tabular data admits a foundation-model treatment-when its structural idiosyncrasies are respected.
Institutions: Ant Group (China), Zhejiang University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166817281