Year: 2026
Type: preprint
Abstract: Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train such models are rarely public due to privacy constraints. While there are methods to generate synthetic tabular data of arbitrary size, incorporating schema structure and primary-foreign key connectivity for multi-table generation remains challenging. Here we introduce PLUREL, a framework to synthesize multi-tabular relational databases from scratch. In a step-by-step fashion, PLUREL models (1) schemas with directed graphs, (2) inter-table primary-foreign key connectivity with bipartite graphs, and, (3) feature distributions in tables via conditional causal mechanisms. The design space across these stages supports the synthesis of a wide range of diverse databases, while being computationally lightweight. Using PLUREL, we observe for the first time that (1) RFM pretraining loss exhibits power-law scaling with the number of synthetic databases and total pretraining tokens, (2) scaling the number of synthetic databases improves generalization to real databases, and (3) synthetic pretraining yields strong base models for continued pretraining on real databases. Overall, our framework and results position synthetic data scaling as a promising paradigm for RFMs.
Source: arXiv (Cornell University)
Authors: Kothapalli, Vignesh, Ranjan, Rishabh, Hudovernik, Valter, Dwivedi, Vijay Prakash, Hoffart, Johannes, Guestrin, Carlos, Leskovec, Jure
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: Collaborative Research: IHBEM: Data-driven multimodal methods for behavior-based epidemiological modeling, Collaborative Research: III: Medium: Contextualized and Multimodal Foundation Models for Graph Data in Scientific Discovery, Expeditions: Collaborative Research: Global Pervasive Computational Epidemiology
OpenAlex ID: W7127942774