SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics
Year: 2026
Type: preprint
Abstract: Foundation models often obtain powerful results through very large unlabeled training datasets that dwarf available labeled datasets in size, but high energy physics already provides large samples of high-fidelity labeled simulation. We therefore ask whether self-supervised objectives improve over supervised pretraining alone when large labeled samples already exist. We introduce SuperJEPA, a joint-embedding predictive architec- ture augmented with a supervised objective, and find improved downstream jet-tagging performance across multiple tasks, model sizes, and pretraining dataset sizes. The gains persist for both frozen and fine-tuned encoders, motivating predictive self-supervision in future large-scale particle-physics foundation models.
Source: arXiv (Cornell University)
Authors: Rankin, Dylan Sheldon, Tsoi, Ho Fung
Institution: University of Pennsylvania
Language: English
Subfield: Nuclear and High Energy Physics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7224107630