Year: 2024
Type: preprint
Abstract: In this work, we significantly enhance masked particle modeling (MPM), a self-supervised learning scheme for constructing highly expressive representations of unordered sets relevant to developing foundation models for high-energy physics. In MPM, a model is trained to recover the missing elements of a set, a learning objective that requires no labels and can be applied directly to experimental data. We achieve significant performance improvements over previous work on MPM by addressing inefficiencies in the implementation and incorporating a more powerful decoder. We compare several pre-training tasks and introduce new reconstruction methods that utilize conditional generative models without data tokenization or discretization. We show that these new methods outperform the tokenized learning objective from the original MPM on a new test bed for foundation models for jets, which includes using a wide variety of downstream tasks relevant to jet physics, such as classification, secondary vertex finding, and track identification.
Source: arXiv (Cornell University)
Language: English
Subfield: Nuclear and High Energy Physics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: green
Awards: Junior Leader la Caixa Postdoctoral Fellowship Programme: Shaping the new generation of leaders in research, G1897041155, G1952528917, G2371548206, G303061748, G3969041401, G4066435496, EXC 2094: ORIGINS: Vom Ursprung des Universums bis zu den ersten Bausteinen des Lebens, G4711507462, G5570207115, At the two upgrade frontiers: machine learning and the ITk Pixel detector, Robust Deep Density Models for High-Energy Particle Physics and Solar Flare Analysis (RODEM), G969889393
OpenAlex ID: W4403851175