Year: 2024
Type: preprint
Abstract: Machine learning (ML) algorithms, particularly attention-based transformer models, have become indispensable for analyzing the vast data generated by particle physics experiments like ATLAS and CMS at the CERN LHC. Particle Transformer (ParT), a state-of-the-art model, leverages particle-level attention to improve jet-tagging tasks, which are critical for identifying particles resulting from proton collisions. This study focuses on interpreting ParT by analyzing attention heat maps and particle-pair correlations on the $η$-$ϕ$ plane, revealing a binary attention pattern where each particle attends to at most one other particle. At the same time, we observe that ParT shows varying focus on important particles and subjets depending on decay, indicating that the model learns traditional jet substructure observables. These insights enhance our understanding of the model's internal workings and learning process, offering potential avenues for improving the efficiency of transformer architectures in future high-energy physics applications.
Source: arXiv (Cornell University)
Authors: Wang, Aaron, Gandrakota, Abhijith, Ngadiuba, Jennifer, Sahu, Vivekanand, Bhatnagar, Priyansh, Khoda, Elham E, Duarte, Javier
Language: English
Subfield: Nuclear and High Energy Physics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: green
Funders: National Science Foundation, U.S. Department of Energy, Alfred P. Sloan Foundation, Research Corporation for Science Advancement, Office of Science, University of California, San Diego, Office of the President, University of California, Advanced Scientific Computing Research, High Energy Physics, Laboratory Directed Research and Development
Awards: G1387744893, CI-New: Cognitive Hardware and Software Ecosystem Community Infrastructure (CHASE-CI), Collaborative Proposal: Geodynamic Tomography: A New Approach to Studies of Mantle Upwelling and its Application to the Lau Basin and The East Pacific Rise, G2091642509, G2926102538, G3270322552, Towards Deploying a Science-Driven Regional Interoperable Science DMZ, G4404904460, G498139845, CC* NPEO: Toward the National Research Platform, C^2-THE-P^2 = CHICAGOLAND COMPUTATIONAL TRAINEESHIP IN HIGH ENERGY AND PARTICLE PHYSICS, G5590656846, G5616713406, CCRI: ABR: Cognitive Hardware and Software Ecosystem Community Infrastructure (CHASE-CI), G585908603, Category II: A Prototype National Research Platform, HDR Institute: Accelerated AI Algorithms for Data-Driven Discovery, G716412294, CC*DNI DIBBs: The Pacific Research Platform, REAL-TIME ARTIFICIAL INTELLIGENCE FOR PARTICLE RECONSTRUCTION AND HIGGS PHYSICS, REAL-TIME ARTIFICIAL INTELLIGENCE FOR PARTICLE RECONSTRUCTION AND HIGGS PHYSICS, G8623004658, G86560745, CCRI: ENS: Cognitive Hardware and Software Ecosystem Community Infrastructure (CHASE-CI), G8989177975, G969889393
OpenAlex ID: W4405096549