Year: 2023
Type: preprint
Abstract: We explore the equivalence between neural networks and kernel methods by deriving the first exact representation of any finite-size parametric classification model trained with gradient descent as a kernel machine. We compare our exact representation to the well-known Neural Tangent Kernel (NTK) and discuss approximation error relative to the NTK and other non-exact path kernel formulations. We experimentally demonstrate that the kernel can be computed for realistic networks up to machine precision. We use this exact kernel to show that our theoretical contribution can provide useful insights into the predictions made by neural networks, particularly the way in which they generalize.
Source: arXiv (Cornell University)
Authors: Bell, Brian, Geyer, Michael, Glickenstein, David, Fernandez, Amanda, Moore, Juston
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G2158447539, UNIVERSITY OF TEXAS, SAN ANTONIO
PROJECT TITLE: CONNECT- THE CONSORTIUM ON NUCLEAR SECURITY TECHNOLOGIES
PROJECT DESCRIPTION: THE CONSORTIUM ON NUCLEAR SECURITY AND TECHNOLOGIES (CONNECT) WAS ESTABLISHED IN FALL 2019 BY THE AWARD OF AN FY20 NNSA MSIPP GRANT (AWARD NUMBER: DE-NA0003948). CONNECT IS LED BY THE UNIVERSITY OF TEXAS AT SAN ANTONIO (UTSA) IN COLLABORATION WITH UNIVERSITY PARTNERS AT THE UNIVERSITY OF NEVADA LAS VEGAS (UNLV) AND ST. MARY’S UNIVERSITY (STMU). CONNECT IS A STUDENT-FOCUSED PROGRAM THAT HAS PARTNERED THESE THREE MINORITY SERVING INSTITUTIONS (MSI) WITH LOS ALAMOS NATIONAL LABORATORY (LANL) AND ARGONNE NATIONAL LABORATORY (ANL), FORMING A CROSS-DISCIPLINARY NETWORK OF FACULTY, SCIENTISTS, AND ENGINEERS FOCUSED ON TRAINING THE NEXT GENERATION OF NUCLEAR SECURITY SCIENCE AND TECHNOLOGY LEADERS. CONNECT IS IN ITS THIRD AND FINAL YEAR OF PERFORMANCE, APPLYING FOR A FIVE-YEAR RENEWAL STARTING FY23 UNDER THE PRESENT FUNDING OPPORTUNITY ANNOUNCEMENT (DE-FOA-0002494). OVER OUR 2.5 YEARS OF PERFORMANCE, CONNECT HAS PROVIDED FUNDING TO SUPPORT THE NUCLEAR SECURITY RESEARCH AND PROFESSIONAL DEVELOPMENT EFFORTS OF OVER 50 STUDENTS ACROSS OUR THREE INSTITUTIONS, WITH A STUDENT RETENTION RATE OF 86%.
CONNECT HAS PROVIDED FUNDING FOR 21 GRADUATE STUDENTS, 2 POSTDOCTORAL FELLOWS, AND 32 UNDERGRADUATE STUDENTS WITH 56% OF THE PARTICIPANTS IDENTIFIED AS UNDERREPRESENTED MINORITIES (URM). CONNECT HAS BEEN LARGELY SUCCESSFUL AT DIRECTING THESE STUDENTS TO AND PREPARING THEM FOR NUCLEAR SECURITY SCIENCE AND ENGINEERING IN THE DOE COMPLEX. IT IS THE PRIMARY AIM OF THE FUTURE-FOCUSED RENEWAL TO CONTINUE TO SUPPORT STUDENTS IN OUR PIPELINE TO ADVANCE THEM TO GRADUATION, TO RECRUIT AND ENGAGE NEW STUDENTS INTERESTED IN NOT ONLY NUCLEAR SCIENCE AND ENGINEERING BUT ALSO NUCLEAR AND NONPROLIFERATION POLICY, AND TO GROW OUR MULTI-DISCIPLINARY STUDENT AND FACULTY PROGRAMS IN A STRATEGIC WAY TO BOTH INCREASE OUR STUDENT PRESENCE AT THE NATIONAL LABS AND SUPPORT PROGRAM SUSTAINABILITY BEYOND THE RENEWAL., TRIPODS: UA-TRIPODS - Building Theoretical Foundations for Data Sciences, Collaborative Research: Algebraic Framework of Compositional Functions for New Structure, Training, and Explainability of Deep Learning, RTG: Applied Mathematics and Statistics for Data-Driven Discovery
OpenAlex ID: W4385970037