Work

Training Guarantees of Neural Network Classification Two-Sample Tests by Kernel Analysis

Year: 2024

Type: preprint

Abstract: We construct and analyze a neural network two-sample test to determine whether two datasets came from the same distribution (null hypothesis) or not (alternative hypothesis). We perform time-analysis on a neural tangent kernel (NTK) two-sample test. In particular, we derive the theoretical minimum training time needed to ensure the NTK two-sample test detects a deviation-level between the datasets. Similarly, we derive the theoretical maximum training time before the NTK two-sample test detects a deviation-level. By approximating the neural network dynamics with the NTK dynamics, we extend this time-analysis to the realistic neural network two-sample test generated from time-varying training dynamics and finite training samples. A similar extension is done for the neural network two-sample test generated from time-varying training dynamics but trained on the population. To give statistical guarantees, we show that the statistical power associated with the neural network two-sample test goes to 1 as the neural network training samples and test evaluation samples go to infinity. Additionally, we prove that the training times needed to detect the same deviation-level in the null and alternative hypothesis scenarios are well-separated. Finally, we run some experiments showcasing a two-layer neural network two-sample test on a hard two-sample test problem and plot a heatmap of the statistical power of the two-sample test in relation to training time and network complexity.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2407.04806

Authors: Khurana, Varun, Cheng, Xiuyuan, Cloninger, Alexander

Institutions: Brown University, Duke University, University of California San Diego

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, Directorate for Computer and Information Science and Engineering

Awards: Bridging Statistical Hypothesis Tests and Deep Learning for Reliability and Computational Efficiency, Collaborative Research: CIF: Medium:Data Manifolds and Local Geometry in Deep Networks, Collaborative Research: Computational Harmonic Analysis Approach to Active Learning, NSF-BSF: Group Invariant Graph Laplacians: Theory and Computations, CAREER: Learning of graph diffusion and transport from high dimensional data with low-dimensional structures

OpenAlex ID: W4400517440

Cites:

Cited by:

Related to: 10