Residual Tensor Train: A Quantum-Inspired Approach for Learning Multiple Multilinear Correlations
Year: 2022
Type: article
Abstract: States of quantum many-body systems are defined in a high-dimensional Hilbert space, where rich and complex interactions among subsystems can be modeled. In machine learning, complex multiple multilinear correlations may also exist within input features. In this article, we present a quantum-inspired multilinear model, named residual tensor train (ResTT), to capture the multiple multilinear correlations of features, from low to high orders, within a single model. ResTT is able to build a robust decision boundary in a high-dimensional space for solving fitting and classification tasks. In particular, we prove that the fully connected layer and the Volterra series can be taken as special cases of ResTT. Furthermore, we derive the rule for weight initialization that stabilizes the training of ResTT based on a mean-field analysis. We prove that such a rule is much more relaxed than that of tensor train (TT), which means that ResTT can easily address the vanishing and exploding gradient problem that exists in the existing TT models. Numerical experiments demonstrate that ResTT outperforms the state-of-the-art tensor network and benchmark deep learning models on MNIST and Fashion-MNIST datasets. Moreover, ResTT achieves better performance than other statistical methods on two practical examples with limited data, which are known to have complex feature interactions.
Authors: Yiwei Chen, Yu Pan, Daoyi Dong
Language: English
Subfield: Computational Mathematics
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
Awards: G6868492047, G994192881
OpenAlex ID: W4288064556