Efficient Finite Initialization with Partial Norms for Tensorized Neural Networks and Tensor Networks Algorithms
Year: 2023
Type: preprint
Abstract: We present two algorithms to initialize layers of tensorized neural networks and general tensor network algorithms using partial computations of their Frobenius norms and positive lineal entrywise sums, depending on the type of tensor network involved. The core of this method is the use of the norm of subnetworks of the tensor network in an iterative way, so that we normalize by the finite values of the norms that led to the divergence or zero norm. In addition, the method benefits from the reuse of intermediate calculations. We have also applied it to the Matrix Product State/Tensor Train (MPS/TT) and Matrix Product Operator/Tensor Train Matrix (MPO/TT-M) layers and have seen its scaling versus the number of nodes, bond dimension, and physical dimension. All code is publicly available.
Source: arXiv (Cornell University)
Institution: Euskadiko Parke Teknologikoa
Language: English
Topic: Quantum many-body systems
Subfield: Atomic and Molecular Physics, and Optics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4386793375