Year: 2023
Type: preprint
Abstract: Implicit neural representations (INRs) have arisen as useful methods for representing signals on Euclidean domains. By parameterizing an image as a multilayer perceptron (MLP) on Euclidean space, INRs effectively represent signals in a way that couples spatial and spectral features of the signal that is not obvious in the usual discrete representation, paving the way for continuous signal processing and machine learning approaches that were not previously possible. Although INRs using sinusoidal activation functions have been studied in terms of Fourier theory, recent works have shown the advantage of using wavelets instead of sinusoids as activation functions, due to their ability to simultaneously localize in both frequency and space. In this work, we approach such INRs and demonstrate how they resolve high-frequency features of signals from coarse approximations done in the first layer of the MLP. This leads to multiple prescriptions for the design of INR architectures, including the use of complex wavelets, decoupling of low and band-pass approximations, and initialization schemes based on the singularities of the desired signal.
Source: arXiv (Cornell University)
Institution: Rice University
Language: English
Subfield: Applied Mathematics
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
Awards: G1523888516, G1660931560, G2308282352, BIGDATA: F: Collaborative Research: Theory and Practice of Randomized Algorithms for Ultra-Large-Scale Signal Processing, DEEP LEARNING FOR FORECASTING OF FRACTURE AND FAULT EVOLUTION, G3587679958, Collaborative Research: Computational Photo-Scatterography: Unraveling Scattered Photons for Bio-Imaging, G5121974832, G6098521345, G6706517093, G7840735550, G8280646250, CIF: Small: A Probabilistic Theory of Deep Learning via Spline Operators, G8876996369
OpenAlex ID: W4387323652