Work

Space of Functions Computed by Deep-Layered Machines

Year: 2020

Type: article

Abstract: We study the space of functions computed by random-layered machines, including deep neural networks and Boolean circuits. Investigating the distribution of Boolean functions computed on the recurrent and layer-dependent architectures, we find that it is the same in both models. Depending on the initial conditions and computing elements used, we characterize the space of functions computed at the large depth limit and show that the macroscopic entropy of Boolean functions is either monotonically increasing or decreasing with the growing depth.

Source: Physical Review Letters

DOI: 10.1103/physrevlett.125.168301

Authors: Alexander Mozeika, Bo Li, David Saad

Institutions: London Institute for Mathematical Sciences, Aston University

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funders: Leverhulme Trust, Horizon 2020 Framework Programme, Engineering and Physical Sciences Research Council, H2020 Marie Skłodowska-Curie Actions

Awards: Containment, Avalanches and Optimisation in Spreading-processes, G4082884318, Transforming networks - building an intelligent optical infrastructure (TRANSNET), G8109566523