Year: 2024
Type: dissertation
Abstract: We introduce a categorical model to represent neural networks and deep learning systems using parametric spans. This model is based on parametric spans, which are used to represent any single part of a deep learning system, and through their composition the whole system can be formed. Diagrams of arrows can be used to graphically represent neural networks and deep learning systems. Neural network based deep learning is a powerful tool used in a variety of ways in science and many industries. The exceptionally large amount of possible model parameters and the versatility to accommodate different types of data make the modern deep learning architectures expressive enough to surpass human performance in many tasks. There is a lack in theoretical understanding of design choices with respect to the performance of a system. The computational costs of empirical testing make greater theoretical insight valuable for deep learning development. In recent research there has been a focus on the intrinsic geometrical properties of the data given to a neural network as input. Geometrical symmetries can be used for example to generalize images of the same object in different positions, and any given symmetry might beget a certain network architecture that is best able to utilize it. Category theory is an abstract branch of mathematics for mathematical structures and the relations between them, and it has been presented as a possible solution for the lack of unified and mathematically clear representations of neural network deep learning architectures. This thesis presents the fundamentals of typical deep learning architectures, and gives a categorical model for representing them. The model provides a versatile yet informative way to represent neural networks and deep learning systems.
Source: Aaltodoc (Aalto University)
Author: Kettunen, Heikki
Institution: Aalto University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7002462301