Deep Differentiable Logic Gate Networks Based on Fuzzy Zadeh's T-norm
Year: 2025
Type: dissertation
Abstract: Inference efficiency often dictates the practicality of deep learning models in real world applications especially on edge devices and in low latency, resource constrained scenarios, yet most research continues to emphasize training costs. In this work, we revisit the differentiable logic gate network framework of Petersen et al., (2022), which uses a continuous relaxation of discrete logic operations via the Menger T-norm (Menger, 1942) to enable gradient based training of ”and”, ”xor”, and other Boolean gates. We propose an alternative relaxation based on Zadeh’s T-norm (Zadeh, 1965), and address its non-differentiability at equal inputs by defining a consistent backward pass. During training, each neuron learns a softmax distribution over the 16 binary logic gates and at inference time, we replace these distributions with their most likely (hard) gates. The resulting Zadeh based networks require no traditional weights, are extremely sparse (two inputs per neuron), and run millions of inferences per second on a single CPU (MINST dataset (LeCun et al., 1998)). We benchmark Zadeh and Menger-based networks against each other and against conventional fully connected neural networks on tasks including MNIST (LeCun et al., 1998), CIFAR-10 (Krizhevsky, 2012), and tabular benchmarks (MONK (Thrun et al., 1991), Adult (Becker & Kohavi, 1996), Breast Cancer (Zwitter & Soklic, 1988)). Zadeh-based networks achieve up to 20 times faster inference and orders of magnitude lower memory use, at the cost of a modest accuracy gap (1.8-9.5 percentage points). Compared to the Menger variant, the Zadeh formulation is slightly faster and particularly adept at isolating singular patterns, while the Menger variant is more robust to noise. Our work introduces the choice of T-norm as a new axis of architectural optimization in differentiable logic networks, highlights hardware aware considerations such as bit packing strategies, and points toward future exploration of alternative fuzzy relaxations (e.g., Łukasiewicz T-norm) and hybrid logic architectures. The results in this thesis were from a peer-reviewed paper accepted and presented at the 6th Polish Conference on Artificial Intelligence PP-RAI 2025 (Wasilewski & Nguy, 2025).
Author: Nguy, Chan Duong
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7125586149