A Relocated Dependency Is Not a Vanished One: What the Neural-Cellular-Automaton Result on Integer Multiplication Shows About Long-Range Dependency, Why the Neural GPU Line Already Bounds It, and the Controls That Would Separate Locality, Supervision and Carry Coverage
Year: 2026
Type: preprint
Abstract: (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Multi-digit multiplication is a standing failure case for neural sequence models, and a 2025 reverse-engineering study traced the failure of standard fine-tuning to long-range dependencies that the model never learns. A 2026 paper rejects that diagnosis: it lays two binary integers out as an outer-product grid, trains a 321-parameter neural cellular automaton on single update steps, and reports exact products up to 4,096 bits after training on 2 to 6 bits, which it reads as proof that the long-range dependency was a mirage of the chosen representation and architecture. In this analysis the author reviews and synthesises published results to show that the two papers are mostly not in conflict, because "long-range dependency" names three different things. The functional dependency of a high product bit on many input bits is a property of multiplication under any encoding; the reach of a single update step is a property of representation and architecture, and the grid removes it; the number of sequential steps is the price of that removal, and the automaton pays it, taking about n steps for n-bit operands. The paper then argues that the published record already bounds the mirage claim. Local, iterated networks learned binary multiplication from input-output pairs alone in 2015, including a one-dimensional variant that reached 2,000 bits from 20; the same line failed on long carry chains that uniformly random test inputs almost never contain; and on both sides of the dispute, most of the reliable successes supervise intermediate states. The paper consolidates the measured values in Table 1 and Figure 1, states an attribution procedure as Algorithm 1, and specifies six controls, including a carry-adversarial test set and an end-to-end-trained automaton, that would separate locality, supervision and carry coverage.
Author: Mahendrakar, Pranay
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7231476809
