Unsupervised Discovery of Language and Topic Lanes in Transformer Models via Multilingual Co-firing Signatures
Year: 2026
Type: preprint
Abstract: I run ten multilingual prompts through qwen2.5:7B while recording per-prompt activation at every sensor in a one-million-cell Mercury grid. The output is a binary signature for each active cell, encoding which prompts triggered it. From 14,912 active cells, 944 distinct firing patterns appear. Categorizing cells by signature recovers six functional 'lanes' with no supervised labels: a universal backbone (39 cells fire in all ten queries), a Chinese-only language detector (73 cells), an English-only language detector (56 cells), and three topic lanes. The headline result is a cross-language physics-reasoning lane: 50 cells fire only on the two physics inputs (one in Chinese, one in English), and on nothing else. These 50 units are quantitative evidence that the model maintains topic abstractions whose representation is independent of surface tongue. The full pattern analysis runs in under a minute on a laptop after the observation pass; no auxiliary training is involved. Companion records: Mercury method (10.5281/zenodo.20313154), Mercury-Viewer dataset (10.5281/zenodo.20313150).
Author: Chen, Ho Yiing
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7161848347
Cites:
Cited by:
Related to: