Year: 2026
Type: preprint
Abstract: Large language models (LLMs) used for Web information processing rely on retrieving relationships between network entities, such as domain names and IP addresses, but the internal components that causally enable this retrieval remain unclear, hindering targeted safeguards against unsafe actions. Therefore, we investigate whether their retrieval capabilities can be localized to specific attention heads or individual neurons within these LLMs. Across five LLMs from three architecture families, we identify heads through causal ablation screening, test their selectivity against matched negative and context-free controls, rank neurons within these heads using causal ablation and correlational measures, and leverage features from both causally validated heads and individual neurons to build network information retrieval detectors, which we evaluate without retraining on an independent reverse-DNS dataset. In every model, a small set of causally validated heads (1--9 out of 128--1152 candidates) builds a detector with 99.5--100\% held-out accuracy. At the neuron level, the results are model-specific, e.g., in one model, causal and correlational rankings identify the same neuron, matching full-head performance (AUC = 1.000), while in another, the best causally ranked neuron achieves only AUC = 0.665 despite the full head reaching 1.000. On independent CAIDA data, head detectors maintain AUC $\geq 0.971$, while single-neuron detectors fall below chance in three of five models. Overall, these findings highlight the importance of choosing an appropriate level of granularity when causally localizing how LLMs retrieve structured network information from context.
Source: arXiv (Cornell University)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7220909013