Work

On the Emergence and Test-Time Use of Structural Information in Large Language Models

Year: 2026
Type: conference-paper
Abstract: Learning structural information from observational data is central to producing new knowledge outside the training corpus. This holds for mechanistic understanding in scientific discovery as well as flexible test-time compositional generation. We thus study how language models learn abstract structures and utilize the learnt structural information at test-time. To ensure a controlled setup, we design a natural language dataset based on linguistic structural transformations. We empirically show that the emergence of learning structural information correlates with complex reasoning tasks, and that the ability to perform test-time compositional generation remains limited.
Language: English
Open Access status: hybrid
OpenAlex ID: W7166867402
FWCI: 1.771
Cites: 29
Cited by: 2
Related to: 10