Year: 2025
Type: dissertation
Abstract: This thesis explores how interpretable semantic structures can be identified within, or incorporated into, the internal representations of deep neural networks. For the class of large-scale autoregressive models, we define a notion of `meaning' representation based on the distribution over future continuations of a given input. The resulting space of meanings is shown to exhibit geometric and compositional properties which can be shown to align with human annotations. Leveraging text-conditioned image diffusion models, meaning representations can be grounded in visual imagery by adopting an alternate definition based on the distributions over conjured imageries. Semantic similarity scores can be obtained as the Jensen-Shannon divergence between the reverse-time diffusion stochastic differential equations induced by different textual expressions. Beyond the discovery of emergent semantic structures, this thesis shows how compositional structures designed to be interpretable can be achieved through deliberate architectural choices. In addition to their interpretability, these structures offer significant practical advantages, enabling state-of-the-art performance in tasks such as retrieval-augmented generation and model composition.
Author: Liu, Tian Yu
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7038098778