CortexLens: Systematically Decoding Interpretable Visual Patterns from Human Visual Cortex
Year: 2026
Type: preprint
Abstract: Understanding how the human brain encodes visual information remains a fundamental challenge in cognitive neuroscience. Sparse dictionary learning (SDL) has recently been applied to fMRI to discover visual features represented in the visual cortex. Existing approaches, however, depend on a vision-language model to identify the feature at discovery, and report limited discovery of the low-level visual features processed in early retinotopic visual regions that play an essential role in human visual cognition. We propose an automatic framework that systematically discovers visual features encoded in the human visual cortex, encompassing both high-level and low-level visual features. The framework decomposes fMRI responses across the full visual hierarchy with Matryoshka sparse autoencoders applied per region of interest, interprets the resulting fMRI activity patterns with a vision-language model, and introduces a set of feature channels to recover the low-level visual features that language-based interpretation fails to capture. On the Natural Scenes Dataset (NSD) and on the Brain Encoder Generated Dataset (BEGD), a separate set of synthesized responses from a transformer brain encoder, the low-level stream recovers thousands of interpretable low-level visual features from early retinotopic visual regions, where prior language-based pipelines report limited discovery. The high-level stream recovers interpretable semantic visual features on both datasets and generalizes to BOLD5000. Our analysis further reveals a systematic confound in language-based interpretation: fMRI activity patterns whose true representation is a low-level visual feature can be misidentified as a semantic visual feature when that feature co-varies with a salient semantic visual feature.
Language: English
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7223304357