The cell as a token: high-dimensional geometry in language models and cell embeddings
Year: 2025
Type: article
Abstract: MOTIVATION: Single-cell sequencing technology maps cells to a high-dimensional space encoding their internal activity. Recently-proposed virtual cell models extend this concept, enriching cells' representations based on patterns learned from pretraining on vast cell atlases. RESULTS: This review explores how advances in understanding the structure of natural language embeddings informs ongoing efforts to analyze single-cell datasets. Both fields process unstructured data by partitioning datasets into tokens embedded within a high-dimensional vector space. We discuss how the context of tokens influences the geometry of embedding space, and how low-dimensional manifolds shape this space's robustness and interpretation. We highlight how new developments in foundation models for language, such as interpretability probes and in-context reasoning, can inform efforts to construct cell atlases and train virtual cell models. AVAILABILITY AND IMPLEMENTATION: Code is available at https://github.com/williamgilpin/celltoken.
Source: Bioinformatics
Author: William Gilpin
Institution: The University of Texas at Austin
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: gold
Estimated APC paid: $3,989
OpenAlex ID: W4415719861