Year: 2026
Type: conference-paper
Abstract: Representing continuous time is a critical and under-explored challenge in modeling temporal event sequences with large language models (LLMs).Various strategies like byte-level representations or calendar tokens have been proposed.However, the optimal approach remains unclear, especially given the diverse statistical distributions of real-world event data, which range from smooth log-normal to discrete, spiky patterns.This paper presents a systematic empirical study of temporal tokenization for modeling event sequences with LLMs, comparing distinct encoding strategies: naive numeric strings, high-precision byte-level representations, human-semantic calendar tokens, classic uniform binning, and adaptive residual scalar quantization.We evaluate these strategies by fine-tuning LLMs on real-world datasets that exemplify these diverse distributions.Our analysis reveals that no single strategy is universally superior; instead, prediction performance depends heavily on aligning the tokenizer with the data's statistical properties, highlighting temporal tokenization as a critical yet often overlooked design dimension in LLM-based event modeling.
Institutions: Capital One (United States), Georgia Institute of Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4417469573