Joint State Transition Prediction: Goal Generalization with Sparse Action Labels
Year: 2026
Type: preprint
Abstract: Empirical paper: English PDF · 中文论文. Full theoretical manuscript: English translation · 中文原稿. Four-track rules (§5.1, p. 4): After a horizontal probe and reset, a goal card requests up or down on three of four tracks. A vertical pull on A or B moves all four; switching handles reverses every direction. Two matching tracks win a card, and both opposite cards must be won from the same reset state. Layout 113670000 illustrates a winning and a losing pair. 四轨计分:一张卡要求三轨,至少两轨方向正确才过;同一次试探后,两张相反目标都过才得一分。11 月 21 日补明规则,权重、数据与成绩未变。 A controller can learn what an action does without being told when to take it. Does that knowledge help when a new goal arrives? Joint State Transition Prediction (JSTP) treats action and environmental change as components of the same transition; it does not require a full reconstruction of the future. We varied the number of action labels in a fixed bank of 16,384 interactions. Frozen controllers operated a four-track simulator on 192 held-out layouts. A pair counted as successful only if the controller achieved two opposite, unseen goals after the same probe. At 128 labels, JSTP achieved 71.69% paired success. Disconnecting its consequence-to-action path while retaining both training losses reduced success to 66.75% (difference: 4.94 percentage points; 99% paired layout-bootstrap interval [3.97, 5.85]). In an expanded comparison, a pretrained ACT adaptation did better at 128 labels (73.78%); at 1,024 labels ACT and JSTP reached 94.58% and 94.90%, respectively. JSTP thus approached ACT on this apparatus with 18,304 parameters trained from scratch, but was not the best sparse-label method. The matched comparison supports a contribution from consequence feedback; task success alone does not establish emergent object segmentation. What changed in v1.2. Table 2 compares full JSTP with five external adaptations on the same held-out layouts at 128, 1,024 and 16,384 action labels. JSTP, ACT, Diffusion Policy and the V-JEPA 2.1 readout were fitted anew; the existing SmolVLA and DINO-WM adaptations were re-evaluated. The source archive includes per-layout counts and per-seed outcomes. Download the comparison workbook. Public project repository · Fixed comparison archive.
Author: Yue, Ke
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7236848788
