Thinking-Trace Analysis: Methodology for Surfacing Operational Improvements and Behavioral-Contract Candidates from Agent Planning Text
Year: 2026
Type: preprint
Abstract: This paper records a methodology for analyzing AI agent thinking traces: the extended planning text an agent produces before and during task execution. The methodology uses a two-pass structure. The first pass is a piece-by-piece temporal pass that extracts operational improvements, behavioral-contract candidates, vocabulary coinages, and efficiency patterns per trace segment. The second is a structural synthesis pass that ranks improvements by leverage and attributes overhead to either the system (form and protocol gaps) or the agent (judgment patterns). The central analytical question is whether observed overhead is structural or behavioral, because the distinction determines the appropriate remediation. The methodology is demonstrated on one trace from an experimental inquest design task. Limitations include single-trace calibration, analyst-dependent segmentation, and invisibility of absent reasoning.
Author: Neilson, Sophie
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7211896300
Cites:
Cited by:
Related to: