No Abstraction Tax at the Direct Surface: A Pre-registered Paired Study of LangChain Used Directly versus a Native Loop, in Which the Registered Tax Hypothesis Was Refuted and Exactly-Once Recovery Held a Third Time
Year: 2026
Type: preprint
Abstract: Most teams meet LangChain before they meet any graph framework, and they use it directly: chains and runnables, no graph layer. Whether that direct surface costs anything against a plain hand-written loop, on identical work, is an empirical question that is usually settled by fashion. This study pre-registered the measurement as the paired sequel to a LangGraph versus native study on the same suite: the same 12 tasks, the same deterministic scripted policy, the same tools, the same step executor, the same budgets, paired by seed, with the native arm imported verbatim from the sibling repository and hash-gated so the control cannot drift. The LangChain arm sequences each step as one invocation of a RunnableSequence of three RunnableLambda stages (load state, run one executor step, persist state) from langchain-core 1.7.6; because langchain-core ships no checkpointer, the persist stage writes the full state dict to the arm's own SQLite store after every step, the same commit-per-step cadence as the native event log. The scripted policy is a declared scope limit: the study measures the orchestration layer only. On task success the arms are equivalent with evidence: overall pass^1 is 0.979 in both arms (94 of 96 runs each), all 12 paired per-task differences are exactly 0.000, and the 90 percent interval for the mean paired difference is [0.000, 0.000], inside the registered margin of plus or minus 0.05 (H1: CONFIRMED). Consistency (pass^k, k = 1 to 8) is identical in the two arms at every k. On cost, the registered abstraction-tax hypothesis fails: the median wall ratio (LangChain over native) is 0.459, under the frozen 1.00 no-tax line, so H2 is REFUTED. LangChain direct was faster than this native implementation on this suite (median 0.398 seconds per run against 0.869; faster in 96 of 96 paired runs, paired bootstrap 95 percent interval for the ratio [0.414, 0.497]), a comparison the paper flags as implementation-sensitive because the native arm commits state synchronously after every step. Policy invocations are identical (median 5.0 per run in both arms). On recovery after a SIGKILL (48 fault runs per arm, two kill classes), both arms pass every run with every intended side effect applied exactly once and zero duplicated effects (H3: CONFIRMED), and the recovery price is also identical: median recomputed steps 0.5 in both arms, with no recomputation in the step-boundary kill class and exactly one recomputed step in the mid-step class, in all runs of both arms. The LangChain arm's durable state store is the smallest of the three arms now measured on this suite (median 8,192 bytes per run, against 16,384 for the native event log and 36,864 for the sibling study's LangGraph checkpoint store), and its arm-specific code is 95 code lines against the native arm's 96. Exactly-once came from the shared idempotency discipline for the third time (native, LangGraph, now LangChain direct), not from the orchestrator.
Author: Ushno, Jesun Ahmad
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7236041397
