Work

Think Faster Than Words: Efficient LLM Chain-of-Thought Reasoning via Dynamic Shortcut Decoding

Year: 2026

Type: conference-paper

Abstract: This paper proposes shortcut decoding, an efficient framework for accelerating Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs).Existing methods that prune or employ early stopping to reduce latency often compromise reasoning reliability.Motivated by the observation that LLMs frequently converge to correct solutions internally before completing explicit textual reasoning, we propose a dual-signal adaptive controller that integrates lightweight probes over internal hidden states with step-level entropy.This controller detects convergence of reasoning during generation and adaptively selects between a fastexit path and a stability-verified path to remove redundant steps while preserving answer correctness.Experiments across multiple mathematical reasoning benchmarks demonstrate that shortcut decoding reduces token usage by approximately 35%, maintains accuracy comparable to full CoT decoding, and achieves finalanswer accuracy comparable to the full CoT baseline, outperforming existing early-stopping methods without updating the base model.Our code is available at https://github.com/ kuromi9527/shortcut_decoding.

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2026.acl-long.1330