AlphaQT-Bench: Diagnosing the Gap between Financial Code Generation and Quantitative Reasoning in LLMs
Year: 2026
Type: conference-paper
Abstract: Large Language Models (LLMs) are increasingly applied to alpha mining in quantitative finance, marking a shift from generating simple symbolic formulas to producing executable, code-based strategies.While code generation offers greater expressiveness, it introduces critical risks absent in symbolic approaches, including temporal causality violations (look-ahead bias) and stateful logic bugs.Existing benchmarks largely rely on outcome-driven metrics (e.g., backtesting profitability), which often conflate market stochasticity or unintended information leakage with genuine reasoning competence.We introduce ALPHAQT-BENCH, a diagnostic benchmark for instruction-grounded financial code generation under strict semantic and temporal constraints.Unlike general-purpose coding benchmarks, ALPHAQT-BENCH adopts a multi-layer evaluation protocol that assesses: (1) executability, (2) causality safety via a dynamic truncation test, (3) functional accuracy, and (4) structural compliance through vectorization analysis.Experiments across 12 representative LLMs reveal a substantial gap between surface-level success (e.g., executability) and verified quantitative correctness, as many models fail under causal, structural, or functional constraints.By shifting evaluation from profitability to process reliability, ALPHAQT-BENCH provides a principled safety audit for emerging LLM-based quantitative systems.
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: hybrid
OpenAlex ID: W7166834389