Physical-Measure Merton Jump-Diffusion Calibration Surrogates and Direct Realized Volatility Forecasting: An Empirical and Numerical Audit on S&P 500 ETF (SPY) Returns
Year: 2026
Type: preprint
Abstract: Continuous-time Geometric Brownian Motion (GBM) fails to capture the leptokurtic tails and sudden price discontinuities observed in equity index returns. While Merton’s (1976) Jump-Diffusion (MJD) model accounts for sudden shocks by superimposing a compound Poisson jump process onto Brownian diffusion, rolling physical-measure (\(\:P\)) Maximum Likelihood Estimation (MLE) suffers from severe finite-sample weak identification, Kiefer (1978) mixture singularities, and iterative computational overhead. In this study, we design, audit, and evaluate a rigorous, leakage-free dual-task quantitative pipeline on daily S&P 500 ETF (SPY) returns (\(\:N=\text{1,237}\) validated sessions, 2019–2023). We explicitly separate two research questions that are frequently conflated in student and practitioner implementations: (Task 1) whether Ridge regression and hybrid sequence-summary Long Short-Term Memory (LSTM) networks can serve as fast calibration surrogates for rolling 126-session multistart compound-Poisson MLE, and (Task 2) whether direct supervised volatility models or calibrated total-volatility proxies can forecast forward 21-session realized volatility better than classical historical persistence and RiskMetrics EWMA (\(\:{\lambda\:}_{ewma}=0.94\)) baselines. Across 211 chronological calibration windows (\(\:134\) Fit, \(\:31\) Validation, \(\:46\) Test) separated by an explicit \(\:H=21\)-session forward-label purge, raw five-parameter MJD fits hit box boundaries in \(\:165/211\) windows (\(\:78.2\%\)), while a conservative Bayesian Information Criterion (BIC) parsimony rule against an exact nested two-parameter Gaussian submodel (\(\:\lambda\:=0\)) selects jumps in \(\:69/211\) windows (\(\:32.7\%\), concentrated entirely around the 2020 COVID-19 crash). Online surrogate inference achieves a \(\:63.9\times\:\) wall-clock speedup (\(\:0.359\:ms\) for Mean, Ridge, and LSTM combined versus \(\:22.939\:ms\) median multistart MLE), but 5-session parameter persistence (\(\:{\widehat{\theta\:}}_{t}={\widehat{\theta\:}}_{t-5}\)) outperforms ML surrogates on rolling 126-session labels (validation standardized MSE \(\:0.0268\) vs. \(\:1.8052\) for Ridge and \(\:2.2362\) for LSTM) due to \(\:96\%\) window overlap. On the held-out 2023 test split (\(\:n=46\)), the direct hybrid LSTM achieves the lowest future 21-session realized volatility RMSE (\(\:0.026553\), MAE \(\:0.023191\), Campbell–Thompson \(\:{R}_{oos}^{2}=+16.95\%\), \(\:r=0.5049\)), followed by direct Ridge (\(\:RMSE=0.028088\), \(\:{R}_{oos}^{2}=+7.07\%\)) and EWMA (\(\:RMSE=0.028179\), \(\:{R}_{oos}^{2}=+6.46\%\)), outperforming 21-session historical volatility (\(\:RMSE=0.029137\)). However, because chronological validation during the high-volatility 2022 bear market selected the 126-session calibrated total-volatility proxy (mle_total_vol, validation \(\:RMSE=0.056528\)), which subsequently lagged the 2023 volatility decline (\(\:test\:RMSE=0.050488\), \(\:{R}_{oos}^{2}=-200.26\%\)), freezing model selection prior to test evaluation prevents claiming consistent out-of-sample forecasting superiority without a fresh post-2023 holdout.
Source: Research Square
Authors: Rajdeep Purohit, Mayank Goswami
Institution: Indian Institute of Technology Roorkee
Language: English
Subfield: Finance
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7235408944
