An end-to-end direct reinforcement learning approach for multi-factor based mean-variance portfolio optimization
Year: 2025
Type: article
Abstract: This paper introduces an end-to-end online portfolio decision model within the framework of direct reinforcement learning, seamlessly integrating the multi-factor model and mean-variance (MV) portfolio optimization. Recognizing that classical methods, which separate estimation and portfolio optimization into a two-step scheme, may accumulate estimation errors jeopardizing overall performance, our approach unifies these steps into a performance-oriented online decision process. This integration is achieved by tuning the neural network parameters directly with respect to the reward function, designed as a combination of the prediction error and realized MV utility. Specifically, we employ a neural network to estimate future returns and generate the factor loading matrix, enabling the computation of inputs for the MV portfolio optimization model. Implementing the resulting portfolio further provides the realized utility. The network parameters are optimized with respect to the updated reward using the gradient method. We develop an online updating scheme for computing the gradient in backpropagation by providing explicit formulas for MV portfolio derivatives through the portfolio optimization layer. Utilizing real market data, we evaluate the proposed method against several benchmark portfolios in out-of-sample tests. The experiments demonstrate that our approach not only outperforms these benchmarks across various performance metrics but is also transparent to factor analysis, a favorable trait for practitioners.
Source: European Journal of Finance
Authors: Ke Zhou, Xinman Huang, Xinnan Chen, Jianjun Gao
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Awards: 金融决策模型的泛化问题研究, G2912422488
OpenAlex ID: W4414528737