Year: 2020
Type: dissertation
Abstract: In this thesis, we develop machine learning frameworks that are suitable for algorithmic trading, where environments are volatile, heteroscedastic, and driven by latent factors. The thesis contains two main parts increasing in the complexity of the frameworks we develop. In the first part, we model the dynamics of ask-bid prices using and Hidden Markov Models (HMM) and compare its performance against First Order Markov Models (MM). A supervised learning method is developed for making binary (buy or sell) market order predictions using Restricted Boltzmann Machines (RBMs). The RBMs are adapted to Gaussian-Bernoulli (GB-RBM), and later to discriminative GB-RBMs (DGB-RBM). The algorithm's performance is compared to predictions made using the industry standard of naive bid-ask depth imbalances. We demonstrate that the DGB-RBM outperforms the industry standard by up to 18% increment in the area under the receiver operating characteristic curve (AUC). In the second part, we use traditional reinforcement learning (RL) algorithms to solve an optimal execution problem – where an agent seeks to optimally sell stocks to simultaneously mitigate the risk of price movements and price impact in a stochastic environment. We develop optimal policies using variants of Q-Learning, DynaQ-ARIMA and DynaQ-LSTM algorithms, which are used as benchmarks. The analysis shows that idiosyncrasies of the optimal execution problem require delicate treatment of data inefficiency and model bias issues, which we address next. We propose a model-based RL approach, coined Reinforced Deep Kalman Filter (RDKF), designed to integrate desirable properties of a reinforcement learning algorithm acting as an automatic trading system. The network architecture allows for the possibility that market dynamics are partially visible and are modified by the agent's actions. The RDKF filters incomplete and noisy data to create better-behaved input data for RL planning. The policy search optimisation also properly accounts for state uncertainty. We perform ablation studies to understand the contributions of individual components of the RDKF. The experiments show that the RDKF is data-efficient and provides financial gains compared to the benchmarks in the optimal execution problem using real data sets from the limit order book of Facebook, Intel, Vodafone and Microsoft.
Source: TSpace (University of Toronto)
Author: Tadeu Augusto Ferreira
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W3144658031