Structural break-aware pairs trading strategy using deep reinforcement learning

Abstract

Pairs trading is an effective statistical arbitrage strategy considering the spread of paired stocks in a stable cointegration relationship. Nevertheless, rapid market changes may break the relationship (namely structural break), which further leads to tremendous loss in intraday trading. In this paper, we design a two-phase pairs trading strategy optimization framework, namely structural break-aware pairs trading strategy ( SAPT ), by leveraging machine learning techniques. Phase one is a hybrid model extracting frequency- and time-domain features to detect structural breaks. Phase two optimizes pairs trading strategy by sensing important risks, including structural breaks and market-closing risks, with a novel reinforcement learning model. In addition, the transaction cost is factored in a cost-aware objective to avoid significant reduction of profitability. Through large-scale experiments in real Taiwan stock market datasets, SAPT outperforms the state-of-the-art strategies by at least 456% and 934% in terms of profit and Sortino ratio, respectively.

Hybrid Deep Reinforcement Learning for Pairs Trading

“…Deep reinforcement learning methods for pairs trading can be classified into two approaches. The first approach [1,2] indirectly determines trading actions based on trading and stop-loss boundaries. Kim and Kim [1] proposed the pairs trading DQN (PTDQN) algorithm, which dynamically optimizes the boundaries for daily stock data.…”

“…The gap between the trading and stop-loss boundaries is fixed at 2.0 for all actions. Lu et al [2] focused on intraday trading, where the cointegration relationship is much weaker than that of interday trading. To detect structural breaks in which the cointegration relationship vanishes, the authors proposed a spread wavelet-aware hybrid network that combines a continuous wavelet convolutional neural network [38] for frequency-domain features and a long short-term memory (LSTM) network [39] for time-domain features.…”

“…We evaluate the methods in terms of the rate of return and risk indicators, such as the Sharpe ratio (SR) [53] and maximum drawdown (MDD) [2]. The rate of return is computed by Equation (29), where PV start is the initial money of $10,000, and PV end is the portfolio value at the end of the test period, which is the sum of the values of long and short positions and the remaining money.…”

“…In recent years, deep reinforcement learning has received considerable attention in the field of algorithmic trading because it has both the perception ability of deep learning and the decision-making ability of reinforcement learning. Pairs trading methods based on deep reinforcement learning can be classified into two approaches: (1) indirectly determining trading actions, such as long and short, based on the trading and stop-loss boundaries [1,2] and (2) directly determining trading actions based on the spread [3][4][5]. The former approach has a fixed gap between the trading and stop-loss boundaries, as shown in Figure 2, which is certainly not optimal because the trading boundary is completely dependent on the stop-loss boundary.…”

Improving Pairs Trading Strategies Using Two-Stage Deep Learning Methods and Analyses of Time (In)variant Inputs for Trading Performance

“…To further enhance PTS investment quality, Sarmento and Horta [35] and Lu et al [30] indirectly predict and remove unprofitable stock pairs from trading without taking into account the quality of the recommended trigger thresholds. Sarmento and Horta [35] group stocks according to the OPTICS algorithm, then remove pairs whose stocks come from differ-ent groups.…”