Reinforcement Learning in Stock Trading

Dang, Quang-Vinh

doi:10.1007/978-3-030-38364-0_28

Cited by 34 publications

(14 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In quantitative finance, stock trading is essentially making dynamic decisions, namely to decide where to trade, at what price, and what quantity, over a highly stochastic and complex stock market. As a result, DRL provides useful toolkits for stock trading [21,44,48,45,10,8,26]. Taking many complex financial factors into account, DRL trading agents build a multi-factor model and provide algorithmic trading strategies, which are difficult for human traders [3,47,24,22].…”

Section: Introductionmentioning

confidence: 99%

FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance

Liu¹,

Yang²,

Chen³

et al. 2020

Preprint

View full text Add to dashboard Cite

As deep reinforcement learning (DRL) has been recognized as an effective approach in quantitative finance, getting hands-on experiences is attractive to beginners. However, to train a practical DRL trading agent that decides where to trade, at what price, and what quantity involves error-prone and arduous development and debugging. In this paper, we introduce a DRL library FinRL that facilitates beginners to expose themselves to quantitative finance and to develop their own stock trading strategies. Along with easily-reproducible tutorials, FinRL library allows users to streamline their own developments and to compare with existing schemes easily. Within FinRL, virtual environments are configured with stock market datasets, trading agents are trained with neural networks, and extensive backtesting is analyzed via trading performance. Moreover, it incorporates important trading constraints such as transaction cost, market liquidity and the investor's degree of risk-aversion. FinRL is featured with completeness, hands-on tutorial and reproducibility that favors beginners: (i) at multiple levels of time granularity, FinRL simulates trading environments across various stock markets, including NASDAQ-100, DJIA, S&P 500, HSI, SSE 50, and CSI 300; (ii) organized in a layered architecture with modular structure, FinRL provides fine-tuned state-ofthe-art DRL algorithms (DQN, DDPG, PPO, SAC, A2C, TD3, etc.), commonlyused reward functions and standard evaluation baselines to alleviate the debugging workloads and promote the reproducibility, and (iii) being highly extendable, FinRL reserves a complete set of user-import interfaces. Furthermore, we incorporated three application demonstrations, namely single stock trading, multiple stock trading, and portfolio allocation. The FinRL library will be available on Github at link https://github.com/AI4Finance-LLC/FinRL-Library.

show abstract

Section: Introductionmentioning

confidence: 99%

FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance

Liu¹,

Yang²,

Chen³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…They applied the idea of a rolling window, where the best algorithm is picked to trade in the following period. Recently, many researchers provide more DRL solutions for STP tasks [3,8,18]. However, most existing works are based on several assumptions, which limits the practicality.…”

Section: Drl In Financementioning

confidence: 99%

FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance

Li,

Liu,

Zheng

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning techniques are playing more and more important roles in finance market investment. However, finance quantitative modeling with conventional supervised learning approaches has a number of limitations, including the difficulty in defining appropriate labels, lack of consistency in modeling and trading execution, and lack of modeling the dynamic nature of the finance market. The development of deep reinforcement learning techniques is partially addressing these issues. Unfortunately, the steep learning curve and the difficulty in quick modeling and agile development are impeding finance researchers from using deep reinforcement learning in quantitative trading. In this paper, we propose an RLOps in finance paradigm and present a FinRL-Podracer framework to accelerate the development pipeline of deep reinforcement learning (DRL)-driven trading strategy and to improve both trading performance and training efficiency. FinRL-Podracer is a cloud solution that features high performance and high scalability and promises continuous training, continuous integration, and continuous delivery of DRL-driven trading strategies, facilitating a rapid transformation from algorithmic innovations into a profitable trading strategy. First, we propose a generational evolution mechanism with an ensemble strategy to improve the trading performance of a DRL agent, and schedule the training of a DRL algorithm onto a GPU cloud via multi-level mapping. Then, we carry out the training of DRL components with high-performance optimizations on GPUs. Finally, we evaluate the FinRL-Podracer framework for a stock trend prediction task on an NVIDIA DGX SuperPOD cloud. FinRL-Podracer outperforms three popular DRL libraries Ray RLlib, Stable Baseline 3 and FinRL, i.e., 12% ∼ 35% improvements in annual return, 0.1 ∼ 0.6 improvements in Sharpe ratio and 3× ∼ 7× speed-up in training time. We show the high scalability by training a trading agent in 10 minutes with 80 A100 GPUs, on NASDAQ-100 constituent stocks with minute-level data over 10 years.

show abstract

“…005 for three futures and 0. 001 for two stocks; the number of filters is 32; the parameters (S,M,L) for HC and I are set to (3,12,22) and set to (1,10,20) for J, NFLX and AAPL. For the INTRA module of the AuxOut, the learning rate is 0.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…It turns out that one-layer and 100 hidden units are the best. As for AuxIn and the AuxOut, we tune the number of filters being {8, 16, 32, 64}, the parameter being (S,M, L) { (1,10,20), (3,12,22 ), (5,15, 25)} of the INTER module and the size of hidden states being {16, 32, 64} of the INTRA module.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…On the one hand, besides the traditional structured data ( e. g. technical indicators [10][11][12] ), social media information like tweets and blogs [13][14][15][16] and company-related financial news [14,[16][17][18] are utilized to predict price movement. On the other hand, more elaborate classifiers based on deep learning(e. g. the long shortterm memory ( LSTM ) [19,20] , convolutional neural network [20,21] ), the reinforcement learning ( e. g. deep Q-learning [22] ) and the generative adversarial training [12] are proposed to improve the price movement prediction.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An end-to-end multitask method with two targets for high-frequency price movement prediction

Yulian¹,

Cui²

2021

Journal of University of Science and Technology of China

View full text Add to dashboard Cite

High-frequency price movement prediction is to predict the direction(e. g. up, unchanged or down) of the price change in short time ( e. g. one minute). It is challenging to use historical highfrequency transaction data to predict price movement because their relation is noisy, nonlinear and complex. We propose an end-to-end multitask method with two targets to improve high-frequency price movement prediction. Specifically, the proposed method introduces an auxiliary target ( high-frequency rate of price change), which is highly related with the main target(high-frequency price movement) and is useful to improve the high-frequency price movement prediction. Moreover, each task has a feature extractor based on recurrent neural network and convolutional neural network to learn the noisy, nonlinear and complex temporal-spatial relation between the historical transaction data and the two targets. Besides, the shared parts and task-specific parts of each task are separated explicitly to alleviate the potential negative transfer caused by the multitask method. Moreover, a gradient balancing approach is adopted to use the close relation between two targets to filter the temporal-spatial dependency learned from the inconsistent noise and retain the dependency learned from the consistent true information to improve the high-frequency price movement prediction. The experimental results on real-world datasets show that the proposed method manages to utilize the highly related auxiliary target to help the feature extractor of the main task to learn the temporal-spatial dependency with more generalization to improve high-frequency price movement prediction. Moreover, the auxiliary target ( high-frequency rate of the price change) not only improves the generalization of overall temporal-spatial dependency learned by the whole feature extractor but also improve temporal-spatial dependency learned by the different parts of the feature extractor.

show abstract

Reinforcement Learning in Stock Trading

Cited by 34 publications

References 35 publications

FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance

FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance

FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance

An end-to-end multitask method with two targets for high-frequency price movement prediction

Contact Info

Product

Resources

About