Continuous control with Stacked Deep Dynamic Recurrent Reinforcement Learning for portfolio optimization

Aboussalah, Amine Mohamed; Lee, Chi-Guhn

doi:10.1016/j.eswa.2019.112891

Cited by 67 publications

(21 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is shown in Table 2 for the 2nd portfolio, the four RL agents exhibit the same trends to improve the NAV return of the portfolio. The gradual portfolio rebalancing with the LSTM prediction model achieves the best returns at 63.3% than individual assets of AXP, MCD, WMT in this portfolio, as well as better than those of AXP, MCD, and WMT reported in [32] in considering their trading hourly returns and corresponding portfolio weights.…”

Section: Nav Max Dropmentioning

confidence: 84%

“…For the second portfolio consisting of three stock assets from S&P 500 reported in [32], the experiment results of full portfolio rebalancing without predictive modelling are shown in Fig. 5b.…”

Section: Yes Nomentioning

confidence: 99%

“…RL has been utilised for trading of financial assets on the stock and foreign exchange market. Almahdi and Yang [32] introduce RRL-based portfolio management method for computing and optimizing investment decisions with time efficiency by incorporating past investments actions in time-stacks. The experiments have been conducted for a portfolio with ten stocks selected from different sectors of S&P 500 in time frame of January 2013 to July 2017.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic portfolio rebalancing through reinforcement learning

Lim

Cao

Quek

2021

Neural Comput & Applic

View full text Add to dashboard Cite

Portfolio managements in financial markets involve risk management strategies and opportunistic responses to individual trading behaviours. Optimal portfolios constructed aim to have a minimal risk with highest accompanying investment returns, regardless of market conditions. This paper focuses on providing an alternative view in maximising portfolio returns using Reinforcement Learning (RL) by considering dynamic risks appropriate to market conditions through dynamic portfolio rebalancing. The proposed algorithm is able to improve portfolio management by introducing the dynamic rebalancing of portfolios with vigorous risk through an RL agent. This is done while accounting for market conditions, asset diversifications, risk and returns in the global financial market. Studies have been performed in this paper to explore four types of methods with variations in fully portfolio rebalancing and gradual portfolio rebalancing, which combine with and without the use of the Long Short-Term Memory (LSTM) model to predict stock prices for adjusting the technical indicator centring. Performances of the four methods have been evaluated and compared using three constructed financial portfolios, including one portfolio with global market index assets with different risk levels, and two portfolios with uncorrelated stock assets from different sectors and risk levels. Observed from the experiment results, the proposed RL agent for gradual portfolio rebalancing with the LSTM model on price prediction outperforms the other three methods, as well as returns of individual assets in these three portfolios. The improvements of the returns using the RL agent for gradual rebalancing with prediction model are achieved at about 27.9–93.4% over those of the full rebalancing without prediction model. It has demonstrated the ability to dynamically adjust portfolio compositions according to the market trends, risks and returns of the global indices and stock assets.

show abstract

Section: Nav Max Dropmentioning

confidence: 84%

Section: Yes Nomentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic portfolio rebalancing through reinforcement learning

Lim

Cao

Quek

2021

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…Aboussalah and Lee [31] explore policy gradient techniques for continuous action and multi-dimensional state spaces, applying a stacked deep dynamic recurrent reinforcement learning architecture to construct an optimal real-time portfolio. The algorithm adopts the Sharpe ratio as a utility function to learn the market conditions and rebalance the portfolio accordingly.…”

Section: ) More Recent Workmentioning

confidence: 99%

Reinforcement Learning for Systematic FX Trading

2022

View full text Add to dashboard Cite

We explore online inductive transfer learning, with a feature representation transfer from a radial basis function network formed of Gaussian mixture model hidden processing units to a direct, recurrent reinforcement learning agent. This agent is put to work in an experiment, trading the major spot market currency pairs, where we accurately account for transaction and funding costs. These sources of profit and loss, including the price trends that occur in the currency markets, are made available to the agent via a quadratic utility, who learns to target a position directly. We improve upon earlier work by learning to target a risk position in an online transfer learning context. Our agent achieves an annualised portfolio information ratio of 0.52 with a compound return of 9.3%, net of execution and funding cost, over a 7-year test set; this is despite forcing the model to trade at the close of the trading day at 5 pm EST when trading costs are statistically the most expensive.INDEX TERMS policy gradients, recurrent reinforcement learning, online learning, transfer learning, financial time series I. INTRODUCTION

show abstract

“…They found both methods are appropriate for asset trading and concluded trading a single asset is risky and diversifying investments should be preferred. Aboussalah and Lee [18] proposed a method named stacked deep dynamic reinforcement learning (SDDRL) for real-time stock trading, and argued the selection of the appropriate hyper-parameters is especially important in this type of problem. To deal with this issue, they proposed a Bayesian approach for hyper-parameter tuning.…”

Section: Literature Reviewmentioning

confidence: 99%

Reinforcement Learning with Self-Attention Networks for Cryptocurrency Trading

Betancourt

Chen

2021

Applied Sciences

View full text Add to dashboard Cite

This work presents an application of self-attention networks for cryptocurrency trading. Cryptocurrencies are extremely volatile and unpredictable. Thus, cryptocurrency trading is challenging and involves higher risks than trading traditional financial assets such as stocks. To overcome the aforementioned problems, we propose a deep reinforcement learning (DRL) approach for cryptocurrency trading. The proposed trading system contains a self-attention network trained using an actor-critic DRL algorithm. Cryptocurrency markets contain hundreds of assets, allowing greater investment diversification, which can be accomplished if all the assets are analyzed against one another. Self-attention networks are suitable for dealing with the problem because the attention mechanism can process long sequences of data and focus on the most relevant parts of the inputs. Transaction fees are also considered in formulating the studied problem. Systems that perform trades in high frequencies cannot overlook this issue, since, after many trades, small fees can add up to significant expenses. To validate the proposed approach, a DRL environment is built using data from an important cryptocurrency market. We test our method against a state-of-the-art baseline in two different experiments. The experimental results show the proposed approach can obtain higher daily profits and has several advantages over existing methods.

show abstract

Continuous control with Stacked Deep Dynamic Recurrent Reinforcement Learning for portfolio optimization

Cited by 67 publications

References 19 publications

Dynamic portfolio rebalancing through reinforcement learning

Dynamic portfolio rebalancing through reinforcement learning

Reinforcement Learning for Systematic FX Trading

Reinforcement Learning with Self-Attention Networks for Cryptocurrency Trading

Contact Info

Product

Resources

About