TradeR: Practical Deep Hierarchical Reinforcement Learning for Trade Execution

Suri, Karush; Shi, Xiao Qi; Plataniotis, Konstantinos; Lawryshyn, Yuri

doi:10.48550/arxiv.2104.00620

Cited by 2 publications

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hierarchical RL methods hold promise for general artificial intelligence. In trading, they have been used successfully, for example, in portfolio management [26,27], but not in the single asset case.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical Model-Based Deep Reinforcement Learning for Single-Asset Trading

Millea

2023

Analytics

View full text Add to dashboard Cite

We present a hierarchical reinforcement learning (RL) architecture that employs various low-level agents to act in the trading environment, i.e., the market. The highest-level agent selects from among a group of specialized agents, and then the selected agent decides when to sell or buy a single asset for a period of time. This period can be variable according to a termination function. We hypothesized that, due to different market regimes, more than one single agent is needed when trying to learn from such heterogeneous data, and instead, multiple agents will perform better, with each one specializing in a subset of the data. We use k-meansclustering to partition the data and train each agent with a different cluster. Partitioning the input data also helps model-based RL (MBRL), where models can be heterogeneous. We also add two simple decision-making models to the set of low-level agents, diversifying the pool of available agents, and thus increasing overall behavioral flexibility. We perform multiple experiments showing the strengths of a hierarchical approach and test various prediction models at both levels. We also use a risk-based reward at the high level, which transforms the overall problem into a risk-return optimization. This type of reward shows a significant reduction in risk while minimally reducing profits. Overall, the hierarchical approach shows significant promise, especially when the pool of low-level agents is highly diverse. The usefulness of such a system is clear, especially for human-devised strategies, which could be incorporated in a sound manner into larger, powerful automatic systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Hierarchical Model-Based Deep Reinforcement Learning for Single-Asset Trading

Millea

2023

Analytics

View full text Add to dashboard Cite

show abstract

Hierarchical Model-Based Deep Reinforcement Learning For Single-Asset Trading

Millea¹

2023

Preprint

View full text Add to dashboard Cite

We present a hierarchical reinforcement learning (RL) architecture that employs various low-level agents to act in the trading environment, i.e. the market. The highest level agent selects among a group of specialised agents, and then the selected agent decides when to sell or buy a single asset for some period. This period can be variable according to a termination function. We hypothesized that due to different market regimes, more than one single agent is needed when trying to learn from such heterogeneous data, and instead, multiple agents will perform better, with each one specialising in a subset of the data. We use $k-means$ clustering to partition the data and train each agent with a different cluster. Partitioning the input data also helps model-based RL (MBRL), where models can be heterogeneous. We also add two simple decision-making models to the set of low-level agents, diversifying the pool of available agents and thus increasing overall behaviour flexibility. We perform multiple experiments showing the strengths of a hierarchical approach and test various prediction models at both levels. We also use a risk-based reward at the high level, which transforms the overall problem into a risk-return optimization. This type of reward shows a significant reduction in risk while minimally reducing profits. Overall, the hierarchical approach shows significant promise, especially when the pool of low-level agents is highly diverse.

show abstract

TradeR: Practical Deep Hierarchical Reinforcement Learning for Trade Execution

Cited by 2 publications

References 23 publications

Hierarchical Model-Based Deep Reinforcement Learning for Single-Asset Trading

Hierarchical Model-Based Deep Reinforcement Learning for Single-Asset Trading

Hierarchical Model-Based Deep Reinforcement Learning For Single-Asset Trading

Contact Info

Product

Resources

About