What is the value of the cross-sectional approach to deep reinforcement learning?

Aboussalah, Amine Mohamed; Xu, Ziyun; Lee, Chi-Guhn

doi:10.1080/14697688.2021.2001032

Cited by 13 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Examples of reward signals include portfolio return (Jiang et al., 2017; Pendharkar & Cusatis, 2018; Yu et al., 2019), (differential) Sharpe ratio (Du et al., 2016; Pendharkar … Cusatis, 2018), and profit (Du et al., 2016). The benchmark strategies include Constantly Rebalanced Portfolio (CRP) (Yu et al., 2019; Jiang et al., 2017) where at each period the portfolio is rebalanced to the initial wealth distribution among the assets, and the buy‐and‐hold or do‐nothing strategy (Park et al., 2020; Aboussalah, 2020), which does not take any action but rather holds the initial portfolio until the end. The performance measures studied in these papers include the Sharpe ratio (Yu et al., 2019; Wang … Zhou, 2020; Xiong et al., 2018; Jiang et al., 2017; Liang et al., 2018; Park et al., 2020; Wang, 2019), the Sortino ratio (Yu et al., 2019), portfolio returns (Aboussalah, 2020; Liang et al., 2018; Park et al., 2020; Wang, 2019; Xiong et al., 2018; Yu et al., 2019), portfolio values (Jiang et al., 2017; Pendharkar & Cusatis, 2018; Xiong et al., 2018), and cumulative profits (Du et al., 2016).…”

Section: Applications In Financementioning

confidence: 99%

“…(2021) embedded alpha portfolio strategies into a deep policy‐based method and designed a framework, which is easier to interpret. Using Actor–Critic methods, Aboussalah (2020) combined the mean–variance framework (the actor determines the policy using the mean–variance framework) and the Kelly Criterion framework (the critic evaluates the policy using their growth rate). They studied eight policy‐based algorithms including DPG, DDPG, and PPO, among which DPG was shown to achieve the best performance.…”

Section: Applications In Financementioning

confidence: 99%

“…Both value‐based methods such as Q ‐learning (Du et al., 2016; Pendharkar & Cusatis, 2018), SARSA (Pendharkar & Cusatis, 2018), and DQN (Park et al., 2020), and policy‐based algorithms such as DPG and DDPG (Xiong et al., 2018; Jiang et al., 2017; Yu et al., 2019; Liang et al., 2018; Aboussalah, 2020; Cong et al., 2021) have been applied to solve portfolio optimization problems. The state variables are often composed of time, asset prices, asset past returns, current holdings of assets, and remaining balance.…”

Section: Applications In Financementioning

confidence: 99%

See 2 more Smart Citations

Recent advances in reinforcement learning in finance

Hambly

Yang

2023

Mathematical Finance

View full text Add to dashboard Cite

The rapid changes in the finance industry due to the increasing amount of data have revolutionized the techniques on data processing and data analysis and brought new theoretical and computational challenges. In contrast to classical stochastic control theory and other analytical approaches for solving financial decision‐making problems that heavily reply on model assumptions, new developments from reinforcement learning (RL) are able to make full use of the large amount of financial data with fewer model assumptions and to improve decisions in complex financial environments. This survey paper aims to review the recent developments and use of RL approaches in finance. We give an introduction to Markov decision processes, which is the setting for many of the commonly used RL approaches. Various algorithms are then introduced with a focus on value‐ and policy‐based methods that do not require any model assumptions. Connections are made with neural networks to extend the framework to encompass deep RL algorithms. We then discuss in detail the application of these RL algorithms in a variety of decision‐making problems in finance, including optimal execution, portfolio optimization, option pricing and hedging, market making, smart order routing, and robo‐advising. Our survey concludes by pointing out a few possible future directions for research.

show abstract

Section: Applications In Financementioning

confidence: 99%

Section: Applications In Financementioning

confidence: 99%

Section: Applications In Financementioning

confidence: 99%

See 1 more Smart Citation

Recent advances in reinforcement learning in finance

Hambly

Yang

2023

Mathematical Finance

View full text Add to dashboard Cite

show abstract

“…Brandt et al (2009) also assume parametric portfolio policies that exploit the characteristics of the cross-section of returns in an optimal asset allocation context. Aboussalah et al (2021) apply mean-variance (and growth optimal investing) strategies in a cross-sectional setting.…”

Section: Literature Reviewmentioning

confidence: 99%

Optimal Characteristic Portfolios

McGee¹,

Olmo

2020

SSRN Journal

View full text Add to dashboard Cite

“…In our work, we directly optimized the cross-holdings between organizations from a global regulator's perspective in order to mitigate systemic risk of the global network in the face of exogenous shocks. The reinforcement learning agents in 14 and 15 would be good candidates for extending 13 work to use multi-agent learning to better understand and mitigate systemic risk.…”

mentioning

confidence: 99%

Quantum computing reduces systemic risk in financial networks

Aboussalah

Chi

Lee

2023

Sci Rep

Self Cite

View full text Add to dashboard Cite

In highly connected financial networks, the failure of a single institution can cascade into additional bank failures. This systemic risk can be mitigated by adjusting the loans, holding shares, and other liabilities connecting institutions in a way that prevents cascading of failures. We are approaching the systemic risk problem by attempting to optimize the connections between the institutions. In order to provide a more realistic simulation environment, we have incorporated nonlinear/discontinuous losses in the value of the banks. To address scalability challenges, we have developed a two-stage algorithm where the networks are partitioned into modules of highly interconnected banks and then the modules are individually optimized. We developed a new algorithms for classical and quantum partitioning for directed and weighed graphs (first stage) and a new methodology for solving Mixed Integer Linear Programming problems with constraints for the systemic risk context (second stage). We compare classical and quantum algorithms for the partitioning problem. Experimental results demonstrate that our two-stage optimization with quantum partitioning is more resilient to financial shocks, delays the cascade failure phase transition, and reduces the total number of failures at convergence under systemic risks with reduced time complexity.

show abstract

What is the value of the cross-sectional approach to deep reinforcement learning?

Cited by 13 publications

References 26 publications

Recent advances in reinforcement learning in finance

Recent advances in reinforcement learning in finance

Optimal Characteristic Portfolios

Quantum computing reduces systemic risk in financial networks

Contact Info

Product

Resources

About