Towards a fully rl-based market simulator

Ardon, Leo; Vadori, Nelson; Spooner, Thomas; Xu, Mengda; Vann, Jared; Ganesh, Sumitra

doi:10.1145/3490354.3494372

Cited by 10 publications

(9 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As part of the framework, we provide two different implementations of the network that already cover a range of use cases [1,6,12]. The first one is a static network where the connectivity between the agents is defined upfront and remains static throughout training and simulation.…”

Section: Network Modelmentioning

confidence: 99%

“…For instance, the "Turn Based Env" characterizes a Stackelberg game where the agents are categorized into two groups playing alternatively. This type of game can be used to evaluate how one group of agents react to the actions performed by the agents from the other group [1].…”

Section: Modeling Complex Gamesmentioning

confidence: 99%

“…Phantom has been successfully put to contribution in Ardon et al [1], Ganesh et al [6], Vadori et al [19] to study financial Over-The-Counter (OTC) markets. Studying this type of environment is particularly challenging for two main reasons.…”

Section: Application To Financementioning

confidence: 99%

“…Ardon et al [1] trained 25 OTC market participants in parallel by dividing them into 2 families: the Liquidity Providers (LP) and the Liquidity Takers (LT). All the agents from the same family share the same parametrized reward formulation, leading to different behaviors for each set of parameters or "type".…”

Section: Application To Financementioning

confidence: 99%

See 3 more Smart Citations

Phantom -- A RL-driven multi-agent framework to model complex systems

Ardon¹,

Vann²,

Garg³

et al. 2022

Preprint

View full text Add to dashboard Cite

Agent based modeling (ABM) is a computational approach to modeling complex systems by specifying the behavior of autonomous decision-making components or agents in the system and allowing the system dynamics to emerge from their interactions. Recent advances in the field of Multi-agent reinforcement learning (MARL) have made it feasible to learn the equilibrium of complex environments where multiple agents learn at the same time -opening up the possibility of building ABMs where agent behaviors are learned and system dynamics can be analyzed. However, most ABM frameworks are not RL-native, in that they do not offer concepts and interfaces that are compatible with the use of MARL to learn agent behaviors. In this paper, we introduce a new framework, Phantom, to bridge the gap between ABM and MARL. Phantom is an RL-driven framework for agent-based modeling of complex multiagent systems such as economic systems and markets. To enable this, the framework provides tools to specify the ABM in MARLcompatible terms -including features to encode dynamic partial observability, agent utility / reward functions, heterogeneity in agent preferences or types, and constraints on the order in which agents can act (e.g. Stackelberg games, or complex turn-taking environments). In this paper, we present these features, their design rationale and show how they were used to model and simulate Over-The-Counter (OTC) markets. CCS CONCEPTS• Software and its engineering → Application specific development environments; • Theory of computation → Multiagent reinforcement learning; Market equilibria.

show abstract

Section: Network Modelmentioning

confidence: 99%

Section: Modeling Complex Gamesmentioning

confidence: 99%

Section: Application To Financementioning

confidence: 99%

Section: Application To Financementioning

confidence: 99%

See 2 more Smart Citations

Phantom -- A RL-driven multi-agent framework to model complex systems

Ardon¹,

Vann²,

Garg³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Benchmarks of financial reinforcement learning: Many researches applied DRL algorithms in quantitative finance [38,68,69,5,3,12] by building their own market environments. Despite the above-mentioned open-source libraries that provide some useful environments, there are no established benchmarks yet.…”

Section: Introductionmentioning

confidence: 99%

FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning

Liu¹,

Xia²,

Rui³

et al. 2022

Preprint

View full text Add to dashboard Cite

Finance is a particularly difficult playground for deep reinforcement learning. However, establishing high-quality market environments and benchmarks for financial reinforcement learning is challenging due to three major factors, namely, low signal-to-noise ratio of financial data, survivorship bias of historical data, and model overfitting in the backtesting stage. In this paper, we present an openly accessible FinRL-Meta library that has been actively maintained by the AI4Finance community. First, following a DataOps paradigm, we will provide hundreds of market environments through an automatic pipeline that collects dynamic datasets from real-world markets and processes them into gym-style market environments. Second, we reproduce popular papers as stepping stones for users to design new trading strategies. We also deploy the library on cloud platforms so that users can visualize their own results and assess the relative performance via community-wise competitions. Third, FinRL-Meta provides tens of Jupyter/Python demos organized into a curriculum and a documentation website to serve the rapidly growing community.

show abstract