Learning unfair trading: A market manipulation analysis from the reinforcement learning perspective

Abstract

Abstract-Market manipulation is a strategy used by traders to alter the price of financial assets. One type of manipulation is based on the process of buying or selling assets by using several trading strategies, among them spoofing is a popular strategy and is considered illegal by market regulators. Some promising tools have been developed to detect manipulation, but cases can still be found in the markets. In this paper we model spoofing and pinging trading from a macroscopic perspective of profit maximisation, two strategies that differ in the legal background but share the same elemental concept of market manipulation. We use a reinforcement learning framework within the full and partial observability of Markov decision processes and analyse the underlying behaviour of the manipulators by finding the causes of what encourages the traders to perform fraudulent activities. Procedures can be applied to counter the problem as our model predicts the activity of the manipulators.

Bad machines corrupt good morals

“…As a testimony of this trend, thanks to recent breakthroughs in machine learning, algorithms now can establish and sustain cooperation with humans across multiple strategic situations 57,84 . Hence, we may be concerned that they collude with them and break ethical rules for mutual benefits, just as machines may engage in algorithmic collusion among themselves [85][86][87] . Since there are few behavioral insights into unethical behavior in hybrid human-machine teams 88 , much of this section is speculation.…”

Artificial Intelligence Crime: An Interdisciplinary Analysis of Foreseeable Threats and Solutions

“…Such relatively simple design leading to more complex behaviour is a core desideratum of MASs (Hildebrandt 2008, 7). In other cases, a designer may want to prevent emergence, such as when an autonomous trading agent inadvertently coordinates and colludes with other trading agents in furtherance of a shared goal (Martínez-Miranda et al 2016). Clearly, that emergent behaviour may have criminal implications, insofar as it misaligns with the original design.…”

“…This is because an AA, particularly one learning from real or simulated observations, may learn to generate signals that effectively mislead. (Wellman and Rajan 2017, 14) Simulation-based models of markets comprising artificial trading agents have shown (Martínez-Miranda et al 2016) that, through reinforcement learning, an AA can learn the technique of order-book spoofing. This involves placing orders with no intention of ever executing them and merely to manipulate honest participants in the marketplace.…”

“…This involves placing orders with no intention of ever executing them and merely to manipulate honest participants in the marketplace. (Lin 2017(Lin , 1289 In this case, the market manipulation emerged from an AA initially exploring the action space and, through exploration, placing false orders that became reinforced as a profitable strategy, and subsequently exploited for profit (Martínez-Miranda et al 2016). Further market exploitations, this time involving human intent, also include acquiring a position in a financial instrument, like a stock, then artificially inflating the stock through fraudulent promotion before selling its position to unsuspecting parties at the inflated price, which often crashes after the sale.…”

“…AI-fuelled crime may also impact commerce. In the second experiment, three computer scientists (Martínez-Miranda et al 2016) simulated a market and found that trading agents could learn and execute a "profitable" market manipulation campaign comprising a set of deceitful false-orders. These two experiments show that AI provides a feasible and fundamentally novel threat, in the form of AIC.…”