Multi-agent reinforcement learning approach for hedging portfolio problem

Abstract

Developing a hedging strategy to reduce risk of losses for a given set of stocks in a portfolio is a difficult task due to cost of the hedge. In Vietnam stock market, cross-hedge is involved hedging a long position of a stock because there is no put option for the stock. In addition, only VN30 stock index futures contracts are traded on Hanoi Stock Exchange. Inspired by recently achievement of deep reinforcement learning, we explore feasibility to construct a hedging strategy automatically by leveraging cooperative multi-agent in reinforcement learning techniques without advanced domain knowledge. In this work, we use 10 popular stocks on Ho Chi Minh Stock Exchange, and VN30F1M (VN30 Index Futures contracts within one month settlement) to develop a stock market simulator (including transaction fee, tax, and settlement date of transactions) for reinforcement learning agent training. We use daily return as input data for training process. Results suggest that the agent can learn trading and hedging policy to make profit and reduce losses. Furthermore, we also find that our agent can protect portfolios and make positive profit in case market collapses systematically. In practice, this work can help Vietnam’s stock market investors to improve performance and reduce losses in trading, especially when the volatility cannot be controlled.

Deep Reinforcement Learning for Dynamic Stock Option Hedging: A Review

“…A final method used in the analyzed literature is importance weighted actor-learner architecture (IMPALA) [53]. The IMPALA method was first introduced in [67], which describes IMPALA as a distributed DRL technique wherein multiple actors operate in parallel.…”

“…Ref. [53] cites [67] in saying that the IMPALA method enables for better data efficiency and stability.…”

“…Refs. [49,53,54] also frame the action at each time step as the buying of some discrete number of shares.…”

“…Refs. [53,55] both look ahead to the end of an episode before computing the reward. In [55], the episode reward is the difference between the cash held at expiry and the payoff of the short position in the call option (they are short, so the loss is (S T − K) + ).…”

“…In [55], the episode reward is the difference between the cash held at expiry and the payoff of the short position in the call option (they are short, so the loss is (S T − K) + ). In [53], entire episodes are simulated, and the hedging agent is rewarded a +1 if the profit is positive and −1 if the profit is negative. Ref.…”