From deterministic to stochastic: an interpretable stochastic model-free reinforcement learning framework for portfolio optimization

Abstract

As a fundamental problem in algorithmic trading, portfolio optimization aims to maximize the cumulative return by continuously investing in various financial derivatives within a given time period. Recent years have witnessed the transformation from traditional machine learning trading algorithms to reinforcement learning algorithms due to their superior nature of sequential decision making. However, the exponential growth of the imperfect and noisy financial data that is supposedly leveraged by the deterministic strategy in reinforcement learning, makes it increasingly challenging for one to continuously obtain a profitable portfolio. Thus, in this work, we first reconstruct several deterministic and stochastic reinforcement algorithms as benchmarks. On this basis, we introduce a risk-aware reward function to balance the risk and return. Importantly, we propose a novel interpretable stochastic reinforcement learning framework which tailors a stochastic policy parameterized by Gaussian Mixtures and a distributional critic realized by quantiles for the problem of portfolio optimization. In our experiment, the proposed algorithm demonstrates its superior performance on U.S. market stocks with a 63.1% annual rate of return while at the same time reducing the market value max drawdown by 10% when back-testing during the stock market crash around March 2020.

A Multiscale Recursive Attention Gate Federation Method for Multiple Working Conditions Fault Diagnosis

“…F g,2 is then distributed to each client for local multiscale feature fusion. In such a way, the aggregated features F g,n at the n-th scale are obtained, and F g,n is sent down to the client for local multiscale feature fusion to obtain the fused features F f ,i,n , as shown in Equation (5).…”

“…When the data quality is better, the spectrum analysis can find the characteristic frequency of the fault and realize fault diagnosis. However, when the data quality is poor, the feature frequency of the fault is difficult to present on the spectrum [ 4 , 5 ].…”

Management of investment portfolios employing reinforcement learning

“… Song et al (2022) introduced an innovative method for optimizing investment portfolios utilizing stochastic reinforcement learning. To validate their model, the authors conducted an empirical analysis using data from 22 stocks with the highest trading volume in the S&P 500 index from 2005 to 2020.…”

“…The financial markets display a high degree of dynamism and complexity, making the selection of an optimal combination of assets for constructing an investment portfolio a formidable challenge ( Song et al, 2022 ; Xiao & Ihnaini, 2023 among others). In this context, scholars have conducted thorough investigations into Modern Portfolio Theory (MPT) of Markowitz (1952) since its inception.…”