Analytic approach to variance optimization under an ℓ1 constraint

Kondor, Imre; Papp, Gábor; Caccioli, Fabio

doi:10.1140/epjb/e2018-90456-2

Cited by 4 publications

(15 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, from ( 20 ) we find

Combined with the previous equation, this gives

The “true” (

) value of the order parameter

is thus determined by the structural constant

, which is given by the variances of the returns

. This is in accord with the corresponding result found in the case of the

-regularized variance risk measure [ 21 , 29 ]. The above result for

also means that the quantity

introduced in ( 7 ) is equal to 1, and according to ( 8 ) the out-of-sample estimate of ES is equal to its true value

, the estimation error is zero—an obvious result for the case of complete information.…”

Section: Resultssupporting

confidence: 92%

“…In our present work, the large

limit and the averaging over infinitely many samples result in a continuous dependence of the “condensate” density (the relative number

of the dimensions eliminated by

) on the aspect ratio r , the confidence level

, and the strength of

. In a study of

-regularized variance [ 21 ], we found that the stepwise increase of the density of eliminated weights in a numerical experiment nicely follows the continuous curve obtained analytically. It is obvious that the situation is similar in the case of ES, but we have also confirmed this by numerical simulations.…”

Section: Methods and Preliminariesmentioning

confidence: 62%

“…The replica method used below have already been applied with minor variations to various portfolio optimization problems in a number of papers [ 10 , 11 , 12 , 14 , 18 , 21 , 25 , 26 , 27 , 28 ], where the replica derivation of the main formulae were repeatedly explained, so we do not need to go through that exercise again here. Then the natural starting point for our present work is the detailed study of the behavior of ES without regularization in [ 12 ].…”

Section: Methods and Preliminariesmentioning

confidence: 99%

“…The third order parameter,

, is of central importance: According to [ 12 ], the ratio of the out-of-sample estimate

and its true value

is given by the square root of

. For the case of different

s considered here,

has to be amended by a factor depending on the structure of the portfolio [ 21 ] as

Then the ratio of the estimated and true ES will be

that is the relative estimation error is

.…”

Section: Methods and Preliminariesmentioning

confidence: 99%

See 3 more Smart Citations

Optimizing Expected Shortfall under an ℓ1 Constraint—An Analytic Approach

Papp

Kondor

Caccioli

2021

Entropy

Self Cite

View full text Add to dashboard Cite

Expected Shortfall (ES), the average loss above a high quantile, is the current financial regulatory market risk measure. Its estimation and optimization are highly unstable against sample fluctuations and become impossible above a critical ratio r=N/T, where N is the number of different assets in the portfolio, and T is the length of the available time series. The critical ratio depends on the confidence level α, which means we have a line of critical points on the α−r plane. The large fluctuations in the estimation of ES can be attenuated by the application of regularizers. In this paper, we calculate ES analytically under an ℓ1 regularizer by the method of replicas borrowed from the statistical physics of random systems. The ban on short selling, i.e., a constraint rendering all the portfolio weights non-negative, is a special case of an asymmetric ℓ1 regularizer. Results are presented for the out-of-sample and the in-sample estimator of the regularized ES, the estimation error, the distribution of the optimal portfolio weights, and the density of the assets eliminated from the portfolio by the regularizer. It is shown that the no-short constraint acts as a high volatility cutoff, in the sense that it sets the weights of the high volatility elements to zero with higher probability than those of the low volatility items. This cutoff renormalizes the aspect ratio r=N/T, thereby extending the range of the feasibility of optimization. We find that there is a nontrivial mapping between the regularized and unregularized problems, corresponding to a renormalization of the order parameters.

show abstract

“…Then, from ( 20 ) we find

Combined with the previous equation, this gives

The “true” (

) value of the order parameter

is thus determined by the structural constant

, which is given by the variances of the returns

. This is in accord with the corresponding result found in the case of the

-regularized variance risk measure [ 21 , 29 ]. The above result for

also means that the quantity

introduced in ( 7 ) is equal to 1, and according to ( 8 ) the out-of-sample estimate of ES is equal to its true value

, the estimation error is zero—an obvious result for the case of complete information.…”

Section: Resultssupporting

confidence: 92%

“…In our present work, the large

limit and the averaging over infinitely many samples result in a continuous dependence of the “condensate” density (the relative number

of the dimensions eliminated by

) on the aspect ratio r , the confidence level

, and the strength of

. In a study of

Section: Methods and Preliminariesmentioning

confidence: 62%

Section: Methods and Preliminariesmentioning

confidence: 99%

“…The third order parameter,

, is of central importance: According to [ 12 ], the ratio of the out-of-sample estimate

and its true value

is given by the square root of

. For the case of different

s considered here,

has to be amended by a factor depending on the structure of the portfolio [ 21 ] as

Then the ratio of the estimated and true ES will be

that is the relative estimation error is

.…”

Section: Methods and Preliminariesmentioning

confidence: 99%

See 2 more Smart Citations

Optimizing Expected Shortfall under an ℓ1 Constraint—An Analytic Approach

Papp

Kondor

Caccioli

2021

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…If we interchange t and i , we see that according to ( 11 ), this is possible as long as the N points in

with position vectors

do not form a dichotomy. Hence, the probability for zero variance is from ( 2 )

Therefore, the probability of the variance vanishing is almost 1 for small

, decreases to the value 1/2 at

, decreases further to 0 as

increases to 1, and remains identically zero for

[ 30 , 31 ]. This is similar but also somewhat complementary to the curve shown in Figure 2 .…”

Section: Phase Transitions In Portfolio Optimization Under the Variance And The Maximal Loss Risk Measurementioning

confidence: 99%

Aspects of a Phase Transition in High-Dimensional Random Geometry

Prüser

Kondor

Engel

2021

Entropy

Self Cite

View full text Add to dashboard Cite

A phase transition in high-dimensional random geometry is analyzed as it arises in a variety of problems. A prominent example is the feasibility of a minimax problem that represents the extremal case of a class of financial risk measures, among them the current regulatory market risk measure Expected Shortfall. Others include portfolio optimization with a ban on short-selling, the storage capacity of the perceptron, the solvability of a set of linear equations with random coefficients, and competition for resources in an ecological system. These examples shed light on various aspects of the underlying geometric phase transition, create links between problems belonging to seemingly distant fields, and offer the possibility for further ramifications.

show abstract

Optimizing Expected Shortfall under an $\ell_1$ constraint -- an analytic approach

Papp,

Kondor,

Caccioli

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Expected Shortfall (ES), the average loss above a high quantile, is the current financial regulatory market risk measure. Its estimation and optimization are highly unstable against sample fluctuations and become impossible above a critical ratio r = N/T , where N is the number of different assets in the portfolio, and T is the length of the available time series. The critical ratio depends on the confidence level α, which means we have a line of critical points on the α − r plane. The large fluctuations in the estimation of ES can be attenuated by the application of regularizers. In this paper, we calculate ES analytically under an 1 regularizer by the method of replicas borrowed from the statistical physics of random systems. The ban on short selling, i.e. a constraint rendering all the portfolio weights non-negative, is a special case of an asymmetric 1 regularizer. Results are presented for the out-of-sample and the insample estimator of the regularized ES, the estimation error, the distribution of the optimal portfolio weights and the density of the assets eliminated from the portfolio by the regularizer. It is shown that the no-short constraint acts as a high volatility cutoff, in the sense that it sets the weights of the high volatility elements to zero with higher probability than those of the low volatility items. This cutoff renormalizes the aspect ratio r = N/T , thereby extending the range of the feasibility of optimization. We find that there is a nontrivial mapping between the regularized and unregularized problems, corresponding to a renormalization of the order parameters.

show abstract

Analytic approach to variance optimization under an ℓ1 constraint

Cited by 4 publications

References 32 publications

Optimizing Expected Shortfall under an ℓ1 Constraint—An Analytic Approach

Optimizing Expected Shortfall under an ℓ1 Constraint—An Analytic Approach

Aspects of a Phase Transition in High-Dimensional Random Geometry

Optimizing Expected Shortfall under an $\ell_1$ constraint -- an analytic approach

Contact Info

Product

Resources

About