Streaming generalized cross entropy

Abstract

We propose a new method to combine adaptive processes with a class of entropy estimators for the case of streams of data.Starting from a first estimation obtained from a batch of initial data, at each step the parameters of the model are estimated combining the prior knowledge and the new observation (or a block of observations). This allows extending the maximum entropy technique to a dynamical setting distinguishing between entropic contributions of the signal and the error. Furthermore, it gives a suitable approximation of standard GME problems when exact solutions are hard to evaluate.We test this method performing numerical simulations at various sample sizes and batch dimensions. Moreover, we explore intermediate cases between streaming GCE and standard GCE, namely, when the update of estimations involves blocks of observations of different sizes, and we include collinearity effects. Finally, we discuss the results to highlight the main characteristics of this method, the range of application, and future perspectives.

A Data-Weighted Prior Estimator for Forecast Combination

“…The weight (a weighted formulation in an entropy optimization problem has been also proposed by [ 45 ] who proposed a weighted generalized maximum entropy (W-GME) estimator where different weights are assigned to the two entropies (for coefficient distributions and disturbance distributions) in the objective problem. Moreover, under a linear regression model estimation, [ 46 ] proposed a streaming generalized cross entropy (Stre-GCE) method to update the estimation of the parameters by combining prior information and new data) given to the spike prior for each parameter is given by . For each , a discrete support space is specified with n possible values ( and corresponding probability distribution .…”