Entropy Ensemble Filter: A Modified Bootstrap Aggregating (Bagging) Procedure to Improve Efficiency in Ensemble Model Simulation

Abstract

Over the past two decades, the Bootstrap AGGregatING (bagging) method has been widely used for improving simulation. The computational cost of this method scales with the size of the ensemble, but excessively reducing the ensemble size comes at the cost of reduced predictive performance. The novel procedure proposed in this study is the Entropy Ensemble Filter (EEF), which uses the most informative training data sets in the ensemble rather than all ensemble members created by the bagging method. The results of this study indicate efficiency of the proposed method in application to synthetic data simulation on a sinusoidal signal, a sawtooth signal, and a composite signal. The EEF method can reduce the computational time of simulation by around 50% on average while maintaining predictive performance at the same level of the conventional method, where all of the ensemble models are used for simulation. The analysis of the error gradient (root mean square error of ensemble averages) shows that using the 40% most informative ensemble members of the set initially defined by the user appears to be most effective.

The Tsallis entropy and the BKT-like phase transition in the impact parameter space for pp and $\bar{p}p$ collisions

“…The principle of maximum entropy states that the probability function correctly describing a dataset is the one with the largest entropy S. The entropy (12) is about a particular scattering at some fixed-s. To each s i one can construct a dataset taking the pair 0, G inel (s i , b(s i )) , i.e. the line contained in [0, b(s i )].…”

“…The TE (12) can also be related to the amount of information in the area of width k −1 and the curve given by m 1− nG inel (s, b) depending on each s used. Of course, m 1 − nG inel (s, b) is limited to the range ∀ i : 0, G inel (s i , b(s i )) , and, therefore, this area assumes a finite value as well as the amount of information one can obtain from it.…”

From Physics to Bioengineering: Microbial Cultivation Process Design and Feeding Rate Control Based on Relative Entropy Using Nuisance Time

“…By comparison, the authors of [ 48 ] state that “if a model has many free parameters—for instance, a complex budget constraint or complex household preferences—then the model is relatively nonparsimonious”. Such a statement is not relevant, and overfitting issues [ 52 ] do not occur in the scope of ME optimization for the profile fitting. On the contrary, both parsimony and multiple parameters helped to arrive at a competitive local extremum, which has equivalent goodness of fit compared to others extremum candidates.…”

Objective functions for information-theoretical monitoring network design: what is optimal?

“…Once an event is observed, and which of the m classes it belongs to is identified, our uncertainty about the outcome decreases to 0. Therefore, information can be characterized as decrease an observer's uncertainty about the outcome (Krstanovic and Singh, 1992;Mogheir et al, 2006;Samuel et al, 2013;Foroozand and Weijs, 2017;Foroozand et al, 2018). For monitoring networks, the information each sensor provides through its observations (outcomes) is therefore linked to the uncertainty of those outcomes before measurement.…”