Weighted Entropy and its Use in Computer Science and Beyond

Kelbert, Mark; Stuhl, Izabella; Suhov, Yuri

doi:10.1007/978-3-319-71504-9_25

Cited by 9 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…H w (w1,….wn, wn+1, p1,….pn, 0) = H w (w1,….wn, wn+1, p1,….pn) = H w (X) for any wn+1 The properties of the weighted entropy apply only to the discrete case. In the continuous case there are difficulties discussed in (Kelbert, Stuhl, & Suhov, 2017). On the other hand, the continuous or differential version can also be implemented quite easily, provided adequate precautions are taken to double-check the consistency of the results.…”

Section: A Weighting the Various Parts Of The Pdfmentioning

confidence: 99%

A Practical Utility-Based but Objective Approach to Model Selection for Scientific Applications in the Age of Big Data

Murari¹,

Rossi

Spolladore

et al. 2023

Preprint

View full text Add to dashboard Cite

In many fields of science, various types of models are available to describe phenomena, observations and the results of experiments. In the last decades, given the enormous advances of information gathering technologies, also machine learning techniques have been systematically deployed to extract models from the large available databases. However, regardless of their origins, no universal criterion has been found so far to select the most appropriate model given the data. A unique solution is probably a chimera, particularly in applications involving complex systems. Consequently, in this work a utility-based approach is advocated. However, the solutions proposed are not purely subjective but all based on “objective” criteria, rooted in the properties of the data, to preserve generality and to allow comparative assessments of the results. Several methods have been developed and tested, to improve the discrimination capability of basic Bayesian and information theoretic criteria. Both the quality of the fits and the evaluation of model complexity are aspects addressed by the advances proposed. The competitive advantages of the individual alternatives, for both cross sectional data and time series, are clearly identified, together with their most appropriate fields of application. Particular attention has been paid to ensure that the proposed improvements of the criteria are easy to implement in practice, in both confirmatory and exploratory settings. Extensive numerical tests have been performed to support the conceptual and theoretical considerations.

show abstract

Section: A Weighting the Various Parts Of The Pdfmentioning

confidence: 99%

A Practical Utility-Based but Objective Approach to Model Selection for Scientific Applications in the Age of Big Data

Murari¹,

Rossi

Spolladore

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In our opinion, the objective of achieving maximum entropy in RL is to learn a nearly optimal solution which preserves multi-modality. Our work is motivated by weighted entropy(WE) [22,34], which takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function [35]. Zhao et al [36] proposed the similar idea of maximizing the entropy weighted by rewards, whereas we aim to design more arbitrary weight function and more flexible algorithms.…”

Section: Related Workmentioning

confidence: 99%

Weighted Entropy Modification for Soft Actor-Critic

Zhao

Zhu

2020

Preprint

View full text Add to dashboard Cite

We generalize the existing principle of the maximum Shannon entropy in reinforcement learning (RL) to weighted entropy by characterizing the stateaction pairs with some qualitative weights, which can be connected with prior knowledge, experience replay and evolution process of the policy. We propose an algorithm motivated for self-balancing exploration with the introduced weight function, which leads to state-of-the-art performance on Mujoco tasks despite its simplicity in implementation.

show abstract

“…This paper represents an extended version of an earlier note [10]. 1 We also follow earlier publications discussing related topics: [20,21,19,18].…”

Section: Introductionmentioning

confidence: 98%

“…

…”

mentioning

confidence: 99%

Weighted entropy: basic inequalities

Kelbert¹,

Stuhl²,

Suhov³

2017

Modern Stoch. Theory Appl.

Self Cite

View full text Add to dashboard Cite

This paper represents an extended version of an earlier note [10]. The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy contextdependent, through the weight function. We analyse analogs of the Fisher information inequality and entropy power inequality for the weighted entropy and discuss connections with weighted Lieb's splitting inequality. The concepts of rates of the weighted entropy and information are also discussed.Keywords Weighted entropy, Gibbs inequality, Ky-Fan inequality, Fisher information inequality, entropy power inequality, Lieb's splitting inequality, rates of weighted entropy and information 2010 MSC 94A17

show abstract

Weighted Entropy and its Use in Computer Science and Beyond

Cited by 9 publications

References 16 publications

A Practical Utility-Based but Objective Approach to Model Selection for Scientific Applications in the Age of Big Data

A Practical Utility-Based but Objective Approach to Model Selection for Scientific Applications in the Age of Big Data

Weighted Entropy Modification for Soft Actor-Critic

Weighted entropy: basic inequalities

Contact Info

Product

Resources

About