The Metalog Distributions: Virtually Unlimited Shape Flexibility, Combining Expert Opinion in Closed Form, and Bayesian Updating in Closed Form

Abstract

Users of probability distributions frequently need to convert data (empirical, simulated, or elicited) into a continuous probability distribution and to update that distribution when new data becomes available. Often, it is unclear which traditional probability distribution(s) to use, fitting to data is laborious and unsatisfactory, little insight emerges, and updating with Bayes rule is impractical. Here we offer an alternative -- a family of continuous probability distributions, fitting methods, and tools that: provide sufficient shape and boundedness flexibility to closely match virtually any probability distribution and most data sets; involve a single set of simple closed-form equations; stimulate potentially valuable insights when applied to empirical data; are simply fit to data with ordinary least squares; are easy to combine (as when weighting the opinion of multiple experts), and, under certain conditions, are easily updated in closed form according to Bayes rule when new data becomes available. The Bayesian updating method is presented in a way that is readily understandable as a fisherman updates his catch probabilities when changing the river on which he fishes. While metalog applications have been shown to improve decision-making, the methods and results herein are broadly applicable to virtually any use of continuous probability in any field of human endeavor. Diverse data sets may be explored and modeled in these new ways with freely available spreadsheets and tools.

A Method of Assessing the Selection of Carport Power for an Electric Vehicle Using the Metalog Probability Distribution Family

“…Process planning in controlled conditions deviating from the natural, expensive data collection and statistical analysis are carried out offline, and the results obtained are no longer real-time results. If the amount of data increases, you can raise the order of the matched Metalog [52].…”

“…The Metalog family of distributions allows you to determine the percentiles in the production of electricity by the carport and answer the question of what its value will be with the accuracy of the probability distribution. Using the Metalog family of distributions, we obtain information from a knowledge base, not from a database [52]. The difference is essential: in a database, the answer to the question asked is obtained as a result of searching the database; in the case of a knowledge base, the answer to the question is obtained as a result of running an inference algorithm.…”

“…The mathematical description of the Metalog family of probability distributions is presented in the work of its creators. Keelin and Howard provide a detailed description in their article [52] (Appendix A on page 17). They also run a website on which the principles of the operation of the software that is created on the basis of this method are described in detail [53].…”

Introducing the Metalog Distributions

“…By increasing the number of terms, the metalog has been shown to have virtually unlimited shape flexibility. 3 Since the metalog quantile function is differentiable, it has a simple closed-form probability density function (PDF), the shape of which also depends on the a-coefficients. 1 To be a feasible probability distribution, the a-coefficients must be such that this PDF is positive for all y.…”

“…1 Moreover, Bayesian linear regression 4 can be used to update, in light of new data, a metalog-distributed long-run frequency distribution over a variable of interest according to Bayes' theorem in closed form. 3 Simple transformations of the metalog quantile function yield semi-bounded and bounded metalog distributions, 1 where the user can set upper and/or lower bounds as appropriate. Beyond honouring such user-specified bounds, these metalog transforms retain the properties of virtually unlimited shape flexibility and determination of a-coefficients by linear regression.…”