Categorical Aspects of Parameter Learning

Abstract

Parameter learning is the technique for obtaining the probabilistic parameters in conditional probability tables in Bayesian networks from tables with (observed) data -where it is assumed that the underlying graphical structure is known. There are basically two ways of doing so, referred to as maximal likelihood estimation (MLE) and as Bayesian learning. This paper provides a categorical analysis of these two techniques and describes them in terms of basic properties of the multiset monad M, the distribution monad D and the Giry monad G. In essence, learning is about the reltionships between multisets (used for counting) on the one hand and probability distributions on the other. These relationsips will be described as suitable natural transformations.

Category Theory in Machine Learning

“…Jacobs (Jacobs, 2018) describes a similar framework for Bayesian updates to a prior over a joint distribution. He focuses on multinomial distributions and treats multisets with n unique elements as the parameter vectors for n-category multinomial distributions.…”