Bayesian machine learning via category theory

Culbertson, Jared; Sturtz, Kirk

doi:10.48550/arxiv.1312.1445

Cited by 5 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, while Meas is symmetric monoidal, it is not Cartesian closed, since the space of measurable maps into spaces of measurable maps is not always measurable. Culbertson and Sturtz (2013) attempts to address this problem by simply equipping Meas with a different monoidal product. Heunen et al (2017) use a different strategy, and instead generalize measurable spaces to quasi-Borel spaces QBS, which are Cartesian closed (Heunen et al, 2017, Prop.…”

Section: Cartesian Closedness and Quasi-borel Spacesmentioning

confidence: 99%

“…This enables Culbertson and Sturtz (2013) to define a Bayesian model to be a diagram that consists of the following components:…”

Section: Bayes Law; Concretelymentioning

confidence: 99%

“…In order to solve this, Culbertson and Sturtz (2013) define a new monoidal product such that Meas becomes closed. This is in contrast to the solution in "A Convenient Category for Higher-Order Probability Theory" (Heunen et al, 2017), where the authors choose to instead redefine Meas itself.…”

Section: Bayes Law; Concretelymentioning

confidence: 99%

See 2 more Smart Citations

Category Theory in Machine Learning

Shiebler,

Gavranović,

Wilson

2021

Preprint

View full text Add to dashboard Cite

Over the past two decades machine learning has permeated almost every realm of technology. At the same time, many researchers have begun using category theory as a unifying language, facilitating communication between different scientific disciplines. It is therefore unsurprising that there is a burgeoning interest in applying category theory to machine learning. We aim to document the motivations, goals and common themes across these applications. We touch on gradient-based learning, probability, and equivariant learning.

show abstract

Section: Cartesian Closedness and Quasi-borel Spacesmentioning

confidence: 99%

“…This enables Culbertson and Sturtz (2013) to define a Bayesian model to be a diagram that consists of the following components:…”

Section: Bayes Law; Concretelymentioning

confidence: 99%

See 1 more Smart Citation

Category Theory in Machine Learning

Shiebler,

Gavranović,

Wilson

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…[41,42]). Now we recall the direct association between Markov kernels and conditional probabilities, and apply the above embedding to the category P of conditional probabilities, following mainly [130,131] (cf. [132,133]).…”

Section: Asymptotic Behavior Of the Fepmentioning

confidence: 99%

A free energy principle for generic quantum systems

Fields,

Friston,

Glazebrook

et al. 2021

Preprint

View full text Add to dashboard Cite

The Free Energy Principle (FEP) states that under suitable conditions of weak coupling, random dynamical systems with sufficient degrees of freedom will behave so as to minimize an upper bound, formalized as a variational free energy, on surprisal (a.k.a., selfinformation). This upper bound can be read as a Bayesian prediction error. Equivalently, its negative is a lower bound on Bayesian model evidence (a.k.a., marginal likelihood). In short, certain random dynamical systems evince a kind of self-evidencing. Here, we reformulate the FEP in the formal setting of spacetime-background free, scale-free quantum information theory. We show how generic quantum systems can be regarded as observers, which with the standard freedom of choice assumption become agents capable of assigning semantics to observational outcomes. We show how such agents minimize Bayesian prediction error in environments characterized by uncertainty, insufficient learning, and quantum contextuality. We show that in its quantum-theoretic formulation, the FEP is asymptotically equivalent to the Principle of Unitarity. Based on these results, we suggest that biological systems employ quantum coherence as a computational resource and -implicitly -as a communication resource. We summarize a number of problems for future research,

show abstract

“…We aim to formalize these ideas in a way that will be understandable by statisticians and machine learning practitioners who have no prior knowledge of category theory. Category theory has been applied to statistics and machine learning before [1,2,3], and invariance of data representations has been discussed in [4], however to our knowledge the current framework has not yet been described. Introductions to category theory for applied scientists can be found in [5,6], and for mathematicians in [7,8].…”

Section: Introductionmentioning

confidence: 99%

Characterizing the invariances of learning algorithms using category theory

Harris

2019

Preprint

View full text Add to dashboard Cite

Many learning algorithms have invariances: when their training data is transformed in certain ways, the function they learn transforms in a predictable manner. Here we formalize this notion using concepts from the mathematical field of category theory. The invariances that a supervised learning algorithm possesses are formalized by categories of predictor and target spaces, whose morphisms represent the algorithm's invariances, and an index category whose morphisms represent permutations of the training examples. An invariant learning algorithm is a natural transformation between two functors from the product of these categories to the category of sets, representing training datasets and learned functions respectively. We illustrate the framework by characterizing and contrasting the invariances of linear regression and ridge regression.

show abstract

Bayesian machine learning via category theory

Cited by 5 publications

References 0 publications

Category Theory in Machine Learning

Category Theory in Machine Learning

A free energy principle for generic quantum systems

Characterizing the invariances of learning algorithms using category theory

Contact Info

Product

Resources

About