Category Theory in Machine Learning

Shiebler, Dan; Gavranović, Bruno; Wilson, Paul

doi:10.48550/arxiv.2106.07032

Cited by 9 publications

(9 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They do use the Para construction, but do not relate it to lenses nor reverse derivative categories. A general survey of category theoretic approaches to machine learning, covering many of the above papers, can be found in [43]. Lastly, the concept of parametric lenses has started appearing in recent formulations of categorical game theory and cybernetics [9,10].…”

Section: Related Workmentioning

confidence: 99%

Categorical Foundations of Gradient-Based Learning

Cruttwell

Gavranović

Ghani

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We propose a categorical semantics of gradient-based machine learning algorithms in terms of lenses, parametric maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it encompasses a variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions such as MSE and Softmax cross-entropy, shedding new light on their similarities and differences. Our approach to gradient-based learning has examples generalising beyond the familiar continuous domains (modelled in categories of smooth maps) and can be realized in the discrete setting of boolean circuits. Finally, we demonstrate the practical significance of our framework with an implementation in Python.

show abstract

Section: Related Workmentioning

confidence: 99%

Categorical Foundations of Gradient-Based Learning

Cruttwell

Gavranović

Ghani

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…While many authors have explored how an applied category theoretic perspective can help exploit structure and invariance in machine learning [24], relatively few authors have explored applications of Kan extensions to data science and machine learning. That said, some authors have begun to explore Kan extension structure in topological data analysis.…”

Section: Related Workmentioning

confidence: 99%

Kan Extensions in Data Science and Machine Learning

Shiebler¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

A common problem in data science is "use this function defined over this small set to generate predictions over that larger set. " Extrapolation, interpolation, statistical inference and forecasting all reduce to this problem. The Kan extension is a powerful tool in category theory that generalizes this notion. In this work we explore several applications of Kan extensions to data science. We begin by deriving a simple classification algorithm as a Kan extension and experimenting with this algorithm on real data. Next, we use the Kan extension to derive a procedure for learning clustering algorithms from labels and explore the performance of this procedure on real data. We then investigate how Kan extensions can be used to learn a general mapping from datasets of labeled examples to functions and to approximate a complex function with a simpler one.

show abstract

“…It helps to reconcile the expressiveness of descriptive models and the restrictiveness of mathematical ones. This formalism has already been successfully used to address some challenges of computer science, for example to build a general framework for the specification of concurrent systems [14] or to allow the compositionality of machine learning components [46]. To the best of our knowledge, it has never been studied as foundation for a complete formal framework for data lakes.…”

Section: Contributions Of Category Theory To Data Lakesmentioning

confidence: 99%

A Formal Framework for Data Lakes Based on Category Theory

Guyot

Gillet

Leclercq

et al. 2022

International Database Engineered Applications Symposium

View full text Add to dashboard Cite

The management of Big Data requires flexible systems to handle the heterogeneity of data models as well as the complexity of analytical workflows. Traditional systems like data warehouses have reached their limits due to their rigid schema-on-write paradigm, that requires well identified and defined use cases to ingest data. Data lakes, with their schema-on-read paradigm, have been proposed as more flexible systems in which raw data are directly stored in their original format associated with metadata, to be accessed and transformed only when users need to process or analyze them. Thus, it is necessary to define and control the different levels of abstraction and the dependencies among functionalities of a data lake to use it efficiently. In this article, we present a formal framework aiming to define a data lake pattern and to unify the interactions among the functionalities. We use the category theory as theoretical foundations to benefit from its high level of abstraction and its compositionality. By relying on different categories and functors, we ensure the navigation among the functionalities and allow the composition of multiples operations, while keeping track of the entire lineage of data. We also show how our framework can be applied on a simple example of data lake.

show abstract

Category Theory in Machine Learning

Cited by 9 publications

References 46 publications

Categorical Foundations of Gradient-Based Learning

Categorical Foundations of Gradient-Based Learning

Kan Extensions in Data Science and Machine Learning

A Formal Framework for Data Lakes Based on Category Theory

Contact Info

Product

Resources

About