The BisimDist Library: Efficient Computation of Bisimilarity Distances for Markovian Models

Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand; Mardare, Radu

doi:10.1007/978-3-642-40196-1_23

Cited by 8 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we evaluate the sampling-based L * mdp and compare it to the passive IoAlergia [MCJ + 16], by learning several models with both techniques. In each case, we treat the known true MDP model M as a black box for learning and measure similarity to this model using two criteria: 1. bisimilarity distance: we compute the discounted bisimilarity distance between the true model M and the learned MDPs [BBLM13b,BBLM13a]. We adapted this distance measure from MDPs with rewards to labelled MDPs, by defining a distance of 1 between states with different labels.…”

Section: Methodsmentioning

confidence: 99%

L∗-based learning of Markov decision processes (extended version)

et al. 2021

Self Cite

View full text Add to dashboard Cite

Automata learning techniques automatically generate systemmodels fromtest observations. Typically, these techniques fall into two categories: passive and active. On the one hand, passive learning assumes no interaction with the system under learning and uses a predetermined training set, e.g., system logs. On the other hand, active learning techniques collect training data by actively querying the system under learning, allowing one to steer the discovery ofmeaningful information about the systemunder learning leading to effective learning strategies. A notable example of active learning technique for regular languages is Angluin’s $$L^*$$ L ∗ -algorithm. The $$L^*$$ L ∗ -algorithm describes the strategy of a student who learns the minimal deterministic finite automaton of an unknown regular language $$L$$ L by asking a succinct number of queries to a teacher who knows $$L$$ L .In this work, we study $$L^*$$ L ∗ -based learning of deterministic Markov decision processes, a class of Markov decision processes where an observation following an action uniquely determines a successor state. For this purpose, we first assume an ideal setting with a teacher who provides perfect information to the student. Then, we relax this assumption and present a novel learning algorithm that collects information by sampling execution traces of the system via testing.Experiments performed on an implementation of our sampling-based algorithm suggest that our method achieves better accuracy than state-of-the-art passive learning techniques using the same amount of test obser vations. In contrast to existing learning algorithms which assume a predefined number of states, our algorithm learns the complete model structure including the state space.

show abstract

Section: Methodsmentioning

confidence: 99%

L∗-based learning of Markov decision processes (extended version)

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…1. We compute the discounted bisimilarity distance between the true models and the learned MDPs [7,8]. We adapted the distance measure from MDPs with rewards to labelled MDPs by defining a distance of 1 between states with different labels.…”

Section: Methodsmentioning

confidence: 99%

$$L^*$$-Based Learning of Markov Decision Processes

Tappler

Aichernig

Bacci

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Automata learning techniques automatically generate system models from test observations. These techniques usually fall into two categories: passive and active. Passive learning uses a predetermined data set, e.g., system logs. In contrast, active learning actively queries the system under learning, which is considered more efficient. An influential active learning technique is Angluin's L * algorithm for regular languages which inspired several generalisations from DFAs to other automata-based modelling formalisms. In this work, we study L *based learning of deterministic Markov decision processes, first assuming an ideal setting with perfect information. Then, we relax this assumption and present a novel learning algorithm that collects information by sampling system traces via testing. Experiments with the implementation of our sampling-based algorithm suggest that it achieves better accuracy than state-of-the-art passive learning techniques with the same amount of test data. Unlike existing learning algorithms with predefined states, our algorithm learns the complete model structure including the states.

show abstract

“…On the one hand, branching distances, e.g. [1,12,25,24,4,3,2,17], lift the equivalence given by the probabilistic bisimulation of Larsen and Skou [21]. On the other hand, there are linear distances, in particular the total variation distance [8,6] and trace distances [19,5].…”

Section: Introductionmentioning

confidence: 99%

Linear Distances between Markov Chains

Daca¹,

Henzinger²,

Křetínský³

et al. 2016

Preprint

View full text Add to dashboard Cite

We introduce a general class of distances (metrics) between Markov chains, which are based on linear behaviour. This class encompasses distances given topologically (such as the total variation distance or trace distance) as well as by temporal logics or automata. We investigate which of the distances can be approximated by observing the systems, i.e. by black-box testing or simulation, and we provide both negative and positive results.

show abstract

The BisimDist Library: Efficient Computation of Bisimilarity Distances for Markovian Models

Cited by 8 publications

References 9 publications

L∗-based learning of Markov decision processes (extended version)

L∗-based learning of Markov decision processes (extended version)

$$L^*$$-Based Learning of Markov Decision Processes

Linear Distances between Markov Chains

Contact Info

Product

Resources

About