AALpy: an active automata learning library

Muškardin, Edi; Aichernig, Bernhard K.; Pill, Ingo; Pferscher, Andrea; Tappler, Martin

doi:10.1007/s11334-022-00449-3

Cited by 32 publications

(9 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To evaluate the proposed method we have implemented Q A -learning in Python. The implementation uses AALpy's [19] IoAlergia implementation and it interfaces OpenAI gym [2]. The implementation can be used on all gym environments with discrete action and observation space.…”

Section: Discussionmentioning

confidence: 99%

Reinforcement Learning under Partial Observability Guided by Learned Environment Models

Muškardin¹,

Tappler²,

Aichernig³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In practical applications, we can rarely assume full observability of a system's environment, despite such knowledge being important for determining a reactive control system's precise interaction with its environment. Therefore, we propose an approach for reinforcement learning (RL) in partially observable environments. While assuming that the environment behaves like a partially observable Markov decision process with known discrete actions, we assume no knowledge about its structure or transition probabilities. Our approach combines Q-learning with IoAlergia, a method for learning Markov decision processes (MDP). By learning MDP models of the environment from episodes of the RL agent, we enable RL in partially observable domains without explicit, additional memory to track previous interactions for dealing with ambiguities stemming from partial observability. We instead provide RL with additional observations in the form of abstract environment states by simulating new experiences on learned environment models to track the explored states. In our evaluation we report on the validity of our approach and its promising performance in comparison to six state-of-the-art deep RL techniques with recurrent neural networks and fixed memory.

show abstract

Section: Discussionmentioning

confidence: 99%

Reinforcement Learning under Partial Observability Guided by Learned Environment Models

Muškardin¹,

Tappler²,

Aichernig³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Learning setup. For automata learning, we used the learning framework AALPY [22] which is a Python library that implements many state-of-the-art automata learning algorithms. Originally, AALPY mainly focused on active automata learning algorithms.…”

Section: Methodsmentioning

confidence: 99%

Active vs. Passive: A Comparison of Automata Learning Paradigms for Network Protocols

Aichernig

Muškardin

Pferscher

2022

Electron. Proc. Theor. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

Active automata learning became a popular tool for the behavioral analysis of communication protocols. The main advantage is that no manual modeling effort is required since a behavioral model is automatically inferred from a black-box system. However, several real-world applications of this technique show that the overhead for the establishment of an active interface might hamper the practical applicability. Our recent work on the active learning of Bluetooth Low Energy (BLE) protocol found that the active interaction creates a bottleneck during learning. Considering the automata learning toolset, passive learning techniques appear as a promising solution since they do not require an active interface to the system under learning. Instead, models are learned based on a given data set. In this paper, we evaluate passive learning for two network protocols: BLE and Message Queuing Telemetry Transport (MQTT). Our results show that passive techniques can correctly learn with less data than required by active learning. However, a general random data generation for passive learning is more expensive compared to the costs of active learning.

show abstract

“…Mealy machines represent a neat modeling formalism for systems that create observable outputs after an input execution, i.e., reactive systems. Moreover, many state-of-the-art automata learning algorithms and frameworks [20,25] support Mealy machines. A Mealy machine is a finite state machine, where the states are connected via transitions that are labeled with input actions and the corresponding observable outputs.…”

Section: Mealy Machinesmentioning

confidence: 99%

“…Since Python enables the usage of convenient libraries for the composition of BLE packets, we aim at a consistent learning framework integration. At present, AALpy [25] is a novel active learning library written in Python. AALpy implements state-of-the-art learning algorithms and conformance testing techniques, including the improved L * variant that is considered here.…”

Section: Learning Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Fingerprinting Bluetooth Low Energy Devices via Active Automata Learning

Pferscher

Aichernig

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Automata learning is a technique to automatically infer behavioral models of black-box systems. Today's learning algorithms enable the deduction of models that describe complex system properties, e.g., timed or stochastic behavior. Despite recent improvements in the scalability of learning algorithms, their practical applicability is still an open issue. Little work exists that actually learns models of physical black-box systems. To fill this gap in the literature, we present a case study on applying automata learning on the Bluetooth Low Energy (BLE) protocol. It shows that not only the size of the system limits the applicability of automata learning. Also, the interaction with the system under learning creates a major bottleneck that is rarely discussed. In this article, we propose a general automata learning architecture for learning a behavioral model of the BLE protocol implemented by a physical device. With this framework, we can successfully learn the behavior of six investigated BLE devices. Furthermore, we extended the learning technique to learn security critical behavior, e.g., key-exchange procedures for encrypted communication. The learned models depict several behavioral differences and inconsistencies to the BLE specification. This shows that automata learning can be used for fingerprinting black-box devices, i.e., characterizing systems via their specific learned models. Moreover, learning revealed a crashing scenario for one device.

show abstract

AALpy: an active automata learning library

Cited by 32 publications

References 31 publications

Reinforcement Learning under Partial Observability Guided by Learned Environment Models

Reinforcement Learning under Partial Observability Guided by Learned Environment Models

Active vs. Passive: A Comparison of Automata Learning Paradigms for Network Protocols

Fingerprinting Bluetooth Low Energy Devices via Active Automata Learning

Contact Info

Product

Resources

About