Regular Decision Processes for Grid Worlds

Lenaers, Nicky; Otterlo, Martijn van

doi:10.1007/978-3-030-93842-0_13

Communications in Computer and Information Science

2022

DOI: 10.1007/978-3-030-93842-0_13

|View full text |Cite

Regular Decision Processes for Grid Worlds

Nicky Lenaers

Martijn van Otterlo

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2022

2024

Publication Types

Select...

Book3

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

(1 citation statement)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A nice tool-set for transforming from non-Markovian properties in temporal logic to RDP and solving them also exists[16] 3. Replacing our automata generation approach with one based on constrained RNNs, such as in[15] might improve this aspect.…”

mentioning

confidence: 99%

Reinforcement Learning in RDPs by Combining Deep RL with Automata Learning

Shahar,

Brafman

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Regular Decision Processes (RDPs) are a recently introduced model for decision-making in non-Markovian domains in which states are not postulated a-priori, and the next observation depends in a regular manner on past history. As such, they provide a more succinct and understandable model of the dynamics and reward function. Existing algorithms for learning RDPs attempt to learn an automaton that reflects the regularity of the underlying domain. However, their scalability is limited due to the practical difficulty of learning automata. In this paper we propose to leverage the power of Deep reinforcement learning in partially observable domain to learn RDPs: First, we learn an RNN-based policy. Then, we generate an automaton that reflects the policy’s structure and use our old data to transform it into an MDP, which we solve. This results in a finite, explainable policy structure, and, as our empirical evaluation on old and new RDP benchmarks shows, much better sample complexity.

show abstract

mentioning

confidence: 99%

Reinforcement Learning in RDPs by Combining Deep RL with Automata Learning

Shahar,

Brafman

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

Regular Decision Processes for Grid Worlds

Lenaers

Otterlo

2022

Communications in Computer and Information Science

View full text Add to dashboard Cite

Regular Reinforcement Learning

Dohmen,

Perez,

Somenzi

et al. 2024

Computer Aided Verification

View full text Add to dashboard Cite

In reinforcement learning, an agent incrementally refines a behavioral policy through a series of episodic interactions with its environment. This process can be characterized as explicit reinforcement learning, as it deals with explicit states and concrete transitions. Building upon the concept of symbolic model checking, we propose a symbolic variant of reinforcement learning, in which sets of states are represented through predicates and transitions are represented by predicate transformers. Drawing inspiration from regular model checking, we choose regular languages over the states as our predicates, and rational transductions as predicate transformations. We refer to this framework as regular reinforcement learning, and study its utility as a symbolic approach to reinforcement learning. Theoretically, we establish results around decidability, approximability, and efficient learnability in the context of regular reinforcement learning. Towards practical applications, we develop a deep regular reinforcement learning algorithm, enabled by the use of graph neural networks. We showcase the applicability and effectiveness of (deep) regular reinforcement learning through empirical evaluation on a diverse set of case studies.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Regular Decision Processes for Grid Worlds

Cited by 3 publications

References 26 publications

Reinforcement Learning in RDPs by Combining Deep RL with Automata Learning

Reinforcement Learning in RDPs by Combining Deep RL with Automata Learning

Regular Decision Processes for Grid Worlds

Regular Reinforcement Learning

Contact Info

Product

Resources

About