Fuel in Markov Decision Processes (FiMDP): A Practical Approach to Consumption

Blahoudek, František; Cubuktepe, Murat; Novotný, Petr; Ornik, Melkior; Thangeda, Pranay; Topcu, Ufuk

doi:10.1007/978-3-030-90870-6_34

Cited by 2 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second benchmark is a partially observable extension of the unmanned underwater vehicle (UUV) benchmark from (Blahoudek et al 2021). Here, the agent operates in a grid-world, with actions performing movements in the cardinal directions.…”

Section: Methodsmentioning

confidence: 99%

“…In formal methods and verification, resource-constrained agents are typically modeled in the consumption framework (used in this paper) or energy framework. The former has been considered in both non-probabilistic (Brázdil et al 2012(Brázdil et al , 2014 and probabilistic (Blahoudek et al 2020(Blahoudek et al , 2021 settings, but not in partially observable ones. The energy framework differs from the consumption one by handling reloads: instead of atomic reloads in reload states, a "negative consumption" is enabled for some state-action pairs, allowing for incremental reloading.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Shielding in Resource-Constrained Goal POMDPs

Ajdarów

Brlej

Novotný

2023

AAAI

View full text Add to dashboard Cite

We consider partially observable Markov decision processes (POMDPs) modeling an agent that needs a supply of a certain resource (e.g., electricity stored in batteries) to operate correctly. The resource is consumed by the agent's actions and can be replenished only in certain states. The agent aims to minimize the expected cost of reaching some goal while preventing resource exhaustion, a problem we call resource-constrained goal optimization (RSGO). We take a two-step approach to the RSGO problem. First, using formal methods techniques, we design an algorithm computing a shield for a given scenario: a procedure that observes the agent and prevents it from using actions that might eventually lead to resource exhaustion. Second, we augment the POMCP heuristic search algorithm for POMDP planning with our shields to obtain an algorithm solving the RSGO problem. We implement our algorithm and present experiments showing its applicability to benchmarks from the literature.

show abstract

Section: Methodsmentioning

confidence: 99%