Expandable-Partially Observable Markov Decision-Process Framework for Modeling and Analysis of Autonomous Vehicle Behavior

Pouya, Parisa; Madni, Azad M.

doi:10.1109/jsyst.2020.3010473

Cited by 15 publications

(7 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to their capability to address uncertainty, POMDPs can also model both the stochasticity in environment transitions and imperfect sensory information [45]. This dual capability becomes vital when dealing with real-time sensor data that inherently contains observational noise and FIGURE 1: A simplified architecture of the ADS with a main focus on the DRA layer strengthening the decision-making task varying environmental states [46].…”

Section: Benefits Of Pomdp In Decision-making Processesmentioning

confidence: 99%

A Risk-Based Decision-Making Process for Autonomous Trains Using POMDP: Case of the Anti-Collision Function

Chelouati,

Boussif,

Beugin

et al. 2024

IEEE Access

View full text Add to dashboard Cite

As the railway domain progresses towards autonomy, maintaining safety at levels comparable to human-operated systems is a crucial challenge. Autonomous trains require advanced systems capable of real-time risk assessment and decision-making, a task traditionally managed by human situational awareness. This paper introduces a novel risk-based decision-making approach for autonomous trains, using Partially Observable Markov Decision Processes (POMDPs) for continuous monitoring and evaluation of environmental collision risks. By consistently maintaining an acceptable risk level through ongoing risk estimation (in terms of occurrence probability and severity degree), the approach supports the decisionmaking capabilities of the autonomous driving system in autonomous trains, enabling safe and informed decisions despite the uncertainties in the train's operational state and environmental conditions. The approach's relevance and effectiveness are illustrated through its application in an anti-collision function for autonomous trains.

show abstract

Section: Benefits Of Pomdp In Decision-making Processesmentioning

confidence: 99%

A Risk-Based Decision-Making Process for Autonomous Trains Using POMDP: Case of the Anti-Collision Function

Chelouati,

Boussif,

Beugin

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…For complex system planning processes, an extendable POMDP was employed by Pouya et al [70] in flexible and partially observable situations. The key benefit of this strategy was that it provided a basic framework (i.e., probabilistic state representation) in contrast to other data-driven approaches that rely on large datasets.…”

Section: Active Inference Context Awareness and Other Full Observable...mentioning

confidence: 99%

“…-Neglected TS abnormal scenarios (e.g., fallen TS, TS back) that ML methods can not recognize. Pouya et al [70] -Developed an extendable POMDP that is not dependent on a large dataset.…”

Section: Authorsmentioning

confidence: 99%

A Survey on Cyber-Physical Security of Autonomous Vehicles Using a Context Awareness Method

Zaboli,

Hong,

Kwon

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Autonomous vehicles face challenges in ensuring cyber-physical security due to their reliance on image data from cameras processed by machine learning. These algorithms, however, are vulnerable to anomalies in the imagery, leading to decreased recognition accuracy and presenting security concerns. Current machine learning models struggle to predict unexpected vehicular situations, particularly with unpredictable objects and unexpected anomalies. To combat this, scholars are focusing on active inference, a method that can adapt models based on human cognition. This paper aims to incorporate active inference into autonomous vehicle systems. Multiple studies have delved into this approach, showing its potential to address security gaps in this field. Specifically, these frameworks have proven effective in handling unforeseen vehicular anomalies.

show abstract

“…Discrete event systems (DESs) [1], [2] encompass a wide variety of human-made systems, including power systems [3], unmanned aerial vehicle systems [4], healthcare service systems [5], traffic systems [6], communication protocols [7], [8], digital systems [9] and robotic systems [10], [11]. A DES typically uses deterministic finite automata (DFA) to describe a plant system and its control specification.…”

Section: Introductionmentioning

confidence: 99%

Reducing the Learning Time of Reinforcement Learning for the Supervisory Control of Discrete Event Systems

et al. 2023

View full text Add to dashboard Cite

Reinforcement learning (RL) can obtain the supervisory controller for discrete-event systems modeled by finite automata and temporal logic. The published methods often have two limitations. First, a large number of training data are required to learn the RL controller. Second, the RL algorithms do not consider uncontrollable events, which are essential for supervisory control theory (SCT). To address the limitations, we first apply SCT to find the supervisors for the specifications modeled by automata. These supervisors remove illegal training data violating these specifications and hence reduce the exploration space of the RL algorithm. For the remaining specifications modeled by temporal logic, the RL algorithm is applied to search for the optimal control decision within the confined exploration space. Uncontrollable events are considered by the RL algorithm as uncertainties in the plant model. The proposed method can obtain a nonblocking supervisor for all specifications with less learning time than the published methods.

show abstract

Expandable-Partially Observable Markov Decision-Process Framework for Modeling and Analysis of Autonomous Vehicle Behavior

Cited by 15 publications

References 45 publications

A Risk-Based Decision-Making Process for Autonomous Trains Using POMDP: Case of the Anti-Collision Function

A Risk-Based Decision-Making Process for Autonomous Trains Using POMDP: Case of the Anti-Collision Function

A Survey on Cyber-Physical Security of Autonomous Vehicles Using a Context Awareness Method

Reducing the Learning Time of Reinforcement Learning for the Supervisory Control of Discrete Event Systems

Contact Info

Product

Resources

About