Collaborative human-autonomy semantic sensing through structured POMDP planning

Burks, Luke; Ahmed, Nisar; Loefgren, Ian; Barbier, Luke; Muesing, Jeremy; McGinley, Jamison; Vunnam, Sousheel

doi:10.1016/j.robot.2021.103753

Cited by 16 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5) Assessment Time Horizon: With regard to the time horizon for the VoI assessment, we can distinguish two types of approaches for VoI: Myopic methods assess the information sequence one step at a time [67], whereas non-myopic methods evaluate information sequentially [68] or make sequential decisions [69]. We did not identify any non-myopic approaches in the studied use cases.…”

Section: Voi Assessment Propertiesmentioning

confidence: 99%

Value of Information in Wireless Sensor Network Applications and the IoT: A Review

Alawad

Kraemer

2022

IEEE Sensors J.

View full text Add to dashboard Cite

Value of Information (VoI) is a concept to assess the usefulness of information for a specific goal, and has in the last decade experienced a growing interest also for Wireless Sensor Network (WSN) applications and the Internet of Things (IoT). By making the value of information explicit in the form of VoI, WSN and IoT applications should be able to better assess which information to spend their constrained resources on. However, the definition of VoI is highly application-dependent, which has led to a fragmented understanding of VoI, and there is a lack of a comprehensive overview. In this structured review, we first categorize application use cases and examine what VoI is used for, and explore the different approaches to defining VoI. We then provide a well-structured and comprehensive discussion of the specific approaches used in the literature to determine VoI, together with examples of use cases. We categorize the different approaches to calculating VoI, describe their properties systematically and distinguish between observed VoI and expected VoI. We also discuss adaptive VoI approaches and point towards future directions within the field.

show abstract

Section: Voi Assessment Propertiesmentioning

confidence: 99%

Value of Information in Wireless Sensor Network Applications and the IoT: A Review

Alawad

Kraemer

2022

IEEE Sensors J.

View full text Add to dashboard Cite

show abstract

“…We then assess whether the proposed PSDA method actually performs better than other data fusion modalities in terms of rate of mission success (i.e. the four target objects are found within the time limit), average numbers of detected objects, and average traveled distance by the ground rover when missions succeeds (similar to the metrics used in [5], [10]). Then, we move on to analyze the behavior of estimated posterior false alarm semantic data association probabilities γ 0 to investigate how the autonomous rover reasons about the validity of human semantic observations in an online setting, when ground truth data for correct/incorrect semantic data are available.…”

Section: B Data Fusion Modalitiesmentioning

confidence: 99%

“…This avoids incorrectly biasing or collapsing state probability distributions in the presence of imperfect human sensing, thereby improving collaborative human-robot sensing for object/world state estimation. PSDA naturally integrates with existing hybrid 'soft-hard' Bayesian data fusion schemes developed in earlier work for collaborative human-robot team sensing [5], [6], [10]- [13]. Furthermore, although PSDA requires knowledge of additional human sensing characteristics like false positive rates, it is robust to modest mismatches between assumed and actual human sensor parameters.…”

Section: Introductionmentioning

confidence: 99%

“…Since PSDA maintains and supports recursive state estimation with full non-Gaussian posterior state pdfs, it also naturally supports belief-based planning and decisionmaking under uncertainty, e.g. via approximate continuous state POMDP solvers [6], [10]. Although this link to planning and decision-making is not explored in detail here, such compatibility gives PSDA additional practical advantages over other semantic data fusion methods that yield less accurate state pdf approximations and are less practical for applications involving robots with limited computing/reasoning abilities.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Probabilistic Semantic Data Association for Collaborative Human-Robot Sensing

Wakayama¹,

Ahmed

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In collaborative human-robot semantic sensing problems, e.g. for scientific exploration, robots could potentially overtrust information given by a human partner, resulting in suboptimal state estimation and poor team performance. When humans cannot be treated as oracles, robots need to update state beliefs to correctly account for possible discrepancies between human semantic observations and the actual world states which lead to those observations. This work develops strategies for rigorous online calculation of probabilistic semantic data association (PSDA) probabilities for semantic likelihoods in general settings, unlike previous work which developed naive or heuristic approximations for specific settings. The new PSDA method is incorporated into a hybrid Bayesian data fusion scheme which uses Gaussian mixture priors for object states and softmax functions for semantic human sensor observation likelihoods, and is demonstrated in Monte Carlo simulations of collaborative multi-object search missions featuring a range of relevant human sensing characteristics (e.g. false detection rate). It is shown that PSDA leads to robust estimation of observation association probabilities under a wide range of conditions whenever semantic human sensor data contain significant target reference ambiguities for autonomous object search and localization.

show abstract

“…Other methods considered humans assisting robots based on both human judgment and robots requests. In [37], an interactive human-robot semantic sensing framework was proposed. Through this framework, the robot can pose questions to the operator, who acts as a human sensor, in order to increase its semantic knowledge or assure its observations in a target search mission.…”

Section: Introductionmentioning

confidence: 99%

Human-Robot Interaction via a Joint-Initiative Supervised Autonomy (JISA) Framework

Sidaoui¹,

Daher²,

Asmar³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose and validate a Joint-Initiative Supervised Autonomy (JISA) framework for Human-Robot Interaction (HRI), in which a robot maintains a measure of its self-confidence (SC) while performing a task, and only prompts the human supervisor for help when its SC drops. At the same time, during task execution, a human supervisor can intervene in the task being performed, based on his/her Situation Awareness (SA). To evaluate the applicability and utility of JISA, it is implemented on two different HRI tasks: grid-based collaborative simultaneous localization and mapping (SLAM) and automated jigsaw puzzle reconstruction. Augmented Reality (AR) (for SLAM) and two-dimensional graphical user interfaces (GUI) (for puzzle reconstruction) are custom-designed to enhance human SA and allow intuitive interaction between the human and the agent. The superiority of the JISA framework is demonstrated in experiments. In SLAM, the superior maps produced by JISA preclude the need for post processing of any SLAM stock maps; furthermore, JISA reduces the required mapping time by approximately 50 percent versus traditional approaches. In automated puzzle reconstruction, the JISA framework outperforms both fully autonomous solutions, as well as those resulting from on-demand human intervention prompted by the agent.

show abstract

Collaborative human-autonomy semantic sensing through structured POMDP planning

Cited by 16 publications

References 18 publications

Value of Information in Wireless Sensor Network Applications and the IoT: A Review

Value of Information in Wireless Sensor Network Applications and the IoT: A Review

Probabilistic Semantic Data Association for Collaborative Human-Robot Sensing

Human-Robot Interaction via a Joint-Initiative Supervised Autonomy (JISA) Framework

Contact Info

Product

Resources

About