Learning an Urban Air Mobility Encounter Model from Expert Preferences

Katz, Sydney M.; Bihan, Anne-Claire Le; Kochenderfer, Mykel J.

doi:10.1109/dasc43569.2019.9081648

Cited by 17 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While having humans provide pairwise comparisons does not suffer from similar problems to collecting demonstrations, each comparison question is much less informative than a demonstration, because comparison queries can provide at most 1 bit of information. Prior works have attempted to tackle this problem by actively generating the comparison questions (Basu et al, 2019;Biyik and Sadigh, 2018;Katz et al, 2019;Sadigh et al, 2017;Wilde et al, 2019). Although they were able to achieve significant gains in terms of the required number of comparisons, we hypothesize that one can attain even better data efficiency by leveraging multiple sources of information, even when some sources might not completely align with the true reward functions, e.g., demonstrations as in the driving work by Basu et al (2017).…”

Section: Learning Reward Functions From Rankingsmentioning

confidence: 91%

Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences

Bıyık

Losey

Palan

et al. 2021

The International Journal of Robotics Research

View full text Add to dashboard Cite

Reward functions are a common way to specify the objective of a robot. As designing reward functions can be extremely challenging, a more promising approach is to directly learn reward functions from human teachers. Importantly, data from human teachers can be collected either passively or actively in a variety of forms: passive data sources include demonstrations (e.g., kinesthetic guidance), whereas preferences (e.g., comparative rankings) are actively elicited. Prior research has independently applied reward learning to these different data sources. However, there exist many domains where multiple sources are complementary and expressive. Motivated by this general problem, we present a framework to integrate multiple sources of information, which are either passively or actively collected from human users. In particular, we present an algorithm that first utilizes user demonstrations to initialize a belief about the reward function, and then actively probes the user with preference queries to zero-in on their true reward. This algorithm not only enables us combine multiple data sources, but it also informs the robot when it should leverage each type of information. Further, our approach accounts for the human’s ability to provide data: yielding user-friendly preference queries which are also theoretically optimal. Our extensive simulated experiments and user studies on a Fetch mobile manipulator demonstrate the superiority and the usability of our integrated framework.

show abstract

Section: Learning Reward Functions From Rankingsmentioning

confidence: 91%

Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences

Bıyık

Losey

Palan

et al. 2021

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…While having humans provide pairwise comparisons does not suffer from similar problems to collecting demonstrations, each comparison question is much less informative than a demonstration, since comparison queries can provide at most 1 bit of information. Prior works have attempted to tackle this problem by actively generating the comparison questions (Sadigh et al 2017;Biyik and Sadigh 2018;Basu et al 2019;Katz et al 2019). While they were able to achieve significant gains in terms of the required number of comparisons, we hypothesize that one can attain even better data-efficiency by leveraging multiple sources of information, even when some sources might not completely align with the true reward functions, e.g., demonstrations as in the driving work by Basu et al (2017).…”

Section: Related Workmentioning

confidence: 93%

Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences

Bıyık¹,

Losey²,

Palan³

et al. 2020

Preprint

View full text Add to dashboard Cite

Reward functions are a common way to specify the objective of a robot. As designing reward functions can be extremely challenging, a more promising approach is to directly learn reward functions from human teachers. Importantly, humans provide data in a variety of forms: these include instructions (e.g., natural language), demonstrations, (e.g., kinesthetic guidance), and preferences (e.g., comparative rankings). Prior research has independently applied reward learning to each of these different data sources. However, there exist many domains where some of these information sources are not applicable or inefficient -while multiple sources are complementary and expressive. Motivated by this general problem, we present a framework to integrate multiple sources of information, which are either passively or actively collected from human users. In particular, we present an algorithm that first utilizes user demonstrations to initialize a belief about the reward function, and then proactively probes the user with preference queries to zero-in on their true reward. This algorithm not only enables us combine multiple data sources, but it also informs the robot when it should leverage each type of information. Further, our approach accounts for the human's ability to provide data: yielding userfriendly preference queries which are also theoretically optimal. Our extensive simulated experiments and user studies on a Fetch mobile manipulator demonstrate the superiority and the usability of our integrated framework.

show abstract

“…The geometries and flight phases explored by each encounter set during the development of the speed logic are summarized in table 1. These encounter sets have been previously used in the evaluation of other ACAS X variants and are generally accepted as standards to measure the performance of collision avoidance systems [11], [12], [13]. The LLCEM encounter set was sampled from a statistical airspace model created from nine months of radar data covering much of the continental United States and captures the correlation between aircraft trajectories due to intervention from air traffic control [11].…”

Section: B Encounter Setsmentioning

confidence: 99%

Collision Risk and Operational Impact of Speed Change Advisories as Aircraft Collision Avoidance Maneuvers

Katz¹,

Alvarez²,

Wu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited. This material is based upon work supported by the Federal Aviation Administration under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Federal Aviation Administration. This document is derived from work done for the FAA (and possibly others), it is not the direct product of work done for the FAA. The information provided herein may include content supplied by third parties. Although the data and information contained herein has been produced or processed from sources believed to be reliable, the Federal Aviation Administration makes no warranty, expressed or implied, regarding the accuracy, adequacy, completeness, legality, reliability, or usefulness of any information, conclusions or recommendations provided herein. Distribution of the information contained herein does not constitute an endorsement or warranty of the data or information provided herein by the Federal Aviation Administration or the U.S. Department of Transportation. Neither the Federal Aviation Administration nor the U.S. Department of Transportation shall be held liable for any improper or incorrect use of the information contained herein and assumes no responsibility for anyone's use of the information. The Federal Aviation Administration and U.S. Department of Transportation shall not be liable for any claim for any loss, harm, or other damages arising from access to or use of data information, including without limitation any direct, indirect, incidental, exemplary, special or consequential damages, even if advised of the possibility of such damages. The Federal Aviation Administration shall not be liable for any decision made or action taken, in reliance on the information contained herein.

show abstract

Learning an Urban Air Mobility Encounter Model from Expert Preferences

Cited by 17 publications

References 12 publications

Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences

Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences

Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences

Collision Risk and Operational Impact of Speed Change Advisories as Aircraft Collision Avoidance Maneuvers

Contact Info

Product

Resources

About