Learning Latent Actions without Human Demonstrations

Mehta, Shaunak A.; Parekh, Sagar; Losey, Dylan P.

doi:10.1109/icra46639.2022.9812230

Cited by 5 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, learning-based methods have been employed to generate personalized mappings to compensate for individual differences [12], [13] and task-specific mappings to leverage latent actions [14], [15]. These learning-based mappings follow manifolds derived from observations and are generally dynamic and non-linear.…”

Section: A Control Coordinate Systems In Telemanipulationmentioning

confidence: 99%

A Design Space of Control Coordinate Systems in Telemanipulation

Wang,

Praveena,

Gleicher

2024

IEEE Access

View full text Add to dashboard Cite

Teleoperation systems map operator commands from an input device into some coordinate frame in the remote environment. This frame, which we call a control coordinate system, should be carefully chosen as it determines how operators should move to get desired robot motions. While specific choices made by individual systems have been described in prior work, a design space, i.e., an abstraction that encapsulates the range of possible options, has not been codified. In this paper, we articulate a design space of control coordinate systems, which can be defined by choosing a direction in the remote environment for each axis of the input device. Our key insight is that there is a small set of meaningful directions in the remote environment. Control coordinate systems in prior works can be organized by the alignments of their axes with these directions and new control coordinate systems can be designed by choosing from these directions. We also provide three design criteria to reason about the suitability of control coordinate systems for various scenarios. To demonstrate the utility of our design space, we use it to organize prior systems and design control coordinate systems for three scenarios that we assess through human-subject experiments. Our results highlight the promise of our design space as a conceptual tool to assist system designers to design control coordinate systems that are effective and intuitive for operators.

show abstract

Section: A Control Coordinate Systems In Telemanipulationmentioning

confidence: 99%

A Design Space of Control Coordinate Systems in Telemanipulation

Wang,

Praveena,

Gleicher

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…F3: Corrective Feedback -Here, the user has a trajectory showcasing imperfect behavior as a reference and needs to instruct via an improved strategy. This can be done either implicitly, e.g., by pushing a robot into a correct position (Mehta & Losey, 2022), or explicitly via specifying a better action. Implicit corrections require an additional step translating the corrected trajectory into a sequence of agent actions.…”

Section: Feedback Typesmentioning

confidence: 99%

“…Reinforcement learning from human feedback (RLHF) is a powerful tool to train agents when it is difficult to specify a reward function or when human knowledge can improve training efficiency. Recently, using multiple forms of human feedback for reward modeling has come into focus (Jeon et al, 2020;Ghosal et al, 2023;Ibarz et al, 2018;Bıyık et al, 2022a;Mehta & Losey, 2022). Using diverse sources of information opens up several possibilities: (1) feedback from different sources allows for correcting poten-Interactive Learning with Implicit Human Feedback Workshop at ICML 2023, Honolulu, Hawaii, USA, 2023.…”

Section: Introductionmentioning

confidence: 99%

VISITOR: Visual Interactive State Sequence Exploration for Reinforcement Learning

Metz

Bykovets

Joos

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Understanding the behavior of deep reinforcement learning agents is a crucial requirement throughout their development. Existing work has addressed the identification of observable behavioral patterns in state sequences or analysis of isolated internal representations; however, the overall decision‐making of deep‐learning RL agents remains opaque. To tackle this, we present VISITOR, a visual analytics system enabling the analysis of entire state sequences, the diagnosis of singular predictions, and the comparison between agents. A sequence embedding view enables the multiscale analysis of state sequences, utilizing custom embedding techniques for a stable spatialization of the observations and internal states. We provide multiple layers: (1) a state space embedding, highlighting different groups of states inside the state‐action sequences, (2) a trajectory view, emphasizing decision points, (3) a network activation mapping, visualizing the relationship between observations and network activations, (4) a transition embedding, enabling the analysis of state‐to‐state transitions. The embedding view is accompanied by an interactive reward view that captures the temporal development of metrics, which can be linked directly to states in the embedding. Lastly, a model list allows for the quick comparison of models across multiple metrics. Annotations can be exported to communicate results to different audiences. Our two‐stage evaluation with eight experts confirms the effectiveness in identifying states of interest, comparing the quality of policies, and reasoning about the internal decision‐making processes.

show abstract