Toward fieldable human-scale mobile manipulation using RoMan

Kessens, Chad C.; Fink, Jonathan; Hurwitz, Arnon M.; Kaplan, Moisés; Osteen, Philip R.; Rocks, Trevor; Rogers, John G.; Stump, Ethan; Quang, Long; DiBlasi, Michael; Gonzalez, Mark; Patel, D.; Patel, Jaymit; Patel, S.; Weiker, Matthew; Bowkett, Joseph; Detry, Renaud; Karumanchi, Sisir; Burdick, Joel W.; Matthies, Larry; Oza, Yash; Agarwal, Aditya; Dornbush, Andrew; Likhachev, Maxim; Schmeckpeper, Karl; Daniilidis, Kostas; Kamat, Ajinkya; Choudhury, Sanjiban; Mandalika, Aditya; Srinivasa, Siddhartha S.

doi:10.1117/12.2559995

Cited by 5 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This choice of reference frame significantly reduces planning times. At this layer, we introduce a planning arbiter known as the Whole Body Planner (WBP) (Kessens et al, 2020) in which coordinates arm movements through a diversity of planners and movement profiles. Successive planning algorithms in the stack enable a mobile manipulator's prehensile manipulation of objects within its task space.…”

Section: Manipulation Plannermentioning

confidence: 99%

“…Successive planning algorithms in the stack enable a mobile manipulator's prehensile manipulation of objects within its task space. Initially, grasp planning using the Grasp Pose Synthesis (Detry et al, 2017) algorithm returns kinematically feasible EE poses that are geometrically suited for our particular gripper within a selected region known as the region-of-interest (ROI) (Kessens et al, 2020). WBP then invokes two motion planning algorithms in parallel: Anytime Repairing A* (ARA*) (Likhachev et al, 2004) and Generalized Lazy Search (GLS) (Mandalika et al, 2019) to search for a solution on a graph in the 15-dimensional configuration space of both arms and torso.…”

Section: Manipulation Plannermentioning

confidence: 99%

“…The second layer, which we call the "Intrinsic Layer", runs underneath the "Extrinsic Layer" and is responsible for reachability mapping based on inverse kinematics, and trajectory discretization/following through a whole body controller termed Intrinsic Controller (IC) (Kessens et al, 2020) and the open-source ROS Joint Trajectory Controller 9 . The IC is capable of performing short Cartesian arm and base movements.…”

Section: Manipulation Plannermentioning

confidence: 99%

“…The second platform, a Clearpath Robotics Husky A200 mobile manipulator outfitted with a Universal Robotics UR5 manipulator, Robotiq FT300 force/torque sensor, and a three-fingered Robotiq Adaptive Robot Gripper, was used for the experiments involving deliberative interactive estimation. The third platform, the RoMan mobile manipulator (Kessens et al, 2020), is used in the language guided mobile manipulation experiment. All three robots shared a common intelligence architecture and language model, differences appear only at the level of implementing navigation and manipulation behaviors based on the physical and hardware interface differences between platforms.…”

Section: Experimental Designmentioning

confidence: 99%

See 3 more Smart Citations

An Intelligence Architecture for Grounded Language Communication with Field Robots

Howard¹,

Stump²,

Fink³

et al. 2022

Self Cite

View full text Add to dashboard Cite

For humans and robots to collaborate effectively as teammates in unstructured environments, robots must be able to construct semantically rich models of the environment, communicate efficiently with teammates, and perform sequences of tasks robustly with minimal human intervention, as direct human guidance may be infrequent and/or intermittent. Contemporary architectures for human-robot interaction often rely on engineered human-interface devices or structured languages that require extensive prior training and inherently limit the kinds of information that humans and robots can communicate. Natural language, particularly when situated with a visual representation of the robot’s environment, allows humans and robots to exchange information about abstract goals, specific actions, and/or properties of the environment quickly and effectively. In addition, it serves as a mechanism to resolve inconsistencies in the mental models of the environment across the human-robot team. This article details a novel intelligence architecture that exploits a centralized representation of the environment to perform complex tasks in unstructured environments. The centralized environment model is informed by a visual perception pipeline, declarative knowledge, deliberate interactive estimation, and a multimodal interface. The language pipeline also exploits proactive symbol grounding to resolve uncertainty in ambiguous statements through inverse semantics. A series of experiments on three different, unmanned ground vehicles demonstrates the utility of this architecture through its robust ability to perform language-guided spatial navigation, mobile manipulation, and bidirectional communication with human operators. Experimental results give examples of component-level behaviors and overall system performance that guide a discussion on observed performance and opportunities for future innovation.

show abstract

Section: Manipulation Plannermentioning

confidence: 99%

Section: Manipulation Plannermentioning

confidence: 99%

Section: Manipulation Plannermentioning

confidence: 99%

Section: Experimental Designmentioning

confidence: 99%

See 2 more Smart Citations

An Intelligence Architecture for Grounded Language Communication with Field Robots

Howard¹,

Stump²,

Fink³

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Figure2. The "RoMan" platform upon which the algorithms herein were developed and demonstrated(Kessens et al, 2020). © SPIE 2020.…”

mentioning

confidence: 99%

Grasping and Transport of Unstructured Collections of Massive Objects

Bowkett¹,

Karumanchi²,

Detry³

2022

View full text Add to dashboard Cite

We present a collection of sensorimotor models which, when paired with a custom, mobile manipulation platform, collectively enable the autonomous deconstruction of piles of debris to facilitate mobility in cluttered spaces. The models address the problems of visual debris segmentation, object selection, and visual grasp planning. They also exercise proprioceptive grasp control, and force-controlled object extraction to enact the grasping plan. The object selector segments a debris pile into a set of parts that appear disjoint from one another and executes a rule-based decision program to select the object that appears easiest to extract. Then a grasp planner identifies a grasping point and hand preshape at a location where the gripper configuration fits the shape of the object, while also satisfying kinematic and collision-avoidance constraints. Our geometric grasp-prototype concept allows the planner to establish grasp suitability by fitting a set of shape-grasp primitives to a 2.5D depth image of the pile. The robot then applies the grasp reactively, pushing against the object until a preset resistance is met. Finally, an admittance controller guides object extraction, allowing a prescribed end-effector compliance along task-frame axes to minimize adverse forces and torques on the hand. We show experiments demonstrating the applicability of those models in isolation and in concert, including grasp tests conducted on objects representative of of a human-scale urban environment.

show abstract