Grasping and Object Reorientation for Autonomous Construction of Stone Structures

Abstract

Building large and stable structures from highly irregular stones is among the most challenging construction tasks with excavators. In this paper, we present a method for grasp planning and object manipulation that enables the world's first autonomous assembly of a large-scale stone wall with an unmanned hydraulic excavator system. Our method utilizes point clouds and mesh surface reconstruction of stones in order to plan grasp configurations with a 2-jaw gripper mounted on the excavator. Besides considering the geometry of the stone to sample force closure grasps, the grasp planner also takes into account collision constraints during object pick and place, informed by a LiDAR based point cloud map. Furthermore, we show an approach to reorient arbitrarily shaped objects that are not feasible to be directly placed at the desired location without violating collision constraints. Using a physics engine, we find a settled intermediate pose that allows direct placement and is reachable from the initial stone pose. The applicability of the proposed grasp planning method is demonstrated with the construction of a dry stone wall composed of over one hundred boulders using an autonomous excavator. We show a high primary grasp success rate (82.2 %) and illustrate how the system recovers from slippage by relocating the object and replanning the grasp correspondingly.

A framework for robotic excavation and dry stone construction using on-site materials

“…At present, these grasp failures are detected manually, but this could be achieved autonomously in the future by comparing the actual gripper closing angle with the specified closing angle for a given grasp. A more detailed overview of grasp planning from segmented object instances can be found in ( 42 , 43 ).…”

“…Picking for placement additionally requires consideration of collision constraints at both ends of the sequence: We first generated grasp hypotheses on the desired stone by transforming its mesh to the target placement pose and sampling noncolliding grasps. These grasp hypotheses were then reprojected to the current stone pose (for picking), evaluated for collision and force closure, and ranked by task-specific heuristic costs ( 43 ).…”

“…Our research improves on these studies by targeting the in situ construction of arbitrarily shaped double-faced dry stone walls built from large-scale boulders and concrete demolition debris (we use "stones" henceforth to describe both of these elements interchangeably). Building on our earlier work in geometric planning, stone segmentation, and grasping (41)(42)(43), we present a synthesized system capable of autonomously detecting, grasping, scanning, and reorienting stones in the wild, allowing our geometric planner to optimize for stable and geometrically registered solutions that are unconstrained in orientation. We constructed permanent robotic stone walls, reaching heights up to 6 m, and used these demonstrators to perform an environmental impact assessment for the building method.…”

Analysis Randon Causes Repeatability Errors Inducted by Friction at Joints in Industrial Robots

“…Other authors translate the phenomenon by a description of an object gripper of different shapes as well as the functioning of the gripper jaws [7]. Another method of describing objects of complex shapes using a set of 3D points has been developed in order to position and orient these complex points [8].…”

Identifying enablers and relational ontology networks in design for digital fabrication

“…Summary of resources as the enablers from selected articles. ,15,37,38,40,41,47,46,45,43,53,52,50,49,54,62,61,60,59,58,57,56,64,69,68,67,66,65,70,71,73,74,78,80,[82][83][84][85][86][87] …”