Enhanced directional self-assembly based on active recruitment and guidance

Abstract

We propose a novel human-robot-interaction framework for robust visual scene understanding. Without any a-priori knowledge about the objects, the task of the robot is to correctly enumerate how many of them are in the scene and segment them from the background. Our approach builds on top of state-of-the-art computer vision methods, generating object hypotheses through segmentation. This process is combined with a natural dialog system, thus including a 'human in the loop' where, by exploiting the natural conversation of an advanced dialog system, the robot gains knowledge about ambiguous situations. We present an entropy-based system allowing the robot to detect the poorest object hypotheses and query the user for arbitration. Based on the information obtained from the human-robot dialog, the scene segmentation can be re-seeded and thereby improved. We present experimental results on real data that show an improved segmentation performance compared to segmentation without interaction.

Supervised morphogenesis: Exploiting morphological flexibility of self-assembling multirobot systems through cooperation with aerial robots

“…They presented a form of cooperation that has the potential to merge sensory or guidance information from aerial robots into the control of robots operating on the ground. Application scenarios for such air-ground robot systems have already been presented in [20].…”

“…In this study, we extended the technology behind SWARMORPH-script to take advantage of the higher communication bandwidth and speed provided by the mxRAB device available to the foot-bots. These enhancements allow foot-bots to demonstrate behaviors that were unachievable to their predecessors such as forming multiple connections in parallel [20] and coordinating the motion in target morphologies [20,47,48].…”

“…Larger morphologies can only be formed by self-assembling robots if connections can be formed between connection-inviting robots and connection-seeking robots. We developed enhanced directional self-assembly (EDSA) [20] as a connection forming mechanism for the foot-bots. The mechanism takes advantage of the high-speed situated communication provided by the mxRAB device.…”

“…Spatially targeted communication [19] is then applied to let aerial robots establish a communication link to specific robots based on their location on the ground. Morphology formation instructions [7] are then transmitted through this link to initiate the formation of target [20]. We present two case studies in which we assess the feasibility of the proposed control methodology using real robotic hardware.…”

Spatially targeted communication and self-assembly

“…The flying robot then establishes a communication link with an appropriate number of robots on the ground, and sends the group instructions on how to form the morphology [5]. Finally, the self-assembling robots form the morphology using a recruitment and guidance-based approach [6] and carry out the task.…”