Underwater multi-robot convoying using visual tracking by detection

Abstract

Abstract-We present a robust multi-robot convoying approach that relies on visual detection of the leading agent, thus enabling target following in unstructured 3-D environments. Our method is based on the idea of tracking-by-detection, which interleaves efficient model-based object detection with temporal filtering of image-based bounding box estimation. This approach has the important advantage of mitigating tracking drift (i.e. drifting away from the target object), which is a common symptom of model-free trackers and is detrimental to sustained convoying in practice. To illustrate our solution, we collected extensive footage of an underwater robot in ocean settings, and hand-annotated its location in each frame. Based on this dataset, we present an empirical comparison of multiple tracker variants, including the use of several convolutional neural networks, both with and without recurrent connections, as well as frequency-based model-free trackers. We also demonstrate the practicality of this tracking-by-detection strategy in real-world scenarios by successfully controlling a legged underwater robot in five degrees of freedom to follow another robot's independent motion.

Morphology-Agnostic Visual Robotic Control

“…Reinforcement learning approaches have also been proposed for the IBVS setting [24], [25], [26], [27], [28], [29]. All these IBVS methods produce controllers that are tied to a single robot morphology in some way-for example, they may require visual markers on the robot [20], [21], [22], [23] or a large dataset of interactions specific to the current robot morphology and environment [24], [26], [28], [29], [30], [31], [25], [27]. In contrast, MAVRIC performs automatic selfrecognition to produce a controller that adapts to new or altered robots within a few seconds.…”

Toward a Generic Diver-Following Algorithm: Balancing Robustness and Efficiency in Deep Visual Detection

“…In this case, the SSD (MobileNet V2) model was re-trained on additional data and object categories for ROV and hand gestures (used for human-robot communication [15]). same models can be utilized in a wide range of underwater human-robot collaborative applications such as following a team of divers, robot convoying [5], human-robot communication [15], etc. In particular, if the application do not pose real-time constraints, we can use models such as Faster R-CNN (Inception V2) for better detection performances.…”

“…Consequently, classical model-based detection algorithms fail to achieve good generalization performance [3,4]. On the other hand, model-free algorithms incur significant target drift [5] under such noisy conditions. Figure 1: Snapshots of a set of diverse first-person views of the robot from different diver-following scenarios.…”

Understanding human motion and gestures for underwater human–robot collaboration

“…CNN-based diver detection models have recently been investigated for underwater applications as well [16]. Once trained with sufficient data, these models are quite robust to occlusion, noise, and color distortions.…”

“…Despite the robust performance, the applicability of these models to real-time applications is often limited due to their slow running time. We refer to [16] for a detailed study on the performance and applicability of various deep visual detection models for underwater applications. In this paper, we design a CNNbased model that achieves robust detection performance in addition to ensuring that the real-time operating constraints on board an autonomous underwater robot are met.…”