Toward a Generic Diver-Following Algorithm: Balancing Robustness and Efficiency in Deep Visual Detection

Abstract

This paper explores the design and development of a class of robust diver-following algorithms for autonomous underwater robots. By considering the operational challenges for underwater visual tracking in diverse real-world settings, we formulate a set of desired features of a generic diver following algorithm. We attempt to accommodate these features and maximize general tracking performance by exploiting the state-of-the-art deep object detection models. We fine-tune the building blocks of these models with a goal of balancing the trade-off between robustness and efficiency in an on-board setting under real-time constraints. Subsequently, we design an architecturally simple Convolutional Neural Network (CNN)-based diver-detection model that is much faster than the state-of-the-art deep models yet provides comparable detection performances. In addition, we validate the performance and effectiveness of the proposed diver-following modules through a number of field experiments in closed-water and open-water environments.

Fast Underwater Image Enhancement for Improved Visual Perception

“…These optical artifacts trigger non-linear distortions in the captured images, which severely affect the performance of visionbased tasks such as tracking, detection and classification, Input Generated (a) Perceptual enhancement of underwater images. [23], human body-pose estimation [9], and saliency prediction [40]. segmentation, and visual servoing.…”

Person-following by autonomous robots: A categorical overview

“…Underwater missions are often conducted by a team of human divers and autonomous robots, who cooperatively perform a set of common tasks (Islam et al, 2018c; Sattar et al, 2008). The divers typically lead the tasks and interact with the robots, which follow the divers at certain stages of the mission (Islam et al, 2018a). These situations arise in important applications, such as the inspection of ship hulls and submarine pipelines, the study of marine species migration, and search-and-rescue or surveillance operations.…”

“…Deep learning-based object detection models have recently been investigated for underwater applications as well (Islam et al, 2018a; Shkurti et al, 2012). The state-of-the-art pre-trained models are typically trained (offline) on large underwater datasets and sometimes quantized or pruned in order to get faster inference by balancing robustness and efficiency (Islam et al, 2018a,c). As illustrated in Figure 8(f), once trained with sufficient data, these models are robust to noise and color distortions; additionally, a single model can be used to detect (and track) several objects at once.…”

“…Such assistant robots are being used in many domestic and industrial applications (Piaggio-Fast-Forward, 2017) and in health care (Ilias et al, 2014; Iribe et al, 2011). Moreover, diver-following robots are useful in submarine pipeline and shipwreck inspection, marine life and seabed monitoring, and many other underwater exploration activities (Islam et al, 2018a; Miskovic et al, 2015; Sattar and Dudek, 2009b). Furthermore, the use of person-following aerial robots (Mueller et al, 2016; Naseer et al, 2013; Pestana et al, 2014) has flourished over the last decade in the entertainment industry (Skydio, 2018) as quadcopters have become quite popular for filming outdoor activities, such as mountain climbing, biking, surfing, and many other sporting endeavors.…”

Morphology-Agnostic Visual Robotic Control

“…Reinforcement learning approaches have also been proposed for the IBVS setting [24], [25], [26], [27], [28], [29]. All these IBVS methods produce controllers that are tied to a single robot morphology in some way-for example, they may require visual markers on the robot [20], [21], [22], [23] or a large dataset of interactions specific to the current robot morphology and environment [24], [26], [28], [29], [30], [31], [25], [27]. In contrast, MAVRIC performs automatic selfrecognition to produce a controller that adapts to new or altered robots within a few seconds.…”