Visual Pose Estimation of Rescue Unmanned Surface Vehicle From Unmanned Aerial System

Abstract

This article addresses the problem of how to visually estimate the pose of a rescue unmanned surface vehicle (USV) using an unmanned aerial system (UAS) in marine mass casualty events. A UAS visually navigating the USV can help solve problems with teleoperation and manpower requirements. The solution has to estimate full pose (both position and orientation) and has to work in an outdoor environment from oblique view angle (up to 85 • from nadir) at large distances (180 m) in real-time (5 Hz) and assume both moving UAS (up to 22 m s −1) and moving object (up to 10 m s −1). None of the 58 reviewed studies satisfied all those requirements. This article presents two algorithms for visual position estimation using the object's hue (thresholding and histogramming) and four techniques for visual orientation estimation using the object's shape while satisfying those requirements. Four physical experiments were performed to validate the feasibility and compare the thresholding and histogramming algorithms. The histogramming had statistically significantly lower position estimation error compared to thresholding for all four trials (p-value ranged from ∼ 0 to 8.23263 × 10 −29), but it only had statistically significantly lower orientation estimation error for two of the trials (p-values 3.51852 × 10 −39 and 1.32762 × 10 −46). The mean position estimation error ranged from 7 to 43 px while the mean orientation estimation error ranged from 0.134 to 0.480 rad. The histogramming algorithm demonstrated feasibility for variations in environmental conditions and physical settings while requiring fewer parameters than thresholding. However, three problems were identified. The orientation estimation error was quite large for both algorithms, both algorithms required manual tuning before each trial, and both algorithms were not robust enough to recover from significant changes in illumination conditions. To reduce the orientation estimation error, inverse perspective warping will be necessary to reduce the perspective distortion. To eliminate the necessity for tuning and increase the robustness, a machine learning approach to pose estimation might ultimately be a better solution.

Collection of Marine Debris by Jointly Using UAV-UUV With GUI for Simple Operation

“…A method of estimating the pose of the UUV using its shape has been proposed [24]. Since the method depends on the shape of the hardware, effective implementation is difficult if the UUV shape has few features.…”

A Quadral-Fuzzy Control Approach to Flight Formation by a Fleet of Unmanned Aerial Vehicles

“…The integration of visual sensing techniques in drone applications is a trend for researches on position-attitude control, pose estimation and mapping, obstacle detection as well as target tracking [13], [22]. Following this trending, we use a 3D perception source providing a cloud of points (or PointClouds), which can collect spatial information from the environment that is combined with information from low-cost sensors (multi-fusion sensor), allowing run procedures for collision and obstacle avoidance and also for path planning.…”

Best Viewpoints for External Robots or Sensors Assisting Other Robots

“…Just like a person, even the most intelligent robot may benefit from a helper that can see from angles the primary robot cannot and thus offer feedback. For example, an autonomous rescue unmanned surface vehicle (USV) can be autonomously navigated to victims by an assistant UAS providing external views, enabling to better estimate the relative pose of the USV to the victims (UAS can see behind waves into more distance and elevated view angle provides better depth perception), as was shown in our previous work [14,15,16,17,18,19]. In dangerous or sensitive situations, a robotic visual assistant can help human supervisors confidently monitor a robot's actions, regardless of whether it is autonomous or teleoperated.…”