Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation

Bovcon, Borja; Mandeljc, Rok; Perš, Janez; Kristan, Matej

doi:10.1016/j.robot.2018.02.017

Cited by 115 publications

(77 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bovcon et al explored semantics segmentation assisted by an inertial measurement unit (IMU) for stereo obstacle detection [27]. Cane et al evaluated the semantic segmentation networks on several public maritime datasets and compared their performances [28].…”

Section: Visible Vessel Detection and Trackingmentioning

confidence: 99%

M3C: Multimodel-and-Multicue-Based Tracking by Detection of Surrounding Vessels in Maritime Environment for USV

et al. 2019

View full text Add to dashboard Cite

It is crucial for unmanned surface vessels (USVs) to detect and track surrounding vessels in real time to avoid collisions at sea. However, the harsh maritime environment poses great challenges to multitarget tracking (MTT). In this paper, a novel tracking by detection framework that integrates the multimodel and multicue (M3C) pipeline is proposed, which aims at improving the detection and tracking performance. Regarding the multimodel, we predicted the maneuver probability of a target vessel via the gated recurrent unit (GRU) model with an attention mechanism, and fused their respective outputs as the output of a kinematic filter. We developed a hybrid affinity model based on multi cues, such as the motion, appearance, and attitude of the ego vessel in the data association stage. By using the proposed ship re-identification approach, the tracker had the capability of appearance matching via metric learning. Experimental evaluation of two public maritime datasets showed that our method achieved state-of-the-art performance, not only in identity switches (IDS) but also in frame rates.

show abstract

Section: Visible Vessel Detection and Trackingmentioning

confidence: 99%

M3C: Multimodel-and-Multicue-Based Tracking by Detection of Surrounding Vessels in Maritime Environment for USV

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Modd2 [188], also contains a listing of semantic segmentation method capable to perform in marine environment [186], [187], [189]- [194].…”

Section: A Semantic Segmentationmentioning

confidence: 99%

Collaborative Multi-Robot Search and Rescue: Planning, Coordination, Perception, and Active Vision

et al. 2020

View full text Add to dashboard Cite

Search and rescue (SAR) operations can take significant advantage from supporting autonomous or teleoperated robots and multi-robot systems. These can aid in mapping and situational assessment, monitoring and surveillance, establishing communication networks, or searching for victims. This paper provides a review of multi-robot systems supporting SAR operations, with system-level considerations and focusing on the algorithmic perspectives for multi-robot coordination and perception. This is, to the best of our knowledge, the first survey paper to cover (i) heterogeneous SAR robots in different environments, (ii) active perception in multi-robot systems, while (iii) giving two complementary points of view from the multi-agent perception and control perspectives. We also discuss the most significant open research questions: shared autonomy, sim-to-real transferability of existing methods, awareness of victims' conditions, coordination and interoperability in heterogeneous multi-robot systems, and active perception. The different topics in the survey are put in the context of the different challenges and constraints that various types of robots (ground, aerial, surface, or underwater) encounter in different SAR environments (maritime, urban, wilderness, or other post-disaster scenarios). The objective of this survey is to serve as an entry point to the various aspects of multi-robot SAR systems to researchers in both the machine learning and control fields by giving a global overview of the main approaches being taken in the SAR robotics area.

show abstract

“…However, the optical flow is very complex and time-consuming, and the method is therefore unsuitable for real-time omni-directional obstacle detection. In [31], inertial measurement unit (IMU) and other synchronized sensors are used to detect obstacles. In [22], fewer RGB cameras are applied to realize omni-directional obstacle detection.…”

Section: Related Studiesmentioning

confidence: 99%

Omni-Directional Obstacle Detection for Vehicles Based on Depth Camera

Zhao¹,

Wu²,

Xu³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

At present, the vehicle obstacle detection system usually uses different devices or sensors to perceive and obtain the obstacle information. However, omni-directional obstacle detection is difficult to realize because these devices or sensors are usually easy to be affected by environmental lighting and the material properties of the obstacle surface. Furthermore, most sensors have limited information regarding distance, which limits their application to omni-directional obstacle detection. To solve this problem, this paper proposes a method using depth camera for omni-directional obstacle detection. A method applying region growth for depth image and a fast inpainting method for depth image are proposed to extract and repair the obstacle regions in the depth images obtained by installing depth cameras around the car body. An improved method applying iterative normalized cut is also proposed to cluster and segment fragmentary and irregular obstacle regions to generate the complete obstacle regions. Finally, the obstacle regions are overviewed using a three-dimensional visualization method to realize omni-directional obstacle viewing. The results of experiments conducted in an environment with different obstacles during the day and night demonstrate that, compared with other methods, our proposed approach can effectively promote the ability to detect complex obstacles, and largely improve the detection speed; furthermore, obstacle detection using our method will be unaffected by environmental lighting. Each of these advantages provided by our method can significantly promote the driving safety of unmanned or other types of vehicles.

show abstract

Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation

Cited by 115 publications

References 41 publications

M3C: Multimodel-and-Multicue-Based Tracking by Detection of Surrounding Vessels in Maritime Environment for USV

M3C: Multimodel-and-Multicue-Based Tracking by Detection of Surrounding Vessels in Maritime Environment for USV

Collaborative Multi-Robot Search and Rescue: Planning, Coordination, Perception, and Active Vision

Omni-Directional Obstacle Detection for Vehicles Based on Depth Camera

Contact Info

Product

Resources

About