An Augmented Reality Periscope for Submarines with Extended Visual Classification

Abstract

Submarines are considered extremely strategic for any naval army due to their stealth capability. Periscopes are crucial sensors for these vessels, and emerging to the surface or periscope depth is required to identify visual contacts through this device. This maneuver has many procedures and usually has to be fast and agile to avoid exposure. This paper presents and implements a novel architecture for real submarine periscopes developed for future Brazilian naval fleet operations. Our system consists of a probe that is connected to the craft and carries a 360 camera. We project and take the images inside the vessel using traditional VR/XR devices. We also propose and implement an efficient computer vision-based MR technique to estimate and display detected vessels effectively and precisely. The vessel detection model is trained using synthetic images. So, we built and made available a dataset composed of 99,000 images. Finally, we also estimate distances of the classified elements, showing all the information in an AR-based interface. Although the probe is wired-connected, it allows for the vessel to stand in deep positions, reducing its exposure and introducing a new way for submarine maneuvers and operations. We validate our proposal through a user experience experiment using 19 experts in periscope operations.

Improving Maritime Domain Awareness in Brazil through Computer Vision Technology

“…Computer vision appears as a solution to support human visual tasks to interpret and understand the maritime domain using Deep Learning Neural Network (DNN) models. The DNN models accurately identify and classify objects [12,25]. Yet, the DNN solutions have been used in several MDA challenges, including detecting and classifying vessel types using synthetic or real-world images.…”

“…Furthermore, many countries have proposed specific MDA frameworks to improve the MDA, such as Greece [10], Canada [8], and the Philippines [11]. On the other hand, the literature presents exhaustive and unreliable MDA solutions using generic methods, such as movement extraction, data aggregation, data analysis, computer vision, and deep learning, through automatic or synthetic data [12].…”

Using Conventional Cameras as Sensors for Estimating Confidence Intervals for the Speed of Vessels from Single Images

“…Using techniques such as YOLO [19] and the training model that was used by Breitinger et al [20] to detect the vessels is straightforward. However, as this was a separate problem that was subject to the quality of the detection technique without the loss of generality, we chose to manually set x 1 and u in image space in our experiments.…”

Augmenting Human Cognition With a Digital Submarine Periscope

“…With this new technology, the periscope watchkeeper, previously tethered to the conventional periscope via eyepieces and handles (Figure 1(a)) providing a partial view of the horizon (Figure 1(b)), could potentially observe the external environment in a variety of alterative ways, including on a computer monitor. The technical and engineering considerations involved in digitally representing sensor/periscope imagery on monitors have received considerable attention, including technologies required for presenting 360° imagery such as rotating components, software algorithms for multiple sensors, and image stitching (Duryea et al, 2008), as well as novel augmented reality architecture (Breitinger et al, 2021). However, to our knowledge, little consideration has been given to the potential impact of alternative digital periscope prototypes 1 (i.e.…”