Lightweight deep network-enabled real-time low-visibility enhancement for promoting vessel detection in maritime video surveillance

Abstract

Maritime video surveillance has become an essential part of the vessel traffic services system, intended to guarantee vessel traffic safety and security in maritime applications. To make maritime surveillance more feasible and practicable, many intelligent vision-empowered technologies have been developed to automatically detect moving vessels from maritime visual sensing data (i.e., maritime surveillance videos). However, when visual data is collected in a low-visibility environment, the essential optical information is often hidden in the dark, potentially resulting in decreased accuracy of vessel detection. To guarantee reliable vessel detection under low-visibility conditions, the paper proposes a low-visibility enhancement network (termed LVENet) based on Retinex theory to enhance imaging quality in maritime video surveillance. LVENet is a lightweight deep neural network incorporating a depthwise separable convolution. The synthetically-degraded image generation and hybrid loss function are further presented to enhance the robustness and generalisation capacities of LVENet. Both full-reference and no-reference evaluation experiments demonstrate that LVENet could yield comparable or even better visual qualities than other state-of-the-art methods. In addition, it takes LVENet just 0⋅0045 s to restore degraded images with size 1920 × 1080 pixels on an NVIDIA 2080Ti GPU, which can adequately meet real-time requirements. Using LVENet, vessel detection performance can be greatly improved with enhanced visibility under low-light imaging conditions.

A Guide to Image and Video based Small Object Detection using Deep Learning : Case Study of Maritime Surveillance

“…For image enhancement, deep learning is often combined with physical models. For instance, to improve maritime vessel detection, Guo et al [168] proposed a low-light image enhancement method based on deep learning and the Retinex theory [169]. According to the Retinex theory, the observed image can be decomposed into reflectance and illumination components, so image quality can be improved by enhancing the illumination.…”

“…According to the Retinex theory, the observed image can be decomposed into reflectance and illumination components, so image quality can be improved by enhancing the illumination. To this end, Guo et al [168] proposed to learn a mapping between low-light images and their illumination-enhanced counterparts through a CNNbased model. This model was supervised by pairs of synthetic low-light and normal-light images.…”

“…Prior works for video-based maritime small object detection are typically categorized into: (i) spatial-based (i.e., frame-based) detection and (ii) spatio-temporal based detection. The first category of methods, e.g., [168], [173], [171], [170], [199], [186], generally developed similar strategies compared to their image-based counterparts (see Section 5.1), and detected maritime small objects in videos frame by frame. While these methods (by only using the spatial information) have been able to achieve good detection accuracy and speed in several video based maritime applications, we believe that using the temporal information across video frames could lead to better performance by inferring relationships between moving objects.…”

LiCENt: Low-Light Image Enhancement Using the Light Channel of HSL

“…Retinex-Net [30] used an end-to-end image decomposition model with a successive low-light enhancement network to improve illumination, whereas LightenNet [31] used an incredibly small network to enhance the images. Similarly, LVENet [32] proposed to use retinex theory to estimate the illumination component VOLUME 10, 2022 using a lightweight depthwise separable convolutional. However, their performance is often unsatisfactory, and Lighten-Net [31] unnaturally brightens the center of the image, which varies from the ground truth.…”

Low-Light Image Enhancement: A Comparative Review and Prospects

“…In the scenario of autonomous driving, unevenly-casted shadows often yield irregular dark areas on lanes, thus adjustment of such challenging conditions guides the algorithms for scene understanding, e.g., object detection and recognition in road environments, to be more reliable. For safety and security of the vessel traffic in maritime applications, it is essential to guarantee reliable vessel detection under low-visibility conditions via low-light image enhancement [3]. Based on these practical examples, it is naturally considered that the demand for the deployment of low-light image enhancement into various industrial fields keeps growing year after year.…”