Wave-Tracking in the Surf Zone Using Coastal Video Imagery with Deep Neural Networks

Abstract

In this paper, we propose a series of procedures for coastal wave-tracking using coastal video imagery with deep neural networks. It consists of three stages: video enhancement, hydrodynamic scene separation and wave-tracking. First, a generative adversarial network, trained using paired raindrop and clean videos, is applied to remove image distortions by raindrops and to restore background information of coastal waves. Next, a hydrodynamic scene of propagated wave information is separated from surrounding environmental information in the enhanced coastal video imagery using a deep autoencoder network. Finally, propagating waves are tracked by registering consecutive images in the quality-enhanced and scene-separated coastal video imagery using a spatial transformer network. The instantaneous wave speed of each individual wave crest and breaker in the video domain is successfully estimated through learning the behavior of transformed and propagated waves in the surf zone using deep neural networks. Since it enables the acquisition of spatio-temporal information of the surf zone though the characterization of wave breakers inclusively wave run-up, we expect that the proposed framework with the deep neural networks leads to improve understanding of nearshore wave dynamics.

Raindrop-Aware GAN: Unsupervised Learning for Raindrop-Contaminated Coastal Video Enhancement

“…It shows that the best reconstruction performance can be obtained when the coastal video is used for fine-tuning even in the same model architecture of Raindrop-aware GAN. By creating timestack image and visually assessment it, we can confirm that the performance of the proposed method is the best and it also has high applicability in studying nearshore wave dynamics with video remote sensing, in particular data preparation step, such as breaking wave height estimation from coastal video [31,32], video sensing of nearshore bathymetry evolution [33,34], nearshore wave transform with video imagery [35], shoreline response and resilience through video monitoring [36][37][38][39], wave run-up prediction [40,41], and nearshore wave tracking through coastal video [42,43].…”

Deep visual domain adaptation and semi-supervised segmentation for understanding wave elevation using wave flume video images

“…Although it is a method that can estimate the sea surface elevation from the top-view video images and is highly applicable to the coastal area, a varying wave incidence angle, multi-directional seas, breaking waves, natural lighting condition, location of camera must be considered in the real world domain. It is expected that deep learning-based coastal video enhancement and hydrodynamic scene separation algorithms such as upsampling can be used in the image pre-processing step by collecting enough field data for resolution degradation due to camera position or large variability in natural light except at night 4 , 14 . In addition, even in the present 2D wave flume experiment, some errors in water elevation estimation occurs due to the afterimage effect caused by wave breaking (see Fig.…”

“…In particular, land-based remote sensing devices, such as shore-based camera and video systems, enable synoptic surface and subsurface observations with high temporal resolutions over long time scales, even in the case of extreme events 1 . These devices have also been used to measure shoreline positions and infer subsurface morphology as well as to measure the water waves of the inner surf and swash, in addition to sub-aerial bathymetry 2 – 4 .…”

The Coastal Imaging Research Network (CIRN)

“…In the five years since CIRN's inception, the use of machine learning algorithms to exploit coastal imagery has grown. Data-driven approaches have been used to extract nearshore parameters from imagery including bathymetry [78,79], wave heights [80], wave breaking type [81] and occurrence [82][83][84][85], shoreline position [86], and land cover [87]; to classify nearshore morphology [88]; and to identify dangerous flows, like rip currents [89]. These data-driven approaches can increase analysis capabilities, enabling rapid extraction of data and improving ease of use in engineering and science applications.…”