Coupling Rendering and Generative Adversarial Networks for Artificial SAS Image Generation

Reed, Albert; Gerg, Isaac D.; McKay, John; Brown, Daniel C.; Williamsk, David P.; Jayasuriya, Suren

doi:10.23919/oceans40490.2019.8962733

Cited by 18 publications

(14 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently generative adversarial neural networks [1] have been applied to such problems. Such methods perform image processing so that artificial objects matches the background in colour and lighting [7,8,9]. However, the geometric position and shape of the embedded objects are still not taken into account.…”

Section: Related Work 1synthetic Image Generation and Processingmentioning

confidence: 99%

Road images augmentation with synthetic traffic signs using neural networks

2021

View full text Add to dashboard Cite

Traffic sign recognition is a well-researched problem in computer vision. However, the state of the art methods works only for frequent sign classes, which are well represented in training datasets. We consider the task of rare traffic sign detection and classification. We aim to solve that problem by using synthetic training data. Such training data is obtained by embedding synthetic images of signs in the real photos. We propose three methods for making synthetic signs consistent with a scene in appearance. These methods are based on modern generative adversarial network (GAN) architectures. Our proposed methods allow realistic embedding of rare traffic sign classes that are absent in the training set. We adapt a variational autoencoder for sampling plausible locations of new traffic signs in images. We demonstrate that using a mixture of our synthetic data with real data improves the accuracy of both classifier and detector.

show abstract

Section: Related Work 1synthetic Image Generation and Processingmentioning

confidence: 99%

Road images augmentation with synthetic traffic signs using neural networks

2021

View full text Add to dashboard Cite

show abstract

“…For the pre-training step of the TransfGAN, explained in Section 2.3, ray-traced images from tires are used. Similar to [10,12], the ray-tracer Povray is used to generate synthetic sidescan sonar snippets. Those images can be calculated quickly and in different scenarios, e.g., distance from the AUV to the object.…”

Section: Ray-traced Sonar Snippetsmentioning

confidence: 99%

“…Recently, GANs have been applied to the field of underwater acoustic imaging [9][10][11][12]. By generating synthetic high-quality sonar images, the necessity of a large database of real images may be reduced.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generating Synthetic Sidescan Sonar Snippets Using Transfer-Learning in Generative Adversarial Networks

Steiniger

Kraus

Meisen

2021

JMSE

View full text Add to dashboard Cite

The training of a deep learning model requires a large amount of data. In case of sidescan sonar images, the number of snippets from objects of interest is limited. Generative adversarial networks (GAN) have shown to be able to generate photo-realistic images. Hence, we use a GAN to augment a baseline sidescan image dataset with synthetic snippets. Although the training of a GAN with few data samples is likely to cause mode collapse, a combination of pre-training using simple simulated images and fine-tuning with real data reduces this problem. However, for sonar data, we show that this approach of transfer-learning a GAN is sensitive to the pre-training step, meaning that the vanishing of the gradients of the GAN’s discriminator becomes a critical problem. Here, we demonstrate how to overcome this problem, and thus how to apply transfer-learning to GANs for generating synthetic sidescan snippets in a more robust way. Additionally, in order to further investigate the GAN’s ability to augment a sidescan image dataset, the generated images are analyzed in the image and the frequency domain. The work helps other researchers in the field of sonar image processing to augment their dataset with additional synthetic samples.

show abstract

“…To combat this issue, general adversarial network (GANs) have recently been applied to SAS for the purpose of generating more training data to balance the classes. In [31], a hybrid simulation and GAN based approach is used to generate a simulated, optical version of the desired scene and then a learned transform is applied to the simulated scene to give the appearance of a real SAS image. Their hybrid approach gives fine control over the generated scene content so the data balancing procedure can be accomplished with precision; particular objects, their orientations, and their range from the sonar can be specifically generated.…”

Section: Previous Workmentioning

confidence: 99%

Structural Prior Driven Regularized Deep Learning for Sonar Image Classification

Gerg¹,

Monga²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning has been recently shown to improve performance in the domain of synthetic aperture sonar (SAS) image classification. Given the constant resolution with range of a SAS, it is no surprise that deep learning techniques perform so well. Despite deep learning's recent success, there are still compelling open challenges in reducing the high false alarm rate and enabling success when training imagery is limited, which is a practical challenge that distinguishes the SAS classification problem from standard image classification set-ups where training imagery may be abundant. We address these challenges by exploiting prior knowledge that humans use to grasp the scene. These include unconscious elimination of the image speckle and localization of objects in the scene. We introduce a new deep learning architecture which incorporates these priors with the goal of improving automatic target recognition (ATR) from SAS imagery. Our proposal -called SPDRDL, Structural Prior Driven Regularized Deep Learning -incorporates the previously mentioned priors in a multi-task convolutional neural network (CNN) and requires no additional training data when compared to traditional SAS ATR methods. Two structural priors are enforced via regularization terms in the learning of the network:(1) structural similarity prior -enhanced imagery (often through despeckling) aids human interpretation and is semantically similar to the original imagery and (2) structural scene context priors -learned features ideally encapsulate target centering information; hence learning may be enhanced via a regularization that encourages fidelity against known ground truth target shifts (relative target position from scene center). Experiments on a challenging real-world dataset reveal that SPDRDL outperforms state-of-the-art deep learning and other competing methods for SAS image classification.

show abstract

Coupling Rendering and Generative Adversarial Networks for Artificial SAS Image Generation

Cited by 18 publications

References 40 publications

Road images augmentation with synthetic traffic signs using neural networks

Road images augmentation with synthetic traffic signs using neural networks

Generating Synthetic Sidescan Sonar Snippets Using Transfer-Learning in Generative Adversarial Networks

Structural Prior Driven Regularized Deep Learning for Sonar Image Classification

Contact Info

Product

Resources

About