CycleGAN-Based Image Translation for Near-Infrared Camera-Trap Image Recognition

Gao, Renwu; Zheng, Siting; He, Jinghan; Shen, Linlin

doi:10.1007/978-3-030-59830-3_39

Cited by 2 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Suggested future research would be to explore using different datasets to create these classes, creating a more diverse pool of imagery for the Background class and using camera trap imagery and/or UK species to inform the Animal Other class. Other areas for extended research include: investigating whether the masked boundary around a segmented image has a positive or negative effect on classification, the potential to use style transfer (Generative Adversarial Networks) to map from the captive imagery data domain to that of the wild (to assist generalisation) [40], [41], and the use of monte carlo dropout during testing to estimate confidence of the network [42].…”

Section: Discussionmentioning

confidence: 99%

Application of deep learning to camera trap data for ecologists in planning / engineering -- Can captivity imagery train a model which generalises to the wild?

Curry¹,

Trotter²,

McGough³

2021

Preprint

View full text Add to dashboard Cite

Understanding the abundance of a species is the first step towards understanding both its long-term sustainability and the impact that we may be having upon it. Ecologists use camera traps to remotely survey for the presence of specific animal species. Previous studies have shown that deep learning models can be trained to automatically detect and classify animals within camera trap imagery with high levels of confidence. However, the ability to train these models is reliant upon having enough high-quality training data. What happens when the animal is rare or the data sets are non-existent? This research proposes an approach of using images of rare animals in captivity (we focus on the Scottish wildcat) to generate the training dataset. We explore the challenges associated with generalising a model trained on captivity data when applied to data collected in the wild. The research is contextualised by the needs of ecologists in planning / engineering. Following precedents from other research, this project establishes an ensemble system of object detection, image segmentation and image classification models which are then tested using different image manipulation and class structuring techniques to encourage model generalisation. The research concludes, in the context of Scottish wildcat, that models trained on captivity imagery cannot be generalised to wild camera trap imagery using existing techniques. However, final model performances based on a two-class model (Wildcat vs Not Wildcat) achieved an overall accuracy score of 81.6% and Wildcat accuracy score of 54.8% on a test set in which only 1% of images contained a wildcat. This suggests using captivity images is feasible with further research. This is the first research which attempts to generate a training set based on captivity data and the first to explore the development of such models in the context of ecologists in planning / engineering.

show abstract

Section: Discussionmentioning

confidence: 99%

Application of deep learning to camera trap data for ecologists in planning / engineering -- Can captivity imagery train a model which generalises to the wild?

Curry¹,

Trotter²,

McGough³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Additionally, in recent years the combination of camera trapping and of artificial intelligence (AI), especially of deep learning approaches, has emerged as a breakthrough in the field of wildlife research and conservation [ 3 , 7 , 11 , 12 ]. However, many deep learning approaches are trained for and can benefit from colored images [ 13 ] like humans do [ 10 ]. This raises the question if deep learning approaches can also benefit from such artificially colored images.…”

Section: Introductionmentioning

confidence: 99%

“…It uses ResNet [ 24 ] as the generator network [ 23 ]. Gao et al [ 13 ] trained CycleGAN on a wildlife dataset and showed improved recognition results on the generated images compared to the NIR images. Mehri and Sappa [ 14 ] proposed a version of CycleGAN, specifically designed for the task of colorizing NIR images that incorporates enhanced loss functions and utilizes U-Net as a generator.…”

Section: Introductionmentioning

confidence: 99%

The Potential of Diffusion-Based Near-Infrared Image Colorization

Borstelmann,

Haucke,

Steinhage

2024

Sensors

View full text Add to dashboard Cite

Camera traps, an invaluable tool for biodiversity monitoring, capture wildlife activities day and night. In low-light conditions, near-infrared (NIR) imaging is commonly employed to capture images without disturbing animals. However, the reflection properties of NIR light differ from those of visible light in terms of chrominance and luminance, creating a notable gap in human perception. Thus, the objective is to enrich near-infrared images with colors, thereby bridging this domain gap. Conventional colorization techniques are ineffective due to the difference between NIR and visible light. Moreover, regular supervised learning methods cannot be applied because paired training data are rare. Solutions to such unpaired image-to-image translation problems currently commonly involve generative adversarial networks (GANs), but recently, diffusion models gained attention for their superior performance in various tasks. In response to this, we present a novel framework utilizing diffusion models for the colorization of NIR images. This framework allows efficient implementation of various methods for colorizing NIR images. We show NIR colorization is primarily controlled by the translation of the near-infrared intensities to those of visible light. The experimental evaluation of three implementations with increasing complexity shows that even a simple implementation inspired by visible-near-infrared (VIS-NIR) fusion rivals GANs. Moreover, we show that the third implementation is capable of outperforming GANs. With our study, we introduce an intersection field joining the research areas of diffusion models, NIR colorization, and VIS-NIR fusion.

show abstract

CycleGAN-Based Image Translation for Near-Infrared Camera-Trap Image Recognition

Cited by 2 publications

References 14 publications

Application of deep learning to camera trap data for ecologists in planning / engineering -- Can captivity imagery train a model which generalises to the wild?

Application of deep learning to camera trap data for ecologists in planning / engineering -- Can captivity imagery train a model which generalises to the wild?

The Potential of Diffusion-Based Near-Infrared Image Colorization

Contact Info

Product

Resources

About