The poor generalization of deep convolutional networks to aerial imagery from new geographic locations: an empirical study with solar array detection

Wang, Rui; Camilo, Joseph A.; Collins, Leslie M.; Bradbury, Kyle; Malof, Jordan M.

doi:10.1109/aipr.2017.8457965

Cited by 13 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparing the results in Table 2 and Table 1, we also see that there is a substantial performance loss when models are tested on a new domain, corroborating recent evidence [7], [8] that this is a problem. The results here indicate that synthetic imagery may be a viable avenue to help overcome this practical challenge, acting as a complement to other techniques for visual domain adaptation [11], [39].…”

Section: Testing On Previously Unseen Cities (Out-of-domain)supporting

confidence: 82%

“…The second goal is to understand the impact of synthetic imagery when the trained model is evaluated on a novel imagery domain (i.e., imagery collected under novel imaging conditions, or at a new geographic location) with respect to the training imagery, versus a similar domain. Within-domain testing has historically been popular in the literature [6], [17], [38], but recent results [7], [8] indicate that the accuracy of deep learning models drops substantially when applied to novel data -a more challenging scenario, but arguably much more important for real-world application.…”

Section: Data Handling and Performance Metricsmentioning

confidence: 99%

“…recent work [7], [8] indicates that they do not. This result is further corroborated by the work presented here.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation

Kong

Huang

Bradbury

et al. 2020

2020 IEEE Winter Conference on Applications of Computer Vision (WACV)

Self Cite

View full text Add to dashboard Cite

Recently deep learningnamely convolutional neural networks (CNNs)have yielded impressive performance for the task of building segmentation on large overhead (e.g., satellite) imagery benchmarks. However, these benchmark datasets only capture a small fraction of the variability present in real-world overhead imagery, limiting the ability to properly train, or evaluate, models for real-world application. Unfortunately, developing a dataset that captures even a small fraction of real-world variability is typically infeasible due to the cost of imagery, and manual pixel-wise labeling of the imagery. In this work we develop an approach to rapidly and cheaply generate large and diverse virtual environments from which we can capture synthetic overhead imagery for training segmentation CNNs. Using this approach, generate and publicly-release a collection of synthetic overhead imagery termed Synthinel-1 with full pixel-wise building labels. We use several benchmark dataset to demonstrate that Synthinel-1 is consistently beneficial when used to augment real-world training imagery, especially when CNNs are tested on novel geographic locations or conditions.

show abstract

Section: Testing On Previously Unseen Cities (Out-of-domain)supporting

confidence: 82%

Section: Data Handling and Performance Metricsmentioning

confidence: 99%

See 1 more Smart Citation

The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation

Kong

Huang

Bradbury

et al. 2020

2020 IEEE Winter Conference on Applications of Computer Vision (WACV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…This study found that generalization of models trained on a single dataset is relatively challenging when applied to other datasets. This result is consistent with conclusions made by other investigators [7], [8]. We observed some limited cases of strong generalization between models, but without an ability to draw strong conclusions on the basis of qualitative interpretation of the datasets alone, however some inferences were possible.…”

Section: Discussionsupporting

confidence: 92%

“…But the need for a flexibility in generating a global PV registry raises the question how well models trained on single-site data generalize to other datasets, particularly for higher resolution aerial imagery datasets. Studies investigating the generalizability [7], [8] have already shown that neural networks generalize poorly when trained and tested on different cities, even when the images originate from the same data source. Different local characteristics, as geography and population density [8] may impact generalizability.…”

Section: Introductionmentioning

confidence: 99%

Generalizability of Neural Network-based Identification of PV in Aerial Images

Ranalli,

Zech

2023

2023 IEEE 50th Photovoltaic Specialists Conference (PVSC)

View full text Add to dashboard Cite

Identification of PV panels from aerial imagery is a potential strategy for building comprehensive behind-the-meter PV datasets. Several previous studies have utilized Convolutional Neural Networks with the goal of producing tools that can perform these identification tasks. Neural Network approaches rely on labelled data for training, with several aerial imagery datasets with labelled PV already available. This study aims to investigate generalizability of models trained on one set of labelled PV data to other datasets, to further understanding of how these models can be applied. Six different PV datasets were utilized, and test data results were compared. Overall, we find that generalizability suffers when models are presented with different data than they were trained on. We describe some dataset features that led to particularly poor generalization. This study highlights the need for further research to investigate strategies for improving generalizability of trained Neural Network models.

show abstract

3D-PV-Locator: Large-scale detection of rooftop-mounted photovoltaic systems in 3D

et al. 2022

View full text Add to dashboard Cite

The poor generalization of deep convolutional networks to aerial imagery from new geographic locations: an empirical study with solar array detection

Cited by 13 publications

References 10 publications

The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation

The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation

Generalizability of Neural Network-based Identification of PV in Aerial Images

3D-PV-Locator: Large-scale detection of rooftop-mounted photovoltaic systems in 3D

Contact Info

Product

Resources

About