Learning a Dynamic Map of Visual Appearance

Salem, Tawfiq; Workman, Scott; Jacobs, Nathan

doi:10.1109/cvpr42600.2020.01245

Cited by 23 publications

(20 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this, our method must extract discriminative features from the scene appearance and contrast them with the expected appearance for that specific time-of-capture. As variations in appearance over time are highly dependent on the location of the scene, it is essential to provide, as additional context, geographic cues of where the picture was taken [24,33].…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Salem et al [24] learn a dynamic map of visual attributes that captures the relationship between location, time, and the expected appearance of a photograph. Their approach predicts visual attributes from a combination of satellite imagery, time, and geographic location.…”

Section: Metadata Tampering Detectionmentioning

confidence: 99%

“…This secondary task introduces an explainability component to our model, aiding in understanding its decisions. We evaluate the proposed approach both quantitatively and qualitatively, achieving state-of-the-art results on a reference benchmark dataset [24].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Content-Aware Detection of Temporal Metadata Manipulation

Padilha,

Salem,

Workman

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Most pictures shared online are accompanied by a temporal context (i.e., the moment they were taken) that aids their understanding and the history behind them. Claiming that these images were captured in a different moment can be misleading and help to convey a distorted version of reality. In this work, we present the nascent problem of detecting timestamp manipulation. We propose an end-to-end approach to verify whether the purported time of capture of an image is consistent with its content and geographic location. The central idea is the use of supervised consistency verification, in which we predict the probability that the image content, capture time, and geographical location are consistent. We also include a pair of auxiliary tasks, which can be used to explain the network decision. Our approach improves upon previous work on a large benchmark dataset, increasing the classification accuracy from 59.03% to 81.07%. Finally, an ablation study highlights the importance of various components of the method, showing what types of tampering are detectable using our approach.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Metadata Tampering Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Content-Aware Detection of Temporal Metadata Manipulation

Padilha,

Salem,

Workman

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this way, we can extend the information learned from social media to areas where these media are absent, and simultaneously reduce the need to manually annotate satellite images. This approach has been applied for a variety of tasks, including mapping scenes categories [34], and time-varying visual attributes [35]. However, there remain significant issues to address, including:…”

Section: Multimodal Approaches With Social Mediamentioning

confidence: 99%

Toward a Collective Agenda on AI for Earth Science Data Analysis

Tuia¹,

Roscher

Wegner

et al. 2021

IEEE Geosci. Remote Sens. Mag.

Self Cite

View full text Add to dashboard Cite

This is the pre-acceptance version, to read the final version published in the Geoscience and Remote Sensing Magazine, please go to: 10.1109/MGRS.2020.3043504 In the last years we have witnessed the fields of geosciences and remote sensing and artificial intelligence to become closer. Thanks to both the massive availability of observational data, improved simulations, and algorithmic advances, these disciplines have found common objectives and challenges to advance the modeling and understanding of the Earth system. Despite such great opportunities, we also observed a worrying tendency to remain in disciplinary comfort zones applying recent advances from artificial intelligence on well resolved remote sensing problems. Here we take a position on research directions where we think the interface between these fields will have the most impact and become potential game changers. In our declared agenda for AI on Earth sciences, we aim to inspire researchers, especially the younger generations, to tackle these challenges for a real advance of remote sensing and the geosciences.

show abstract

“…There is little literature in which the presence and/or repercussions of geographic representation bias in RSI datasets is either mentioned [54][55][56] or investigated [33,34]. Descriptions of new RSI datasets often reference the need for model's likely performance on real-world data, including any limits of model reliability due to spatial effects [121][122][123][124][125][126][127].…”

Section: A Current Challengesmentioning

confidence: 99%

Methods for Measuring Geodiversity in Large Overhead Imagery Datasets

Wesley

Matisziw

2021

IEEE Access

View full text Add to dashboard Cite

Geographic variation in the appearance of objects on Earth is readily observable in remotely sensed imagery (RSI) and somewhat intuitive to understand for most peoplemany classes of objects (houses, vehicles, crop fields etc.) simply look different depending on their location. This variation has recently been shown to have important implications when training machine learning models on geotagged image datasets for specific object detection and classification tasks. For example, models trained on datasets with ethnocentric biases in image content have been shown to misclassify objects in under-sampled regions, particularly in least-developed countries. The need to evaluate the growing corpus of RSI datasets for representativeness, heterogeneity and geodiversity is therefore high; yet scalable methods for measuring these concepts are absent in the remote sensing domain. This paper introduces the first dataset analysis methods for detecting and assessing geodiversity problems in large RSI datasets, based on geospatial adaptations of the Fréchet Inception Distance and Inception Score in the deep learning framework. Geospatial Fréchet Distance is proposed as a dissimilarity measure for image features of an object class across geographic regions useful for comparing differences in object class appearance in different locations and/or spatial scales. A complementary Geospatial Inception Score is proposed to quantify heterogeneity of geographic context present in dataset labels within particular regions/locations, taking into account the labels themselves as well as their immediate surroundings. Rigorous tests of these methods on simulated RSI datasets demonstrate their stability, sensitivity, and the broad range of dataset analyses to which they can be applied.INDEX TERMS Big geodata, deep convolutional neural networks, explainable ai, geographic domain shift, GIS-remote sensing fusion, imagery interpretation, spatial data analysis This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.

show abstract

Learning a Dynamic Map of Visual Appearance

Cited by 23 publications

References 36 publications

Content-Aware Detection of Temporal Metadata Manipulation

Content-Aware Detection of Temporal Metadata Manipulation

Toward a Collective Agenda on AI for Earth Science Data Analysis

Methods for Measuring Geodiversity in Large Overhead Imagery Datasets

Contact Info

Product

Resources

About