Street-level imagery analytics and applications

Zhang, Fan; Wegner, Jan Dirk; Wang, Jinhu; Liu, Yu

doi:10.1016/j.isprsjprs.2023.04.005

Cited by 4 publications

(1 citation statement)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on this set of data, we can construct a mapping relationship between image features and image attributes to achieve property prediction for unknown street view images. Furthermore, by adding annotations of receptive fields to the street view images, we can also carry out tasks such as building classification, building extraction, and building height prediction (Kang et al., 2018; Yan & Huang, 2022; Zhang et al., 2023).…”

Section: Discussionmentioning

confidence: 99%

Multimodal learning with only image data: A deep unsupervised model for street view image retrieval by fusing visual and scene text features of images

Wu,

Yu,

Zhang

et al. 2024

Transactions in GIS

View full text Add to dashboard Cite

As one of the classic tasks in information retrieval, the core of image retrieval is to identify the images sharing similar features with a query image, aiming to enable users to find the required information from a large number of images conveniently. Street view image retrieval, in particular, finds extensive applications in many fields, such as improvements to navigation and mapping services, formulation of urban development planning scheme, and analysis of historical evolution of buildings. However, the intricate foreground and background details in street view images, coupled with a lack of attribute annotations, render it among the most challenging issues in practical applications. Current image retrieval research mainly uses the visual model that is completely dependent on the image visual features, and the multimodal learning model that necessitates additional data sources (e.g., annotated text). Yet, creating annotated datasets is expensive, and street view images, which contain a large amount of scene texts themselves, are often unannotated. Therefore, this paper proposes a deep unsupervised learning algorithm that combines visual and text features from image data for improving the accuracy of street view image retrieval. Specifically, we employ text detection algorithms to identify scene text, utilize the Pyramidal Histogram of Characters encoding predictor model to extract text information from images, deploy deep convolutional neural networks for visual feature extraction, and incorporate a contrastive learning module for image retrieval. Upon testing across three street view image datasets, the results demonstrate that our model holds certain advantages over the state‐of‐the‐art multimodal models pre‐trained on extensive datasets, characterized by fewer parameters and lower floating point operations. Code and data are available at https://github.com/nwuSY/svtRetrieval.

show abstract

Section: Discussionmentioning

confidence: 99%