Multi-branch offset architecture for unaligned cross-view geo localization

Abstract

Cross-view geo-localization usually requires matching a ground-view image with the corresponding aerial-view image. This task has attracted researchers'attention in recent years due to the huge differences in perspectives and the occlusion of information between the two views. Previous work has focused more on alignment conditions, where aerial and ground views are aligned north. The explicit alignment information is beneficial to model learning. However, in this work, we extend this task to unaligned conditions and expect the model to perform well even with random offsets in the ground-view image. To solve this problem, we propose a new multi-branch offset (MBO) architecture. Specifically, we divide the feature extracted by the CNN into N parts and shift the feature along the horizontal direction. Features of different offset sizes are then fed into subsequent multi-branch Transformer blocks, and the relationships between channels are further expressed by using channel attention. Finally, the enhanced features are fused with the output of the original CNN. We validated our results on popular dataset, where 78.13% of the Top-1 recalls and 33.78 FPS were made on CVUSA, which is higher than the existing model and achieves efficient inference speed.