Learning Semantic Correspondence with Sparse Annotations

College students’ perceptions of AI-based writing learning tools: With a focus on Google Translate, Naver Papago, and Grammarly

“…The capability of heavy matching networks hinges on data quantity at first, but the training data remains significantly smaller than other computer vision tasks (e.g., 1.2M images in ImageNet-1K (Russakovsky et al 2015)). Therefore, we argue that previous approaches (Laskar and Kannala 2018;Kim et al 2022;Truong et al 2022;Huang et al 2022), attempting to densify points for training, may not be an underlying solution for the data-hungry problem.…”

“…Recent methods for semantic correspondence (Min et al 2019a;Liu et al 2020;Li et al 2020a;Li et al 2021;Zhao et al 2021;Min et al 2020;Cho et al 2021) inevitably train complicated matching networks to maximize performance in a supervised manner with limited qualified dataset (Ham et al 2017;Min et al 2019b), which leads to high computational demands and poor generalization capability across datasets. Some unsupervised strategies (Laskar and Kannala 2018;Truong et al 2022;Kim et al 2022;Huang et al 2022) extend their unsupervised loss to the supervised regime and significantly improve the performance of the previous supervised approaches. This shows that the performance of the existing supervised model was not fully learned due to a lack of data.…”

“…This shows that the performance of the existing supervised model was not fully learned due to a lack of data. Specifically, the methods (Laskar and Kannala 2018;Truong et al 2022) use a cycle consistency for unsupervised loss signal, and the others (Kim et al 2022;Huang et al 2022) utilize pseudo-labels, generated by the model's prediction between real images, combined with confidence measures to guarantee the quality of pseudo-labels. Unlike previous methods (Laskar and Kannala 2018;Truong et al 2022;Kim et al 2022;Huang et al 2022), we introduce a fundamental approach to address the lack of both image and point pairs by utilizing massive unlabeled data.…”

“…To overcome the lack of labeled image pairs, various methods focused on unsupervised strategies (Rocco, Arandjelovic, and Sivic 2017;Jeon et al 2020;Sarlin et al 2020;Laskar and Kannala 2018;Truong et al 2022;Kim et al 2022;Huang et al 2022) to increase the amount of correspondences supervision on unlabeled data in a self-supervised or weaklysupervised way. In particular, the weakly-supervised methods attempted to solve the label sparsity problem by using a cycle consistency (Laskar and Kannala 2018;Truong et al 2022) or pseudo-labels (Kim et al 2022;Huang et al 2022) on real image pairs for unsupervised loss signal, but they only rely on image pairs provided by the dataset. The capability of heavy matching networks hinges on data quantity at first, but the training data remains significantly smaller than other computer vision tasks (e.g., 1.2M images in ImageNet-1K (Russakovsky et al 2015)).…”

“…In this paper, we present a fundamental approach that focuses on overcoming the insufficiency of both image pairs and point pairs, different from previous methods (Laskar and Kannala 2018;Kim et al 2022;Truong et al 2022;Huang et al 2022). Our key idea is that a large corpus of unlabeled image pairs, having rich semantic information, has not been fully used since they remain unannotated.…”