Learning Backward Compatible Embeddings

Hu, Weihua; Rajas, Bansal,; Cao, Kaidi; Rao, Nikhil; Subbian, Karthik; Leskovec, Jure

doi:10.1145/3534678.3539194

Cited by 10 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This has led rise to the study of compatible representation learning. Shen et al (2020); Budnik & Avrithis (2021); ; Ramanujan et al (2022); Hu et al (2022); Zhao et al (2022); Duggal et al (2021) all proposed methods to update the embedding model to achieve better performance whilst still being compatible with features generated by the old model (see Figure 1-left-top). Despite relative success, compatibility learning is not perfect: performing retrieval with a mixture of old and new features achieves lower accuracies than when we replace all the old features with new ones.…”

Section: Introductionmentioning

confidence: 99%

“…In this work, we focus on closing this performance gap. Further, some previous methods degrade the new model performance when trying to make it more compatible with the old model (Shen et al, 2020;Hu et al, 2022) or requiring availability of side-information, extra features from a separate self-supervised model, (Ramanujan et al, 2022) (which may not be available for an existing system). We relax both constraints in this work.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FastFill: Efficient Compatible Model Update

Jaeckle¹,

Faghri²,

Farhadi³

et al. 2023

Preprint

View full text Add to dashboard Cite

In many retrieval systems the original high dimensional data (e.g., images) is mapped to a lower dimensional feature through a learned embedding model. The task of retrieving the most similar data from a gallery set to a given query data is performed through a similarity comparison on features. When the embedding model is updated, it might produce features that are not comparable/compatible with features already in the gallery computed with the old model. Subsequently, all features in the gallery need to be re-computed using the new embedding model -a computationally expensive process called backfilling. Recently, compatible representation learning methods have been proposed to avoid backfilling. Despite their relative success, there is an inherent trade-off between the new model performance and its compatibility with the old model. In this work, we introduce FastFill: a compatible model update process using feature alignment and policy based partial backfilling to promptly elevate retrieval performance. We show that previous backfilling strategies suffer from decreased performance and demonstrate the importance of both the training objective and the ordering in online partial backfilling. We propose a new training method for feature alignment between old and new embedding models using uncertainty estimation. Compared to previous works, we obtain significantly improved backfilling results on a variety of datasets: mAP on ImageNet (+4.4%), Places-365 (+2.7%), and VGG-Face2 (+1.3%). Further, we demonstrate that when updating a biased model with FastFill, the minority subgroup accuracy gap promptly vanishes with a small fraction of partial backfilling. 1

show abstract