Towards Backward-Compatible Representation Learning

Shen, Yantao; Xiong, Yuanjun; Xia, Wei; Soatto, Stefano

doi:10.1109/cvpr42600.2020.00640

Cited by 73 publications

(123 citation statements)

References 31 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…The backward compatibility representation learning first comes into sight in Shen et al (2020) on learning inter-operabile visual embeddings for image retrieval tasks. Later, Yan et al (2020) formalize the model update regression problem in machine learning and explore solutions on image classification tasks.…”

Section: Model Update Regression and Solutionsmentioning

confidence: 99%

See 1 more Smart Citation

Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates

Xie¹,

Lai²,

Xiong³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

Behavior of deep neural networks can be inconsistent between different versions. Regressions 1 during model update are a common cause of concern that often over-weigh the benefits in accuracy or efficiency gain. This work focuses on quantifying, reducing and analyzing regression errors in the NLP model updates. Using negative flip rate as regression measure, we show that regression has a prevalent presence across tasks in the GLUE benchmark. We formulate the regression-free model updates into a constrained optimization problem, and further reduce it into a relaxed form which can be approximately optimized through knowledge distillation training method. We empirically analyze how model ensemble reduces regression. Finally, we conduct CHECKLIST behavioral testing to understand the distribution of regressions across linguistic phenomena, and the efficacy of ensemble and distillation methods. * * Work done while at Amazon AWS AI. 1 Here regression refers to bugs in software testing instead of the statistical estimation method.

show abstract

Section: Model Update Regression and Solutionsmentioning

confidence: 99%

“…The model regression issue in deep learning first comes into sight in Shen et al (2020), where they inspect compatible representation learning for image retrieval. Yan et al (2020) proposed the positive-congruent training (PCT) for image classification that minimizes prediction errors and model regression at the same time.…”

Section: Introductionmentioning

confidence: 99%

Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates

Xie¹,

Lai²,

Xiong³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

show abstract

“…Unlike other tasks, parameters of the last classifier for a face recognition model are crucial for recognition performance but strongly associated with privacy. These parameters can be regarded as mean embeddings of identities (Wang et al, 2018;, 2021b; Shen et al, 2020) (also called as class centers), from where individual privacy could be spied out as studied by plenty of works (Kumar Jindal et al, 2018;Boddeti, 2018;Mai et al, 2020;Dusmanu et al, 2021). That prevents the FL approach from broadcasting the whole model among clients and the central server, and consequently leads to conflicts in the aggregation of local updates.…”

Section: Introductionmentioning

confidence: 99%

Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters

Meng¹,

Zhu²,

Ren³

et al. 2022

Preprint

View full text Add to dashboard Cite

The growing public concerns on data privacy in face recognition can be greatly addressed by the federated learning (FL) paradigm. However, conventional FL methods perform poorly due to the uniqueness of the task: broadcasting class centers among clients is crucial for recognition performances but leads to privacy leakage. To resolve the privacy-utility paradox, this work proposes PrivacyFace, a framework largely improves the federated learning face recognition via communicating auxiliary and privacy-agnostic information among clients. PrivacyFace mainly consists of two components: First, a practical Differentially Private Local Clustering (DPLC) mechanism is proposed to distill sanitized clusters from local class centers. Second, a consensus-aware recognition loss subsequently encourages global consensuses among clients, which ergo results in more discriminative features. The proposed framework is mathematically proved to be differentially private, introducing a lightweight overhead as well as yielding prominent performance boosts (e.g., +9.63% and +10.26% for TAR@FAR=1e-4 on IJB-B and IJB-C respectively). Extensive experiments and ablation studies on a large-scale dataset have demonstrated the efficacy and practicability of our method.

show abstract

“…To harvest the reward of the new model immediately, Shen et al (2020) introduces the compatible representation learning to train the new model with backward compatibility constraints, so that queries encoded by the new model can be directly indexed by the old gallery features. Meanwhile, as the new features and old features are interchangeable with each other, the gallery images can be backfilled on-the-fly, and the retrieval performances would be gradually improved to approach the optimal accuracy of the new model, dubbed as hot-refresh model upgrades (see Figure 1 (a)).…”

Section: Introductionmentioning

confidence: 99%

“…Meanwhile, as the new features and old features are interchangeable with each other, the gallery images can be backfilled on-the-fly, and the retrieval performances would be gradually improved to approach the optimal accuracy of the new model, dubbed as hot-refresh model upgrades (see Figure 1 (a)). Although existing compatible training methods (Meng et al, 2021;Shen et al, 2020) make it possible to upgrade the model in a hot-refresh manner, they still face the challenge of model regression (Yan et al, 2020;Li & Hoiem, 2018), which is actually caused by negative flips, i.e., queries correctly indexed by the old model are incorrectly recognized by the new model (see Figure 1 (b)). We claimed that, during the procedure of hot-refresh model upgrades, the negative flips occur when the new-to-new similarities between negative query-gallery pairs are larger than the new-to-old similarities between compatible positive query-gallery pairs.…”

Section: Introductionmentioning

confidence: 99%

Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrieval

Zhang¹,

Ge²,

Shen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The task of hot-refresh model upgrades of image retrieval systems plays an essential role in the industry but has never been investigated in academia before. Conventional cold-refresh model upgrades can only deploy new models after the gallery is overall backfilled, taking weeks or even months for massive data. In contrast, hot-refresh model upgrades deploy the new model immediately and then gradually improve the retrieval accuracy by backfilling the gallery on-the-fly.Compatible training has made it possible, however, the problem of model regression * with negative flips poses a great challenge to the stable improvement of user experience. We argue that it is mainly due to the fact that new-to-old positive query-gallery pairs may show less similarity than new-to-new negative pairs. To solve the problem, we introduce a Regression-Alleviating Compatible Training (RACT) method to properly constrain the feature compatibility while reducing negative flips. The core is to encourage the new-to-old positive pairs to be more similar than both the new-to-old negative pairs and the new-to-new negative pairs. An efficient uncertainty-based backfilling strategy is further introduced to fasten accuracy improvements. Extensive experiments on large-scale retrieval benchmarks (e.g., Google Landmark) demonstrate that our RACT effectively alleviates the model regression for one more step towards seamless model upgrades § . † Corresponding authors. ‡ Work done when Binjie, Yantao and Yu are at ARC Lab, Tencent PCG. * Throughout this paper, the term model regression indicates the degradation of retrieval performance when the hot-refresh model upgrades proceed.§ The code will be available at https://github.com/binjiezhang/RACT_ICLR2022.

show abstract

Towards Backward-Compatible Representation Learning

Cited by 73 publications

References 31 publications

Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates

Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates

Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters

Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrieval

Contact Info

Product

Resources

About