Bayesian Federated Neural Matching That Completes Full Information

Abstract

Federated learning is a contemporary machine learning paradigm where locally trained models are distilled into a global model. Due to the intrinsic permutation invariance of neural networks, Probabilistic Federated Neural Matching (PFNM) employs a Bayesian nonparametric framework in the generation process of local neurons, and then creates a linear sum assignment formulation in each alternative optimization iteration. But according to our theoretical analysis, the optimization iteration in PFNM omits global information from existing. In this study, we propose a novel approach that overcomes this flaw by introducing a Kullback-Leibler divergence penalty at each iteration. The effectiveness of our approach is demonstrated by experiments on both image classification and semantic segmentation tasks.

Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning

“…It solves the matched averaging objective by iterative optimization. Through theoretical analysis, Xiao and Cheng [41] found that global information can be omitted by PFNM. To fix this missing global information issue, an algorithm that conducts neural aggregation with full information (NAFI) is developed.…”