FedExg: Federated Learning with Model Exchange

Mao, Zhicheng; Dai, Wenrui; Li, Chenglin; Xu, Yuhui; Wang, Shanghong; Xiong, Hongkai

doi:10.1109/iscas45731.2020.9181029

Cited by 7 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notably, the crosstraining strategy, as an orthogonal approach to regularization techniques, can be combined with them to further enhance performance gains. For instance, FedExg [15] and FedMe [16] leverage cross-training strategies to enhance the generalization of the global model and the personalized capabilities of the local models, respectively. PGCT alleviates the problem of local knowledge forgetting due to data heterogeneity and employs prototypes to guide clients in learning similar decision boundaries [17].…”

Section: Related Work a Federated Learning With Non-iid Datamentioning

confidence: 99%

“…We conduct comparisons between FedCT and existing methods, categorized into three groups: 1) local training without federated learning, SOLO; 2) federated learning (FL) methods without cross-training, including FedAvg [2], FedProx [8], MOON [13], FedDC [11], FedASAM [61], FedProc [18], FedDecorr [27], FedIOD [62]; 3) FL methods with crosstraining, including FedExg [15] and PGCT [17]. The results presented in Table I…”

Section: B Performance Comparisonmentioning

confidence: 99%

“…To mitigate the issue of data distribution heterogeneity, existing methods typically focus on improving the performance of local models on multi-source datasets, rather than on a single data source. They either integrate cross-training strategies to assist local models in training on multi-source datasets [15]- [17], which can expand the knowledge base that local models can learn from, or use external knowledge to guide the training of local models. This adjustment in their optimization objectives enhances the cooperative modeling capabilities among local models [13], [18], [19].…”

mentioning

confidence: 99%

“…This adjustment in their optimization objectives enhances the cooperative modeling capabilities among local models [13], [18], [19]. For example, FedExg is recognized as the first cross-training method, which achieves an impressive performance [15]. Moreover, FedMe employs mutual learning techniques to enable client models to acquire knowledge from other data sources [16].…”

mentioning

confidence: 99%

“…It has been shown that regularization methods offer a means of addressing the challenge of data heterogeneity.Moreover, cross-training has emerged as a promising strategy for addressing the problem of dataset bias. It aims to perform model exchange in a privacy-preserving manner to re-train a local model across different clients[15]-[17]. This expands the training data for local models, which enables them to learn comprehensive knowledge.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Clustering-based Curriculum Construction for Sample-Balanced Federated Learning

Wang

Chen

et al. 2022

Artificial Intelligence

View full text Add to dashboard Cite

Federated learning benefits from cross-training strategies, which enables models to train on data from distinct sources to improve the generalization capability. However, the data heterogeneity between sources may lead models to gradually forget previously acquired knowledge when undergoing crosstraining to adapt to new tasks or data sources. We argue that integrating personalized and global knowledge to gather information from multiple perspectives could potentially improve performance. To achieve this goal, this paper presents a novel approach that enhances federated learning through a cross-training scheme incorporating multi-view information. Specifically, the proposed method, termed FedCT, includes three main modules, where the consistency-aware knowledge broadcasting module aims to optimize model assignment strategies, which enhances collaborative advantages between clients and achieves an efficient federated learning process. The multi-view knowledge-guided representation learning module leverages fused prototypical knowledge from both global and local views to enhance the preservation of local knowledge before and after model exchange, as well as to ensure consistency between local and global knowledge. The mixup-based feature augmentation module aggregates rich information to further increase the diversity of feature spaces, which enables the model to better discriminate complex samples. Extensive experiments were conducted on four datasets in terms of performance comparison, ablation study, in-depth analysis and case study. The results demonstrated that FedCT alleviates knowledge forgetting from both local and global views, which enables it outperform state-of-the-art methods.

show abstract

Section: Related Work a Federated Learning With Non-iid Datamentioning

confidence: 99%