Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT

Wang, Jiaqi; Shenglai, Zeng,; Long, Zewei; Wang, Yaqing; Xiao, Houping; Ma, Fenglong

doi:10.1137/1.9781611977653.ch56

Cited by 6 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some unimodal FL studies have investigated the field of self-supervised learning [80,81] and semi-supervised learning [7,[82][83][84] techniques in FL. These studies focused on scenarios in which the system contains no or limited data labels.…”

Section: Weakly Supervised Learningmentioning

confidence: 99%

“…However, the majority of previous works have focused on the unimodal setting, where all the clients in the federated system hold the same data modality, as shown in Figure 1 (left). Among these studies, statistical heterogeneity [3], i.e., the non-IID challenge, caused by the skew of labels, features, and data quantity among clients, is one of the most critical challenges that has attracted much attention [4][5][6][7][8]. In contrast, multimodal federated learning, as shown in Figure 1 (right), further introduced the modality heterogeneity challenge, which led to significant differences in model structures, local tasks, and parameter spaces among clients, thereby exposing the substantial limitations of traditional unimodal algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Federated Learning: A Survey

Che¹,

Wang²,

Zhou³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning (FL) has become a burgeoning and attractive research area, which provides a collaborative training scheme for distributed data sources with privacy concerns. Most existing FL studies focus on taking unimodal data, such as images and text, as the model input and resolving the heterogeneity challenge, i.e., the non-identically distributed (non-IID) challenge caused by data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been proposed to improve the system performance by utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic – multimodal federated learning (MFL) and perform a literature review on the state-of-art MFL methods. Furthermore, we categorize multi-modal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research.

show abstract