Sparse Stochastic Online AUC Optimization for Imbalanced Streaming Data

Yang, Min; Cai, Xufen; Hu, Ruimin; Ye, Long; Zhu, Rong

doi:10.1007/978-3-319-77383-4_94

Cited by 1 publication

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a long line of research that investigated the imbalanced data mining with AUROC metric [14,26,52,55,66], which highlight the value of the AUC metric in imbalanced data mining. Earlier works about AUROC focused on linear models with pairwise surrogate losses [21].…”

Section: Related Work 21 Auroc Maximizationmentioning

confidence: 99%

“…Earlier works about AUROC focused on linear models with pairwise surrogate losses [21]. Furthermore, Ying et al [52] solved the AUC square surrogate loss using a stochastic gradient descent ascending approach and provided a minimax reformulation of the loss to address the scaling problem of AUC optimization. Later, Liu et al [26] studied the application of AUROC in deep learning and reconstructed deep AUC as a minimax problem, which offers a strategy to resolve the stochastic AUC maximization problem with a deep neural network as the predictive model.…”

Section: Related Work 21 Auroc Maximizationmentioning

confidence: 99%

See 1 more Smart Citation

Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining

Pei

et al. 2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

To address the big data challenges, multi-party collaborative training, such as distributed learning and federated learning, has recently attracted attention. However, traditional multi-party collaborative training algorithms were mainly designed for balanced data mining tasks and are intended to optimize accuracy (e.g., cross-entropy). The data distribution in many real-world applications is skewed and classifiers, which are trained to improve accuracy, perform poorly when applied to imbalanced data tasks since models could be significantly biased toward the primary class. Therefore, the Area Under Precision-Recall Curve (AUPRC) was introduced as an effective metric. Although single-machine AUPRC maximization methods have been designed, multi-party collaborative algorithm has never been studied. The change from the single-machine to the multi-party setting poses critical challenges. For example, existing single-machine-based AUPRC maximization algorithms maintain an inner state for local each data point, thus these methods are not applicable to large-scale online multi-party collaborative training due to the dependence on each local data point.To address the above challenge, we study serverless multi-party collaborative AUPRC maximization problem since serverless multiparty collaborative training can cut down the communications cost by avoiding the server node bottleneck, and reformulate it as a conditional stochastic optimization problem in a serverless multi-party collaborative learning setting and propose a new ServerLess biAsed sTochastic gradiEnt (SLATE) algorithm to directly optimize the AUPRC. After that, we use the variance reduction technique and propose ServerLess biAsed sTochastic gradiEnt with Momentumbased variance reduction (SLATE-M) algorithm to improve the

show abstract