Fast Stochastic Recursive Momentum Methods for Imbalanced Data Mining

Wu, Xidong; Huang, Feihu; Huang, Heng

doi:10.1109/icdm54844.2022.00068

Cited by 5 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The momentum average is applied to both the outer and inner estimators to track individual ranking scores. More recently, algorithms proposed in [41] reduce convergence complexity with the parallel speed-up and Jiang et al [20], Wu et al [47] introduced the momentum-based variance-reduction technology into AUPRC maximization to reduce the convergence complexity. While we developed distributed AUPRC optimization concurrently with [13], they pay attention to X-Risk Optimization in federated learning.…”

Section: Auprc Maximizationmentioning

confidence: 99%

“…Because X-Risk optimization is a sub-problem in conditional stochastic optimization and federated learning could be regarded as decentralized learning with a specific network topology (seen 3.1), our methods could also be applied to their problem. Overall, existing methods mainly focus on finite-sum singlemachine setting [20,37,41,42,47]. To solve the biased stochastic gradient, they maintain an inner state for local each data point.…”

Section: Auprc Maximizationmentioning

confidence: 99%

“…However, AUROC is not suitable for data with a much larger number of negative examples than positive examples, and AUPRC can address this issue because it doesn't rely on true negatives. Given that an algorithm that maximizes AUROC does not necessarily maximize AUPRC [7] and matching of loss and metric is important [10,11,33], the design of AUPRC maximization algorithms has attracted attention [20,37,41,42,47]. Nonetheless, the multi-party algorithm for AUPRC maximization problems has not been studied.…”

Section: Introductionmentioning

confidence: 99%

“…Traditional gradient-based gradient descent techniques cannot directly be used with the original concept of AP. Most existing optimization works use the surrogate function to replace the ranking function in the AP function [14,20,26,37,42,47]. We can follow these works and substitute a surrogate loss for the ranking function in the AP function.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining

Pei

et al. 2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

To address the big data challenges, multi-party collaborative training, such as distributed learning and federated learning, has recently attracted attention. However, traditional multi-party collaborative training algorithms were mainly designed for balanced data mining tasks and are intended to optimize accuracy (e.g., cross-entropy). The data distribution in many real-world applications is skewed and classifiers, which are trained to improve accuracy, perform poorly when applied to imbalanced data tasks since models could be significantly biased toward the primary class. Therefore, the Area Under Precision-Recall Curve (AUPRC) was introduced as an effective metric. Although single-machine AUPRC maximization methods have been designed, multi-party collaborative algorithm has never been studied. The change from the single-machine to the multi-party setting poses critical challenges. For example, existing single-machine-based AUPRC maximization algorithms maintain an inner state for local each data point, thus these methods are not applicable to large-scale online multi-party collaborative training due to the dependence on each local data point.To address the above challenge, we study serverless multi-party collaborative AUPRC maximization problem since serverless multiparty collaborative training can cut down the communications cost by avoiding the server node bottleneck, and reformulate it as a conditional stochastic optimization problem in a serverless multi-party collaborative learning setting and propose a new ServerLess biAsed sTochastic gradiEnt (SLATE) algorithm to directly optimize the AUPRC. After that, we use the variance reduction technique and propose ServerLess biAsed sTochastic gradiEnt with Momentumbased variance reduction (SLATE-M) algorithm to improve the

show abstract

Section: Auprc Maximizationmentioning

confidence: 99%

Section: Auprc Maximizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining

Pei

et al. 2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

show abstract

“…These methods innovatively updated the biased estimations for each data point and applied momentum averages to both the outer and inner estimators, tracking individual ranking scores. Further advancements in the field [16,17,18] have introduced advanced algorithms that leverage techniques like parallel speed-up and variance-reduction to improve convergence rates. These approaches hold the potential to boost the efficacy of doublet detection tools, facilitating a more accurate annotation of doublets in imbalanced datasets.…”

Section: Introductionmentioning

confidence: 99%

Robust and Accurate Doublet Detection of Single-Cell Sequencing Data via Maximizing Area Under Precision-Recall Curve

Chen,

Wu,

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Single-cell sequencing has revolutionized our understanding of cellular heterogeneity by offering detailed profiles of individual cells within diverse specimens. However, due to the limitations of sequencing technology, two or more cells may be captured in the same droplet and share the same barcode. These incidents, termed doublets or multiplets, can lead to artifacts in single-cell data analysis. While explicit experimental design can mitigate these issues with the help of auxiliary cell markers, computationally annotating doublets has a broad impact on analyzing the existing public single-cell data and reduces potential experimental costs. Considering that doublets form only a minor fraction of the total dataset, we argue that current doublet detection methods, primarily focused on optimizing classification accuracy, might be inefficient in performing well on the inherently imbalanced data in the area under the precision-recall curve (AUPRC) metric. To address this, we introduce RADO (Robust and Accurate DOublet detection) - an algorithm designed to annotate doublets by maximizing the AUPRC, effectively tackling the imbalance challenge. Benchmarked on 18 public datasets, RADO outperforms other methods in terms of doublet score and achieves similar performance to the current best methods in doublet calling. Furthermore, beyond its application in single-cell RNA-seq data, we demonstrate RADO’s adaptability to single-cell assays for transposase-accessible chromatin sequencing (scATAC-seq) data, where it outperforms other scATAC-seq doublet detection methods. RADO’s open-source implementation is available at:https://github.com/poseidonchan/RADO.

show abstract