Online Active Learning for Drifting Data Streams

Liu, Sanmin; Xue, Shan; Wu, Jia; Zhou, Chuan; Yang, Jian; Li, Zhao; Cao, Jie

doi:10.1109/tnnls.2021.3091681

Cited by 39 publications

(7 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep models are extensive applied with their unparalleled ability to learn representations (Liu et al 2021;Cao et al 2019;Xu et al 2021a,b). As a part of them, self-supervised learning methods have attracted a deal of attention with their outstanding performance in areas such as computer vision (Song et al 2018;.…”

Section: Related Work Contrastive Representation Learningmentioning

confidence: 99%

“…Existing works generally treat node representations or the graph summary as anchors (Velickovic et al 2019;Zeng and Xie 2021;Ren, Bai, and Zhang 2021;Cao et al 2021;Sun et al 2019). For instance, DGI and MVGRL treat the graph summary as anchors, which is first convolved by GCN and then summarized by a readout function.…”

Section: Anchor and Negative Embedding Generationmentioning

confidence: 99%

See 1 more Smart Citation

Simple Unsupervised Graph Representation Learning

Liang

et al. 2022

AAAI

View full text Add to dashboard Cite

In this paper, we propose a simple unsupervised graph representation learning method to conduct effective and efficient contrastive learning. Specifically, the proposed multiplet loss explores the complementary information between the structural information and neighbor information to enlarge the inter-class variation, as well as adds an upper bound loss to achieve the finite distance between positive embeddings and anchor embeddings for reducing the intra-class variation. As a result, both enlarging inter-class variation and reducing intra-class variation result in small generalization error, thereby obtaining an effective model. Furthermore, our method removes widely used data augmentation and discriminator from previous graph contrastive learning methods, meanwhile available to output low-dimensional embeddings, leading to an efficient model. Experimental results on various real-world datasets demonstrate the effectiveness and efficiency of our method, compared to state-of-the-art methods. The source codes are released at https://github.com/YujieMo/SUGRL.

show abstract

Section: Related Work Contrastive Representation Learningmentioning

confidence: 99%

Section: Anchor and Negative Embedding Generationmentioning

confidence: 99%

Simple Unsupervised Graph Representation Learning

Liang

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…Active learning, also known as query learning, assumes that different samples in the dataset have different values for the model training, and tries to select a small quantity of data to achieve high performance gains [37]. Based on the query principle they employ, active learning methods can be split into three categories: uncertainty-based methods [4,23,34,42], diversity-based methods [5,13,16,29,46], and expected model change-based ones [12,14,38,41]. In this work we follow the principle of diversity and propose an unbalanced sampling strategy.…”

Section: Active Learningmentioning

confidence: 99%

From Known to Unknown: Quality-aware Self-improving Graph Neural Network for Open Set Social Event Detection

Ren¹,

Jiang²,

Peng³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

State-of-the-art Graph Neural Networks (GNNs) have achieved tremendous success in social event detection tasks when restricted to a closed set of events. However, considering the large amount of data needed for training a neural network and the limited ability of a neural network in handling previously unknown data, it remains a challenge for existing GNN-based methods to operate in an open set setting. To address this problem, we design a Quality-aware Self-improving Graph Neural Network (QSGNN) which extends the knowledge from known to unknown by leveraging the best of known samples and reliable knowledge transfer. Specifically, to fully exploit the labeled data, we propose a novel supervised pairwise loss with an additional orthogonal inter-class relation constraint to train the backbone GNN encoder. The learnt, already-known events further serve as strong reference bases for the unknown ones, which greatly prompts knowledge acquisition and transfer. When the model is generalized to unknown data, to ensure the effectiveness and reliability, we further leverage the reference similarity distribution vectors for pseudo pairwise label generation, selection and quality assessment. Following the diversity principle of active learning, our method selects diverse pair samples with the generated pseudo labels to fine-tune the GNN encoder. Besides, we propose a novel quality-guided optimization in which the contributions of pseudo labels are weighted based on consistency. We thoroughly evaluate our model on two large real-world social

show abstract

“…It aims to identify data points that do not conform to expected behaviors. Since anomalies usually provide critical information, AD has been widely-used in various applications, such as health care [33], [35], network intrusion detection [21], [39], fraud detection [30] and other areas [8], [26]. In many realistic scenarios, there is no ground truth available to distinguish anomalous instances from the normal ones.…”

Section: Introductionmentioning

confidence: 99%

Diminishing Empirical Risk Minimization for Unsupervised Anomaly Detection

Wang¹,

Liu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Unsupervised anomaly detection (AD) is a challenging task in realistic applications. Recently, there is an increasing trend to detect anomalies with deep neural networks (DNN). However, most popular deep AD detectors cannot protect the network from learning contaminated information brought by anomalous data, resulting in unsatisfactory detection performance and overfitting issues. In this work, we identify one reason that hinders most existing DNN-based anomaly detection methods from performing is the wide adoption of the Empirical Risk Minimization (ERM). ERM assumes that the performance of an algorithm on an unknown distribution can be approximated by averaging losses on the known training set. This averaging scheme thus ignores the distinctions between normal and anomalous instances. To break through the limitations of ERM, we propose a novel Diminishing Empirical Risk Minimization (DERM) framework. Specifically, DERM adaptively adjusts the impact of individual losses through a well-devised aggregation strategy. Theoretically, our proposed DERM can directly modify the gradient contribution of each individual loss in the optimization process to suppress the influence of outliers, leading to a robust anomaly detector. Empirically, DERM outperformed the stateof-the-art on the unsupervised AD benchmark consisting of 18 datasets.

show abstract

Online Active Learning for Drifting Data Streams

Cited by 39 publications

References 46 publications

Simple Unsupervised Graph Representation Learning

Simple Unsupervised Graph Representation Learning

From Known to Unknown: Quality-aware Self-improving Graph Neural Network for Open Set Social Event Detection

Diminishing Empirical Risk Minimization for Unsupervised Anomaly Detection

Contact Info

Product

Resources

About