Temporal-aware Language Representation Learning From Crowdsourced Labels

Abstract

Learning effective language representations from crowdsourced labels is crucial for many real-world machine learning tasks. A challenging aspect of this problem is that the quality of crowdsourced labels suffer high intraand inter-observer variability. Since the highcapacity deep neural networks can easily memorize all disagreements among crowdsourced labels, directly applying existing supervised language representation learning algorithms may yield suboptimal solutions. In this paper, we propose TACMA, a temporal-aware language representation learning heuristic for crowdsourced labels with multiple annotators. The proposed approach (1) explicitly models the intra-observer variability with attention mechanism; (2) computes and aggregates per-sample confidence scores from multiple workers to address the inter-observer disagreements. The proposed heuristic is extremely easy to implement in around 5 lines of code. The proposed heuristic is evaluated on four synthetic and four real-world data sets. The results show that our approach outperforms a wide range of state-of-the-art baselines in terms of prediction accuracy and AUC. To encourage the reproducible results, we make our code publicly available at https://github.com/ CrowdsourcingMining/TACMA.

scGCL: an imputation method for scRNA-seq data based on graph contrastive learning

“…Moreover, scNAME ignores the relationships between cells. With the rise of graph contrastive learning in the field of graph representation learning ( Thakoor et al , 2021 ), it will become a new idea to use graph contrastive learning to capture the relationship between cells and recover missing gene expression values.…”