Supervised Contrastive Regression

Abstract

Deep regression models typically learn in an end-to-end fashion and do not explicitly try to learn a regression-aware representation. Their representations tend to be fragmented and fail to capture the continuous nature of regression tasks. In this paper, we propose Supervised Contrastive Regression (SupCR), a framework that learns a regression-aware representation by contrasting samples against each other based on their target distance. SupCR is orthogonal to existing regression models, and can be used in combination with such models to improve performance. Extensive experiments using five real-world regression datasets that span computer vision, human-computer interaction, and healthcare show that using SupCR achieves the state-of-the-art performance and consistently improves prior regression baselines on all datasets, tasks, and input modalities. SupCR also improves robustness to data corruptions, resilience to reduced training data, performance on transfer learning, and generalization to unseen targets.

Online triplet contrastive learning enables efficient cliff awareness in molecular activity prediction

“…Relevant research 12 has shown that end-to-end deep learning (DL) methods often neglect the optimization of the latent space, resulting in fragmented feature distributions. This issue becomes more pronounced in regression tasks and when dealing with activity cliffs (ACs).…”

Online triplet contrastive learning enables efficient cliff awareness in molecular activity prediction

“…AC refers to cases where structurally similar compounds have significant differences in activity, indicating the complexity and discontinuity of the molecular activity space. Current end-to-end DL methods neglect latent feature space optimization, resulting in fragmented feature distribution 12 . Samples with significant target differences may be close in feature space, leading to suboptimal predictions.…”