DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices

Abstract

Unsupervised Domain Adaptation for Regression (UDAR) aims to adapt a model from a labeled source domain to an unlabeled target domain for regression tasks. Recent successful works in UDAR mostly focus on subspace alignment, involving the alignment of a selected subspace within the entire feature space. This contrasts with the feature alignment methods used for classification, which aim at aligning the entire feature space and have proven effective but are less so in regression settings. Specifically, while classification aims to identify separate clusters across the entire embedding dimension, regression induces less structure in the data representation, necessitating additional guidance for efficient alignment. In this paper, we propose an effective method for UDAR by incorporating guidance from uncertainty. Our approach serves a dual purpose: providing a measure of confidence in predictions and acting as a regularization of the embedding space. Specifically, we leverage the Deep Evidential Learning framework, which outputs both predictions and uncertainties for each input sample. We propose aligning the parameters of higher-order evidential distributions between the source and target domains using traditional alignment methods at the feature or posterior level. Additionally, we propose to augment the feature space representation by mixing source samples with pseudo-labeled target samples based on label similarity. This cross-domain mixing strategy produces more realistic samples than random mixing and introduces higher uncertainty, facilitating further alignment. We demonstrate the effectiveness of our approach on four benchmarks for UDAR, on which we outperform existing methods.

Transport-based transfer learning on Electronic Health Records: Application to detection of treatment disparities

“…To assess the empirical performance of OTTEHR , we benchmarked OTTEHR against the standard statistical methods Transfer Component Analysis (TCA) [Pan et al, 2010], Correlation Analysis (CA) [Sun et al, 2017], Geodesic Flow Kernel (GFK) [Gong et al, 2012], machine learning OT-based method deepJDOT [Damodaran et al, 2018], and machine learning non-OT-based methods Representation Subspace Distance (RSD) [Chen et al, 2021] and inverse Gram matrices (daregram) [Nejjar et al, 2023] (also see Section 2) on the predictions for target groups for the transfer learning tasks detailed in Section 4.2 using mean absolute error ( MAE ) and root mean square error ( RMSE ) [Chai and Draxler, 2014].…”

“…Adversarial learning methods minimize the distribution discrepancy by optimizing a selected function over the hypothesis space, concurrently learning feature representations to bridge the gap between domains [Ganin and Lempitsky, 2015, Tzeng et al, 2015, Ganin et al, 2016, Luo et al, 2017, Long et al, 2018, Zhang et al, 2019, Peng et al, 2019]. For regression tasks, most recent TL methods using Representation Subspace Distance ( RSD ) [Chen et al, 2021] and inverse GRAM matriecs ( daregram ) [Nejjar et al, 2023] learn a shared feature extractor by minimizing some discrepancies of source and target features.…”