Transfer Learning under High-dimensional Generalized Linear Models

Abstract

In this work, we study the transfer learning problem under high-dimensional generalized linear models (GLMs), which aim to improve the fit on target data by borrowing information from useful source data. Given which sources to transfer, we propose an oracle algorithm and derive its 2 -estimation error bounds. The theoretical analysis shows that under certain conditions, when the target and source are sufficiently close to each other, the estimation error bound could be improved over that of the classical penalized estimator using only target data. When we don't know which sources to transfer, an algorithm-free transferable source detection approach is introduced to detect informative sources. The detection consistency is proved under the high-dimensional GLM transfer learning setting. Extensive simulations and a real-data experiment verify the effectiveness of our algorithms.

Targeting Underrepresented Populations in Precision Medicine: A Federated Transfer Learning Approach

“…In contrast to existing transfer learning methods based on GLM, the above procedure has benefits in estimation accuracy and flexibility to be implemented in the federated setting. Compared to a recent work (Tian and Feng, 2021), the above procedure has a faster convergence rate, which is in fact minimax optimal under mild conditions. Moreover, our method learns w (k) independently in Step 1 and Step 2, while in other related methods (Tian and Feng, 2021;Li et al, 2020a), a pooled analysis is conducted with data from multiple populations.…”

“…Compared to a recent work (Tian and Feng, 2021), the above procedure has a faster convergence rate, which is in fact minimax optimal under mild conditions. Moreover, our method learns w (k) independently in Step 1 and Step 2, while in other related methods (Tian and Feng, 2021;Li et al, 2020a), a pooled analysis is conducted with data from multiple populations. In a federated setting, finding a proper initialization is challenging for such a pooled estimator due to various levels of heterogeneity.…”

“…Li et al (2021) studies transfer learning in high-dimensional generalized linear models (GLMs) and establishes the minimax optimality. Tian and Feng (2021) studies adaptation to the unknown similarity level in transfer learning under high-dimensional GLMs. These individual data-based methods cannot be directly extended to the federated setting due to data sharing constraints and the potential heterogeneity across sites.…”

“…In a federated setting, finding a proper initialization is challenging for such a pooled estimator due to various levels of heterogeneity. In addition, compared to Tian and Feng (2021), the above approach has fewer assumptions on the level of heterogeneity for data from different populations.…”

Adaptive and Robust Multi-task Learning

“…Another active area of integrative data analysis is transfer learning, which extracts knowledge from source task(s) to help better solve a different but relevant target task. Results of transfer learning are established for linear models (Yang et al, 2020;Li et al, 2020a;Tripuraneni et al, 2021), sparse (generalized) linear models (Bastani, 2021;Lei et al, 2021;Tian and Feng, 2021) and nonparametric models (Kpotufe and Martinet, 2021;Cai and Wei, 2021;Reeve et al, 2021). The above list is far from exhaustive.…”