Transfer Learning for High-Dimensional Linear Regression: Prediction, Estimation and Minimax Optimality

Li, Sai; Cai, T. Tony; Li, Hongzhe

doi:10.1111/rssb.12479

Cited by 67 publications

(32 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A possible strategy is to combine multiple calibrated estimators, with each trained on a candidate set of multiple auxiliary outcomes. A similar strategy has been investigated in Li et al (2020) for heterogeneous transfer learning in linear regression.…”

Section: Discussionmentioning

confidence: 99%

“…Imposing γ to be a constant can resolve the identifiability issue. For example, the calibration procedure in Li et al (2020) imposes γ = 1. However, these constraints may lead to a suboptimal estimation error as presented in Section 3.…”

Section: Calibration Stepmentioning

confidence: 99%

“…Condition e) is required to control the estimation error in estimating γ * . In Li et al (2020), they fixed γ = 1, and such a condition is hence not needed. According to the proof of Theorem 3, the estimation error of γ * is amplified by a factor of βpool − β * pool 1…”

Section: Estimation Error Of the Calibrated Estimatormentioning

confidence: 99%

“…The direct transfer learning approach implements a modified Algorithm 1, where one fixes γ = 1 and does not split the samples. This modified algorithm can be considered an extension of the oracle Trans-Lasso Algorithm (TransferDirect) in Li et al (2020). The multi-task learning approach 1 (MultiTask1) extends the algorithm proposed in Obozinski et al (2008) using a logistic loss with a grouped lasso penalty.…”

Section: Extension Of Pooled Estimation On a Separate Datasetmentioning

confidence: 99%

“…Recently, Li et al (2020) addressed a heterogeneous transfer learning problem in high-dimensional linear regression. They assumed that the contrast between the parameters in the target model and those in the auxiliary models are sufficiently sparse.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Robust and flexible learning of a high-dimensional classification rule using auxiliary outcomes

Liang¹,

Park²,

Xiang³

2020

Preprint

View full text Add to dashboard Cite

Correlated outcomes are common in many practical problems. Based on a decomposition of estimation bias into two types, within-subspace and against-subspace, we develop a robust approach to estimating the classification rule for the outcome of interest with the presence of auxiliary outcomes in high-dimensional settings. The proposed method includes a pooled estimation step using all outcomes to gain efficiency, and a subsequent calibration step using only the outcome of interest to correct both types of biases. We show that when the pooled estimator has a low estimation error and a sparse against-subspace bias, the calibrated estimator can achieve a lower estimation error than that when using only the single outcome of interest. An inference procedure for the calibrated estimator is also provided. Simulations and a real data analysis are conducted to justify the superiority of the proposed method.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Calibration Stepmentioning

confidence: 99%

Section: Estimation Error Of the Calibrated Estimatormentioning

confidence: 99%

Section: Extension Of Pooled Estimation On a Separate Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Robust and flexible learning of a high-dimensional classification rule using auxiliary outcomes

Liang¹,

Park²,

Xiang³

2020

Preprint

View full text Add to dashboard Cite

show abstract

What can we Learn from Quantum Convolutional Neural Networks?

Umeano,

Paine,

Elfving

et al. 2024

Adv Quantum Tech

View full text Add to dashboard Cite

Quantum machine learning (QML) shows promise for analyzing quantum data. A notable example is the use of quantum convolutional neural networks (QCNNs), implemented as specific types of quantum circuits, to recognize phases of matter. In this approach, ground states of many‐body Hamiltonians are prepared to form a quantum dataset and classified in a supervised manner using only a few labeled examples. However, this type of dataset and model differs fundamentally from typical QML paradigms based on feature maps and parameterized circuits. In this study, how models utilizing quantum data can be interpreted through hidden feature maps, where physical features are implicitly embedded via ground‐state feature maps is demonstrated. By analyzing selected examples as case studies for understanding QCNNs, it is shown that high performance in quantum phase recognition comes from generating a highly effective basis set with sharp features at critical points. The learning process adapts the measurement to create sharp decision boundaries. The analysis highlights improved generalization when working with quantum data, particularly in the limited‐shots regime. Furthermore, translating these insights into the domain of quantum scientific machine learning, it is demonstrated that ground‐state feature maps can be applied to fluid dynamics problems, expressing shock wave solutions with good generalization and proven trainability.

show abstract

Efficient Risk Assessment of Time‐to‐Event Targets With Adaptive Information Transfer

Ding,

Li,

Xie

et al. 2024

Statistics in Medicine

View full text Add to dashboard Cite

Using informative sources to enhance statistical analysis in target studies has become an increasingly popular research topic. However, cohorts with time‐to‐event outcomes have not received sufficient attention, and external studies often encounter issues of incomparability due to population heterogeneity and unmeasured risk factors. To improve individualized risk assessments, we propose a novel methodology that adaptively borrows information from multiple incomparable sources. By extracting aggregate statistics through transitional models applied to both the external sources and the target population, we incorporate this information efficiently using the control variate technique. This approach eliminates the need to load individual‐level records from sources directly, resulting in low computational complexity and strong privacy protection. Asymptotically, our estimators of both relative and baseline risks are more efficient than traditional results, and the power of covariate effects testing is much enhanced. We demonstrate the practical performance of our method via extensive simulations and a real case study.

show abstract

Transfer Learning for High-Dimensional Linear Regression: Prediction, Estimation and Minimax Optimality

Cited by 67 publications

References 45 publications

Robust and flexible learning of a high-dimensional classification rule using auxiliary outcomes

Robust and flexible learning of a high-dimensional classification rule using auxiliary outcomes

What can we Learn from Quantum Convolutional Neural Networks?

Efficient Risk Assessment of Time‐to‐Event Targets With Adaptive Information Transfer

Contact Info

Product

Resources

About