Prior to the COVID-19 pandemic, studies in the US have identified wide variations in telehealth use across medical specialties. This is an intriguing problem, because the US has historically lacked a standardized set of telehealth coverage and reimbursement policies, which has posed a barrier to telehealth use across all specialties. Although all medical specialties in the US have been affected by these macro (policy-level) barriers, some specialties have been able to integrate telehealth use into mainstream practice, while others are just gaining momentum with telehealth during COVID-19. Although the temporary removal of policy (coverage) restrictions during the pandemic has accelerated telehealth use, uncertainties remain regarding future telehealth sustainability. Since macro (policy-level) factors by themselves do not serve to explain the variation in telehealth use across specialties, it would be important to examine meso (organizational-level) and micro (individual-level) factors historically influencing telehealth use across specialties, to understand underlying reasons for variation and identify implications for widespread sustainability. This paper draws upon the existing literature to develop a conceptual framework on macro-meso-micro factors influencing telehealth use within a medical specialty. The framework is then used to guide a narrative review of the telehealth literature across six medical specialties, including three specialties with lower telehealth use (allergy-immunology, family medicine, gastroenterology) and three with higher telehealth use (psychiatry, cardiology, radiology) in the US, in order to synthesize themes and gain insights into barriers and facilitators to telehealth use. In doing so, this review addresses a gap in the literature and provides a foundation for future research. Importantly, it helps to identify implications for ensuring widespread sustainability of telehealth use in the post-pandemic future.