Background Telemedicine services worldwide have experienced an unprecedented boom since the beginning of the COVID-19 pandemic. Multiple studies have noted telemedicine as an effective alternative to traditional face-to-face management of patients. This study provides insight into public perception and impression of telemedicine in Hong Kong, specifically among the elderly who are the most vulnerable to COVID-19. Methods Face-to-face surveys were conducted on elderly relatives of current medical students at the Chinese University of Hong Kong who were aged ≥ 60 years. The survey included socio-demographic details; past medical history; and concerns towards telemedicine use. Univariate and multivariable regression analyses were conducted to examine statistically significant associations. The primary outcomes are consideration of telemedicine use during: (1) a severe outbreak; and (2) after the COVID-19 pandemic. Results 109 surveys were conducted. Multivariable regression analyses revealed that expectation of government subsidies for telemedicine services was the strongest common driver, and also the only positive independent predictors of telemedicine use for both during a severe outbreak, as well as after the COVID-19 pandemic. No negative independent predictors of telemedicine use during severe outbreak were found. Negative independent predictors of telemedicine use after the COVID-19 pandemic included old age, and living in the New Territories. Conclusions Government support such as telemedicine-specific subsidies will be crucial in promoting telemedicine use in Hong Kong both during a severe outbreak and after the current COVID-19 pandemic. Robust dissemination of information regarding the pros and cons of telemedicine towards the public, especially towards the elderly population, is warranted.