The COVID-19 pandemic has caused a dramatic shift to telehealth in the U.S. This rapid expansion has educated both patients and providers to the convenience and capabilities of virtual care and, as a result, it has become deeply embedded into the fabric of mainstream care delivery. Patient safety and overall, quality of care is a top a priority for healthcare delivery systems. However, addressing this aim is not yet commensurate with the rapid growth of telehealth. In this study, we investigate the impact of telehealth on patient safety and quality of care, as highlighted in the COVID-19 outbreak while it will continue beyond this global pandemic. We identify barriers of achieving high quality of care in telehealth implementation through systematic investigation and the development of a Fishbone Diagram. There are six main categories of identified barriers: human, healthcare provider; human, patient; organization; technology; rules and regulations; and society. We also provide recommendations to address some of those identified barriers to improve quality of care for patients receiving care through telehealth.