The reimbursement system changed from fee-for-service to fixed prospective payments in Taiwan, the effect on the physician-patient's relationship is worth being studied. We examined the relationship between patient visit experience, cost perceptions and the two important aspects of quality of care, curing and interpersonal performance, and patients' loyalty to the hospital physicians. A total of 404 patients from an acute care hospital in Taiwan, Kaohsiung Medical University Hospital (KMUH), were investigated using a self-administered mailing survey. All measures including patient loyalty (PL), curing service quality (CSQ), interpersonal service quality (ISQ), visit experience (VE) and perceived switching costs (PSC), were adapted and modified from existing scales. Our results showed that the physician's CSQ and ISQ positively affected patients' loyalty to KMUH. The interaction between the main effects of service quality, patients' VE and three types of switching visit costs, yielded additional insights into the importance of service quality for patient retention. The CSQ of physicians becomes a more important determinant of loyalty than ISQ as patients' VE increases. The importance of CSQ and ISQ increases in relation to PL as the perceived procedural and relational costs of changing care providers increases. Neither CSQ nor ISQ has a reduced relationship with PL as the perceived financial costs of switching hospitals increase. Our study indicates that the impact of CSQ and ISQ on loyalty varies according to the perceived visit costs of changing hospitals and the patients' VE.