The emergence of acquired drug resistance in cancer represents a major barrier to treatment success. While research has traditionally focused on genetic sources of resistance, recent findings suggest that cancer cells can acquire transient resistant phenotypes via epigenetic modifications and other non-genetic mechanisms. Although these resistant phenotypes are eventually relinquished by individual cells, they can temporarily 'save' the tumor from extinction and enable the emergence of more permanent resistance mechanisms. These observations have generated interest in the potential of epigenetic therapies for long-term tumor control or eradication. In this work, we develop a mathematical model to study how phenotypic switching at the single-cell level affects resistance evolution in cancer. We highlight unique features of non-genetic resistance, probe the evolutionary consequences of epigenetic drugs and explore potential therapeutic strategies. We find that even short-term epigenetic modifications and stochastic fluctuations in gene expression can drive long-term drug resistance in the absence of any bona fide resistance mechanisms. We also find that an epigenetic drug that slightly perturbs the average retention of the resistant phenotype can turn guaranteed treatment failure into guaranteed success. Lastly, we find that combining an epigenetic drug with an anti-cancer agent can significantly outperform monotherapy, and that treatment outcome is heavily affected by drug sequencing.