Recent breakthroughs in artificial intelligence (AI) language models have elevated the vision of using AI support for mental health, with a growing body of literature indicating varying degrees of efficacy. In this paper we ask when in therapy will it be easier to replace humans, and conversely, in what instances will human connection still be more valued. We suggest that empathy lies at the heart of the answer to this question. First, we define different aspects of empathy and outline potential empathic capabilities of humans versus AI. Next, we consider what determines when these aspects are needed most in therapy, both from the perspective of therapeutic approach, and from the perspective of patient objectives. Ultimately, our goal is to prompt further investigation and dialogue, urging both practitioners and scholars engaged in AI-mediated therapy to keep these questions and considerations in mind when investigating AI implementation in mental health.