In spite of their ubiquity, our interactions with contemporary conversational agents (CA), such as Alexa, are still transactional in nature and lack the expressiveness of human-human communication. Conversational agents equipped with the ability to detect and address users' emotional and cognitive states could make our interactions with them more human. In this work, we investigate whether an empathetic CA can help mitigate boredom. We design a protocol in order to first elicit boredom in users, and explore strategies that attempt to mitigate their boredom with the help of two conversational agents, an empathetic agent and a non-empathetic agent, administered in a Wizard-of-Oz setting. We quantify their efficacy by measuring the effects on user mood and task performance. Our user study with 34 participants shows that the empathetic CA outperforms the non-empathetic CA with respect to modulating users' mood and performance. CCS CONCEPTS • Human-centered computing → User studies; Natural language interfaces.