Currently, most chatbots are unable to detect the emotional state of the interlocutor and respond according to the interlocutor’s emotional state. Over the last few years, there has been growing interest in empathic chatbots. In other disciplines aside from artificial intelligence, e.g., in medicine, there is growing interest in the study and simulation of human emotions. However, there is a fundamental issue that is not commonly addressed, and it is the design of protocols for quantitatively evaluating an empathic chatbot by utilizing the analysis of the conversation between the bot and an interlocutor. This study is motivated by the aforementioned scenarios and by the lack of methods for assessing the performance of an empathic bot; thus, a chatbot with the ability to recognize the emotions of its interlocutor is needed. The main novelty of this study is the protocol with which it is possible to analyze the conversations between a chatbot and an interlocutor, regardless of whether the latter is a person or another chatbot. For this purpose, we have designed a minimally viable prototype of an empathic chatbot, named LENNA, for evaluating the usefulness of the proposed protocol. The proposed approach uses Shannon entropy to measure the changes in the emotional state experienced by the chatbot during a conversation, applying sentiment analysis techniques to the analysis of the conversation. Once the simulation experiments were performed, the conversations were analyzed by applying multivariate statistical methods and Fourier analysis. We show the usefulness of the proposed methodology for evaluating the emotional state of LENNA during conversations, which could be useful in the evaluation of other empathic chatbots.