Disclaimer: At the moment this article is currently awaiting submission review. Since this process usually takes a lot of time, feel free to use it this version as a reference until it is published. Thank you for reading, if you have any feedback please send us a message. Enjoy. The deployment of Socially Intelligent Agents (SIAs) in learning environments has proven to have several advantages in different areas of application. Social Agent Authoring Tools allow scenario designers to create tailored experiences with high control over SIAs behaviour, however, on the flip side, this comes at a cost as the complexity of the scenarios and its authoring can become overbearing. In this paper we introduce the concept of Explainable Social Agent Authoring Tools with the goal of analysing if authoring tools for social agents are understandable and interpretable. To this end we examine whether an authoring tool, FAtiMA-Toolkit, is understandable and its authoring steps interpretable, from the point-of-view of the author. We conducted two user studies to quantitatively assess the Interpretability, Comprehensibility and Transparency of FAtiMA-toolkit from the perspective of a scenario designer. One of the key findings is the fact that FAtiMA-Toolkit's conceptual model is, in general, understandable, however the emotional-based concepts were not as easily understood and used by the authors. Although there are some positive aspects regarding the explainability of FAtiMA-Toolkit, there is still progress to be made to achieve a fully explainable social agent authoring tool. We provide a set of key concepts and possible solutions that can guide developers to build such tools.SectionionIntroduction Socially Intelligent Agents (SIAs) have an ever increasing range of applications from conversational interfaces on websites to tutors or teammates in educational environments [1,2], where they are equipped with tools to conduct human-like interactions. Amongst the most promising applications of SIAs are serious games and social skills training environments. In these virtual environments SIAs behaviours can range from reactive wandering in the background of a scenario to complex social interactions that provide social support or assist the player in some skill training [3]. These autonomous agents sense the environment and act intelligently and independently from the user, allowing them to train and adapt specific verbal and nonverbal behaviors in socially challenging situations [4].