Artificial Intelligence (AI) has rapidly evolved and is increasingly impacting various aspects of life. This rapiddevelopment and deployment of AI technologies have raised significant ethical concerns. Recently, an increasingamount of research has been conducted with the aim to guarantee that AI technologies are developed and employedin a responsible, fair, and an ethical manner, Considering the possible risks and challenges they pose. However, tothe best of our knowledge, no thorough overview and categorization of the different kinds of research achieved onthis subject has been provided yet, even with the growing interest in the subject. To try and bridge this gap, theauthors conducted a systematic mapping of the literature to gain a broad understanding about this subject over thepast seven years (2017-2023). This process resulted in the determination of 127 primary studies that satisfied thecriteria for including and excluding papers that are relevant to the context of our study. Subsequently, these studieswere classified in light of their research type, contribution type, and research focus. The findings revealed that mostof the studies examined in this research were centered around defining AI ethics and its impact on humanity fromdifferent angles such as fairness, transparency and accountability.The findings also show that research is lacking inareas such as empirical evidences on AI ethics and principles, solutions to enforce an ethical AI development andevaluation of ethical AI solutions. This study aims to assist practitioners and domain experts in identifying areas ofcurrent research gaps and prospective directions for future investigations.