With the deployment of the 5G cellular system, the upsurge of diverse mobile applications and devices has increased the potential challenges and threats posed to users. Industry and academia have attempted to address cyber security challenges by implementing automated malware detection and machine learning algorithms. This study expands on previous research on machine learning-based mobile malware detection. We critically evaluate 154 selected articles and highlight their strengths and weaknesses as well as potential improvements. We explore the mobile malware detection techniques used in recent studies based on attack intentions, such as server, network, client software, client hardware, and user. In contrast to other SLR studies, our study classified the means of attack as supervised and unsupervised learning. Therefore, this article aims at providing researchers with in-depth knowledge in the field and identifying potential future research and a framework for a thorough evaluation. Furthermore, we review and summarize security challenges related to cybersecurity that can lead to more effective and practical research.