Ransomware is a serious security concern to mobile devices, as it prevents the use of the device and its contents until a ransom is paid, resulting in considerable financial losses for both people and corporations. The existing anti‐malware measures have shown to be inadequate in combatting new malware variants that utilize advanced evasion strategies like Polymorphic, Metamorphic, Dynamic Code Loading, Time‐based evasion, and Reflection. Furthermore, these primary defences have also suffered from low detection rates, significant false positives, high processing times, and excessive processing and power consumption that is inappropriate for smartphones. This paper offers the binary JAYA (BJAYA) for ransomware detection in Android mobile devices using the BJAYA optimization‐based algorithm. The developed algorithm's effectiveness has been assessed against two datasets, the 0–1 knapsack, and real ransomware dataset. The proposed BJAYA method surpassed the other algorithms on 85% of the 0–1 knapsack datasets. The suggested BJAYA method was also tested on a ransomware dataset in two phases. In the first stage of testing, BJAYA outperformed other standard classifiers with sensitivity and Gmean values of 97% and 98.2%, respectively. In the second stage of testing, BJAYA outperformed other GA, FPA, and PSO metaheuristic algorithms in terms of specificity, sensitivity, and Gmean. These findings indicate the applicability of the suggested BJAYA algorithm for ransomware detection.