Clinical decision support systems (CDSSs) can effectively detect illnesses such as breast cancer (BC) using a variety of medical imaging techniques. BC is a key factor contributing to the rise in the death rate among women worldwide. Early detection will lessen its impact, which may motivate patients to have quick surgical therapy. Computer-aided diagnosis (CAD) systems are designed to provide radiologists recommendations to assist them in diagnosing BC. However, it is still restricted and limited, the interpretability cost, time consumption, and complexity of architecture are not considered. These limitations limit their use in healthcare devices. Therefore, we thought of presenting a revolutionary deep learning (DL) architecture based on recurrent and convolutional neural networks called Bi-xBcNet-96. In order to decrease carbon emissions while developing the DL model for medical image analysis and meet the objectives of sustainable artificial intelligence, this study seeks to attain high accuracy at the lowest computing cost. It takes into consideration the various characteristics of the pathological variation of BC disease in mammography images to obtain high detection accuracy. It consists of six stages: identifying the region of interest, detecting spatial features, discovering the effective features of the BC pathological types that have infected nearby cells in a concentrated area, identifying the relationships between distantly infected cells in some BC pathological types, weighing the extracted features, and classifying the mammography image. According to experimental findings, Bi-xBcNet-96 beat other comparable works on the benchmark datasets, attaining a classification accuracy of 98.88% in DDSM dataset, 100% in INbreast dataset with 5.08% and 0.3% improvements over the state-of-the-art methods, respectively. Furthermore, a 95.79% reduction in computing complexity was achieved.