Invasive Ductal Carcinoma (IDC) is a common form of breast cancer that can be found in women. In traditional medical practice, physicians have to manually test and classify the areas which they suspect to be cancerous. However, the literature strongly shows that that the process manual segmentation done by the medical practitioners, is neither time-efficient nor accurate as it depends on their subjective judgment. The model called Residual Attention Neural Network Breast Cancer Classification (RANN-BCC) is introduced in this paper to help medical practitioners in the cancer diagnostic process. RANN-BCC utilizes Residual Neural Network (ResNet) as an expert-supportive method to help medical practitioners in cancer diagnosis. The implementation of RANN-BCC can support the classification of Whole Slide Imaging (WSI) into non-IDC and IDC without prior information about the presence of a cancerous lesion. The result of classification shows that the RANN-BCC model achieved 92.45% accuracy, 0.98 recall, 0.91 precision, and 0.94 F-score which outperform other models such as CNN, AlexNet, Residual Neural Network 34 (ResNet34), and Feed Forward Neural Network. The developed RANN-BCC model aims to help medical experts classify IDC and non-IDC of breast cancer by learning the feature content of medical images.