Conventional approaches to breast cancer diagnosis are associated with drawbacks that ultimately affect the quality of diagnosis and subsequent treatment, pushing for the need for automatic and precise classification of breast cancer tumors. The advent of deep learning methods has witnessed an increasing interest in their applications in many tasks. The specific case of using convolutional neural networks with transfer learning has witnessed tremendous successes in many classification tasks. Nonetheless, with transfer learning, the sheer number of parameters associated with deep networks coupled with the distance disparity between source data and target data leave networks prone to overfitting, particularly in the case of limited data. Also, negative transfer may occur in the situation where the source and target domains are not related. This work proposes a simple convolutional neural network model trained from scratch for discriminating benign and malignant breast cancer tumors in histopathological images. Four deep learning optimization algorithms are leveraged and explored to ascertain how optimizers aid in finding good sets of parameters that help minimize loss and increase overall classification accuracy. By adopting a polynomial learning rate decay scheduling and implementing several data augmentation techniques that regulate overfitting and improve the generalization ability of the proposed model, accuracy, sensitivity, specificity, and Area Under the Curve values of 89.92%, 94.02%, 86.42%, and 0.884 (88.4%), respectively are reported.