Cancer prognosis and the course of treatment chosen are based on the grade of malignancy present as observed in the images. Deep learning approaches have resulted in encouraging outcomes on histopathology images at different levels, such as detection, segmentation, and classification. In this paper, we examine the application of deep learning and image processing techniques to perform automatic breast cancer malignancy grading from histopathology images of Hematoxylin and Eosin (H&E) stains. To determine the grade of malignancy slide, ROI is first extracted from the pre-processed malignant images. From this features were extracted by an intensity level and texture of images, which describe the proportions of nuclei belonging to the various grades, in conjunction with pixel-level feature, and object level feature. In this study we propose a rough set-based syntactic feature selection method (SFSM)-to create an integrated set of image attributes that can potentially differentiate the varies grades of malignancy image features. Finally, cancer grade was determined by reducing feature vector and classified by the individual Xg-boosting model. Thus, the framework can be clinically used for accurate and rapid diagnosis of breast cancer.