Breast cancer is a fatal disease, among which, its sub-type invasive (or infiltrating) ductal carcinomas (IDC) dominate the death cases of it. Detecting the features of such disease in an X-ray image by human eyes can be challenged, especially in cancer's early stage. Thus, this study is aimed at developing a system to assist doctors' diagnoses and help the patient to have a preliminary understanding of their own health conditions. More specifically, an IDC detection system based on the Convolutional Neural Network (CNN) is developed, where the DenseNet121 is applied here. In fact, DenseNet 169 and DenseNet 201 are also tested but their performances are not as good as DenseNet121 in this study. As is expected, the system can automatically judge whether the region in a breast histology image is IDC positive or not. This method achieves a high precision, 0.9725 validation accuracy, 0.97 test accuracy, 0.96 recall, 0.96 F1-score, and 0.965 AUC in the sub-dataset selected from Kaggle's Breast Histopathology images dataset. The time to predict 200 images is about 54 seconds and so the average prediction time for a single image is 2.7 s, which is fast enough for practical use.