Cancer is one of the leading causes of death in many countries. Breast cancer is one of the most common cancers in women. Especially in remote areas with low medical standards, the diagnosis efficiency of breast cancer is extremely low due to insufficient medical facilities and doctors. Therefore, in-depth research on how to improve the diagnosis rate of breast cancer has become a hot spot. With the development of society and science, people use artificial intelligence to improve the auxiliary diagnosis of diseases in the existing medical system, which can become a solution for detecting and accurately diagnosing breast cancer. The paper proposes an auxiliary diagnosis model that uses deep learning in view of the low rate of human diagnosis by doctors in remote areas. The model uses classic convolutional neural networks, including VGG16, InceptionV3, and ResNet50 to extract breast cancer image features, then merge these features, and finally train the model VIRNets for auxiliary diagnosis. Experimental results prove that for the recognition of benign and malignant breast cancer pathological images under different magnifications, VIRNets have a high generalization and strong robustness, and their accuracy is better than their basic network and other structures of the network. Therefore, the solution provides a certain practical value for assisting doctors in the diagnosis of breast cancer in real scenes.