Most butterfly larvae are agricultural pests and forest pests, but butterflies have important ornamental value and the ability to sense and respond to changes in the ecological environment. There are many types of butterflies, and the research on classification of butterfly species is of great significance in practical work such as environmental protection and control of agricultural and forest pests. Butterfly classification is a fine-grained image classification problem that is more difficult than generic image classification. Common butterfly photos are mostly specimen photos (indoor photos) and ecological photos (outdoor photos/natural images). At present, research on butterfly classification is more based on specimen photos. Compared with specimen photos, classification based on ecological photos is relatively difficult. This paper mainly takes ecological photos as the research object, and presents a new classification network that combines the dilated residual network, squeeze-and-excitation (SE) module, and spatial attention (SA) module. The SA module can make better use of the long-range dependencies in the images, while the SE module takes advantage of global information to enhance useful information features and suppress less useful features. The results show that the integrated model achieves higher recall, precision, accuracy, and f1-score than the state-of-the-art methods on the introduced butterfly dataset.