Key information for the maintenance and diagnosis of structures including bridges can be obtained from the processing of digital images acquired by unmanned aerial vehicle (UAV). However, lowquality images caused by various problems such as UAV movement, inspection environment, and camera parameters can lead to inappropriate structural evaluation due to the difficulty of digital image processing. Therefore, an appropriate assessment method for image quality considering the deterioration of the inspection image in the structural inspection procedure is required. In this study, a new image quality assessment (IQA) using a convolutional neural network (CNN) is proposed in consideration of various degradation factors that may occur in the structure inspection image. The first stage presents a method to obtain consistent quality against various interference factors of deterioration that may occur in inspection images. Adjusting the camera parameters minimizes the degradation of the inspection image. Subsequently, low-and high-quality images are distinguished according to the proposed image acquisition method. The second stage is the classification of the inspection dataset using the CNN-based image quality classifier model through training of data classified according to their quality. Experimental validation of the proposed method shows that the results are similar to the Human Visual System (HVS), which means subjective quality classification, and that the inspection image can be classified with more accurate and shorter processing time.