With the explosive development of the computer vision technology, more and more vision-based inspection methods enabled by unmanned aerial vehicle technologies have been researched on the crack inspection of the sundry concrete structures. However, because of the limitation of the low-cost unmanned aerial vehicle hardware, whose cost is around US$500, most of the vision-based methods are difficult to be implemented on the low-cost unmanned aerial vehicle for real-time crack inspection. To address this challenge, in this article, a new computationally efficient vision-based crack inspection method is designed and successfully implemented on a low-cost unmanned aerial vehicle. Furthermore, to reduce the acquired data samples, a new algorithm entitled crack central point method is designed to extract the effective information from the pre-processed images. The proposed vision-based crack detection method includes the following three major components: (1) the image pre-processing algorithm, (2) crack central point method, and (3) the support vector machine model–based classifier. To demonstrate the effectiveness of the new inspection method, a concrete structure inspection experiment is implemented. The experimental results indicate that this new method is able to accurately and rapidly inspect the cracks of concrete structure in real time. This new vision-based crack inspection method shows great promise for the practical application.