The use of image processing for the detection and classification of defects has been a reality for some time in science and industry. New methods are continually being presented to improve every aspect of this process. However, these new approaches are applied to a small, private collection of images, which makes a real comparative study of these methods very difficult. The objective of this paper was to compile a public annotated benchmark, that is, an extensive set of images with and without defects, and make these public, to enable the direct comparison of detection and classification methods. Moreover, different methods are reviewed and one of these is applied to the set of images; the results of which are also presented in this paper.