Natural leather is a durable, breathable, stretchable, and pliable material that comes in various styles, colors, finishes, and prices. It is an ideal raw material to manufacture luxury products such as shoes, dresses, and luggage. The leather will be categorized into different grades that are determined by visual appearance, softness, and natural defects. This grading process requires a manual visual inspection from experienced experts to ensure proper quality assurance and quality control. To facilitate the inspection process, this paper introduces an efficient automated defect classification framework that is capable to evaluate if the sample patches contain defective segments. A six-step preprocessing procedure is introduced to enhance the quality of the leather image in terms of visibility and to preserve important features representation. Then, multiple classifiers are utilized to differentiate between defective and nondefective leather patches. The proposed framework is capable to generate a classification accuracy rate of 94% from a collection of samples of 1600 pieces of calf leather patches.