In this paper we present a novel application work for grading of apple fruits by machine vision. Following precise segmentation of defects by minimal confusion with stem/calyx areas on multispectral images, statistical, textural and geometric features are extracted from the segmented area. Using these features, statistical and syntactical classifiers are trained for two-and multi-category grading of the fruits. Results showed that feature selection provided improved performance by retaining only the important features, and statistical classifiers outperformed their syntactical counterparts. Compared to the state-of-the-art, our two-category grading solution achieved better recognition rates (93.5% overall accuracy). In this work we further provided a more realistic multi-category grading solution, where different classification architectures are evaluated. Our observations showed that the single-classifier architecture is computationally less demanding, while the cascaded one is more accurate.