Classification of variety of fruits is an important framework in agriculture field for import and export. Many algorithms were employed for classification of fruits. To remove the noises from the images Gaussian filter is applied during the pre-processing. Bunch of fruits are classified into different classes such as apple, orange and banana. Also their quality is taken into account for preventing health hazards. In this work, the various combinations of fruits are classified into proper variety and after that the quality of the fruit is checked as whether it is defect or non-defect. For the first part, Convolutional Neural Network, AlexNet and MobileNetV2 are employed. MobileNetV2 achieved 100% accuracy for fruit type classification. In the second part, the same kind of fruits are fed into classifier for quality checking. The above said classifiers are used for defect classification also. For defect classification, MobileNetV2 gives 99.89% accuracy for orange and 100% for apple.