The image processing and computer vision systems have been widely used for identification, classification, grading andquality evaluation in the agriculture area. Defect identification and maturity detection of mango fruits are challenging task for thecomputer vision to achieve near human levels of recognition. The proposed framework is useful in the supermarkets and can beutilized in computer vision for the automatic sorting of fruits from a set, consisting of different kind of fruits. The objective of thiswork is to develop an automated tool, which can be capable of identifying defect and detect maturity of mango fruits based on shape,size and color features by digital image analysis. MATLAB have been used as the programming tool for identification andclassification of fruits using Image Processing toolbox. Proposed method can be used to detect the visible defects, stems, size andshape of mangos, and to grade the mango in high speed and precision.