Agro export industries generate a substantial amount of revenue to Indian economy. In the fruit industry, various fruits like banana, mango, apple and pomegranate, etc. are transported in the conveyor for a post harvest process like classification, sorting, grading and juice extraction. The manual discrimination of various fruits consumes time and, it can be automated. This research work is intended to build an image processing algorithm that ensures automatic discrimination of banana (Musa Species.) from other fruits like Citrus, Apple, and Pomegranate. The input object is segmented using Background subtraction and threshold method. Morphological operations are performed to obtain the clear contour of the segmented objects. The shape of the banana and non-banana are described by scale and translation invariant signature. Binary SVM with signature feature vectors detect the banana fruit from the non-banana fruit automatically. The accuracy rate is 95%.