Object recognition and classification has become important in a surveillance video situated at prominent areas such as airports, banks, military installations, etc. Outdoor environments are more challenging for moving object classification because of incomplete appearance details of moving objects due to illumination changes and large distance between the camera and moving objects. As such, there is a need to monitor and classify the moving objects by considering the challenges of video in the real time. Training the classifiers using feature-based approaches is easier and faster than pixel-based approaches in object classification. Extraction of a set of features from the object of interest is most important for classification. Viewpoint and sources of light illumination plays major role in the appearance of an object. Abrupt transitions are identified using Chi-square and corners are detected using Harris corner detection. Silhouettes are captured using background subtraction and feature extraction is done using ORB. k-NN classifier is used for classification.