An Anomaly can be defined as an observation that does follow the normal activities with respect to others. Anomaly is polysemy that varies in a different context. For video sequences, an anomaly can be defined in terms of motion or state of motion that obtrude concerning to state and time. This work is based on Pedestrian Video Surveillance, having a normal event as a pedestrian and abnormal event as cars, skaters, bikers, wheelchairs, etc. i.e., non-pedestrian. A comprehensive study has been carried out of many existing systems for the Anomaly detection system. Based on many existing techniques we select a method called optical flow for the feature presentation of the detection of motion and Histogram of optical flow (HOOF) for the action contour of optical flow for every time instant irrespective of scale and direction of motion. As HOOF represents the motion of an object in a region, so there may be cases that the information related to the motion is not well represented. We used these two techniques along with the labelling of the frame inorderto overcome the problem of HOOF for the classifier. And gives the improved results.