Abstract-with increase in threats and concerns in security, detection of suspicious activities in public areas has attracted an enormous level of attention. In general, video processing system are been employed for post-event analysis. However, there is a need to build an intelligent video surveillance system so as to find ways to prevent such events. The proposed system is used todetect the abandoned object from the surveillance videos with the use of dual background model. The division of video into frames is done and are pre-processed. In this approach, dual background method is used to subtract the foreground objects from the background, which generates two backgrounds called buffered background and current background. The foreground blobs are generated using subtraction of the two backgrounds and it is tracked to detect the abandoned objects. Tracking is done by maintaining a Track set which includes blobProperties, namely Area, Centroid, Major Axis length, Minor Axis Length and Convex Area, and two separate count set. The system is tested on various videoswhich are publically available.