In this paper we address the basic limitation of Siammask -the state of the art single object tracking and segmentation algorithm. SiamMask requires semi-supervision in that it needs a bounding box to be drawn manually around the object that has to be tracked. This is however not always possible or feasible, and slows down the pipeline even in the best case. We overcome this limitation by using state-of-the-art object detection algorithms: Detectron and YOLO to automatically detect the object and then track using Siammask. We note that YOLO gives better and more meaningful detection of objects in the scene. However, Detectron gives a higher detection speed than YOLO, making the overall detection and tracking process faster.