A robust and stable scene segmentation is a prerequisite for the object based coding. Various approaches to this complex task have been proposed, including segmentation of optic flow, grey-level based segmentation or simple division of the scene into moving and stationary regions.In this paper, we propose an algorithm which combines all three approaches in order to get a more robust and accurate segmentation of the moving objects. The experimental results show that the proposed algorithm can significantly reduce over-segmentation and maintain accurate motion boundaries. The use of the proposed approach in video coding can increase the PSNR and reduce the bit rate.