Abstract-Efficient and accurate image segmentation is an important task in computer vision and object recognition. Since fully automatic image segmentation is hard to handle with natural images and texture images with complex background, thus interactive scheme with a few simple user inputs is a very good addition to image segmentation. For the purpose to accurately extract objects from different images, this paper presents a color histogram and Contourlet transform based interactive image segmentation. In the initialization stage, a superpixel based initial segmentation is applied to the original image. After that, the original image will be divided into a certain number of superpixels. Then, each superpixel will be represented by a novel superpixel feature based on color histogram and Contourlet transform. Finally, by using user-defined strokes, we merge the superpixels which are similar to marked object superpixels and merge the superpixels similar to marked background superpixels, until classify all superpixels. The computational complexity is analyzed, and comparative experimental results show that the proposed scheme can reliably and rapidly extract the desired object from the complex background.