A generalized graph reduction framework for interactive segmentation of large images

Abstract

The speed of graph-based segmentation approaches, such as random walker (RW) and graph cut (GC), depends strongly on image size. For high-resolution images, the time required to compute a segmentation based on user input renders interaction tedious. We propose a novel method, using an approximate contour sketched by the user, to reduce the graph before passing it on to a segmentation algorithm such as RW or GC. This enables a significantly faster feedback loop. The user first draws a rough contour of the object to segment. Then, the pixels of the image are partitioned into "layers" (corresponding to different scales) based on their distance from the contour. The thickness of these layers increases with distance to the contour according to a Fibonacci sequence. An initial segmentation result is rapidly obtained after automatically generating foreground and background labels according to a specifically selected layer; all vertices beyond this layer are eliminated, restricting the segmentation to regions near the drawn contour. Further foreground / background labels can then be added by the user to refine the segmentation. All iterations of the graphbased segmentation benefit from a reduced input graph, while maintaining full resolution near the object boundary. A user study with 16 participants was carried out for RW segmentation of a multi-modal dataset of 22 medical images, using either a standard mouse or a stylus pen to draw the contour. Results reveal that our approach significantly reduces the overall segmentation time compared with the status quo approach (p < 0.01). The study also shows that our approach works well with both input devices. Compared to super-pixel graph reduction, our approach provides full resolution accuracy at similar speed on a high-resolution benchmark image with both RW and GC segmentation methods. However, graph reduction based on super-pixels does not allow interactive correction of clustering errors. Finally, our approach can be combined with super-pixel clustering methods for further graph reduction, resulting in even faster segmentation. are weighted as a function of their likelihood of crossing an object boundary. For interactive segmentation to be practical, the computation of the edge weights and segmentation must be fast, enabling a tight feedback loop. However, in graph-based segmentation, computation time increases with graph size, often precluding interactive segmentation of high-resolution images. The total time to perform a segmentation also depends on human factors, such as the input device used and the kind of input required. These challenges are addressed in this paper. We present a graph reduction approach that is guided by a rough drawing of the object boundary provided by the user. Our preliminary work [14] used automatically simulated input drawings to show that this approach speeds up computations for random walker segmentation. This paper further investigates the approach and extends our analysis to make the following additional contributions:• A controlled exper...

Latency Management in Scribble-Based Interactive Segmentation of Medical Images

“…this study was 100 ms. Therefore, the segmentation method must satisfy this threshold, i.e., provide updates under 100 ms. To achieve this, we developed the FastDRaW 2 method [10], a fast adaptation of the random walker segmentation algorithm [9]. FastDRaW exploits a multi-scale framework combined with a dynamic region of interest (ROI) search to reduce the computation time.…”

“…Previous evaluations of interactive segmentation have focused on algorithmic runtime, with few studies of human factors. One preliminary study [10] found that improvements in computational performance did not yield a commensurate improvement in overall segmentation performance, i.e, the total time including user actions. Although computation time was improved by a factor ∼ 10 (from 1.23 s to 0.13 s), overall segmentation time was only improved by a factor ∼ 2 (from 25.25 s to 17.08 s).…”

“…Depending on the application, a given UIM can be more suitable than another. Moreover, different UIMs can be combined into a single segmentation framework [11,36,40]. Nevertheless, we can expect that approaches sharing similar UIMs lead to consistent and comparable reactions on part of the user.…”

Superpixel Sizes using Topology Preserved Regular Superpixel Algorithm and their Impacts on Interactive Segmentation

“…However, different interactive segmentation algorithms employ different superpixel algorithms of varying sizes. For instance, [6] and [7] used the SLIC superpixel algorithm [8] with 1000 and 2000 superpixels superpixel per image, respectively. In addition, [9] used a meant-shift superpixel algorithm with 100 superpixels per image.…”

Interactive segmentation: a scalable superpixel-based method

“…The two most common application scopes for these kinds of algorithms are segmentation of a particular anatomic structure in medical imagery [5][6][7][8][9][10][11][12][13][14] and natural photograph (such as an image taken with a smartphone or a camera) manipulation for art or graphic design purpose. [15][16][17][18][19][20][21][22][23][24][25][26] These two application scopes correspond to distinct problems, with substantial differences among proceeded image features and the available prior information.…”