We propose an unsupervised video segmentation approach by simultaneously tracking multiple holistic figureground segments. Segment tracks are initialized from a pool of segment proposals generated from a figure-ground segmentation algorithm. Then, online non-local appearance models are trained incrementally for each track using a multi-output regularized least squares formulation. By using the same set of training examples for all segment tracks, a computational trick allows us to track hundreds of segment tracks efficiently, as well as perform optimal online updates in closed-form. Besides, a new composite statistical inference approach is proposed for refining the obtained segment tracks, which breaks down the initial segment proposals and recombines for better ones by utilizing highorder statistic estimates from the appearance model and enforcing temporal consistency. For evaluating the algorithm, a dataset, SegTrack v2, is collected with about 1,000 frames with pixel-level annotations. The proposed framework outperforms state-of-the-art approaches in the dataset, showing its efficiency and robustness to challenges in different video sequences.

The bulk alignment of actin filament sliding movement, powered by randomly oriented myosin molecules, has been observed and studied using an in vitro motility assay. The well established, actin filament gliding assay is a minimal experimental system for studying actomyosin motility. Here, we show that when the assay is performed at densities of actin filaments approaching those found in living cells, filament gliding takes up a preferred orientation. The oriented patterns of movement that we have observed extend over a length scale of 10–100 μm, similar to the size of a mammalian cell. We studied the process of filament alignment and found that it depends critically upon filament length and density. We developed a simple quantitative measure of filament sliding orientation and this enabled us to follow the time course of alignment and the formation and disappearance of oriented domains. Domains of oriented filaments formed spontaneously and were separated by distinct boundaries. The pattern of the domain structures changed on the time scale of several seconds and the collision of neighboring domains led to emergence of new patterns. Our results indicate that actin filament crowding may play an important role in structuring the leading edge of migrating cells. Filament alignment due to near-neighbor mechanical interactions can propagate over a length scale of several microns; much greater than the size of individual filaments and analogous to a log drive. Self-alignment of actin filaments may make an important contribution to cell polarity and provide a mechanism by which cell migration direction responds to chemical cues.

Popular figure-ground segmentation algorithms generate a pool of boundary-aligned segment proposals that can be used in subsequent object recognition engines. These algorithms can recover most image objects with high accuracy, but are usually computationally intensive since many graph cuts are computed with different enumerations of segment seeds. In this paper we propose an algorithm, RIGOR, for efficiently generating a pool of overlapping segment proposals in images. By precomputing a graph which can be used for parametric min-cuts over different seeds, we speed up the generation of the segment pool. In addition, we have made design choices that avoid extensive computations without losing performance. In particular, we demonstrate that the segmentation performance of our algorithm is slightly better than the state-of-the-art on the PASCAL VOC dataset, while being an order of magnitude faster.

We present a supervised learning-based method to estimate a per-pixel confidence for optical flow vectors. Regions of low texture and pixels close to occlusion boundaries are known to be difficult for optical flow algorithms. Using a spatiotemporal feature vector, we estimate if a flow algorithm is likely to fail in a given region. Our method is not restricted to any specific class of flow algorithm and does not make any scene specific assumptions. By automatically learning this confidence, we can combine the output of several computed flow fields from different algorithms to select the best performing algorithm per pixel. Our optical flow confidence measure allows one to achieve better overall results by discarding the most troublesome pixels. We illustrate the effectiveness of our method on four different optical flow algorithms over a variety of real and synthetic sequences. For algorithm selection, we achieve the top overall results on a large test set, and at times even surpass the results of the best algorithm among the candidates.

For two consecutive frames in a video, we identify which pixels in the first frame become occluded in the second. Such general-purpose detection of occlusion regions is difficult and important because one-to-one correspondence of imaged scene points is needed for many tracking, video segmentation, and reconstruction algorithms. Our hypothesis is that an effective trained occlusion detector can be generated on the basis of i) a broad spectrum of visual features, and ii) representative but synthetic training sequences. By using a Random Forest based framework for feature selection and training, we found that the proposed feature set was sufficient to frequently assign a high probability of occlusion to just the pixels that were indeed becoming occluded. Our extensive experiments on many sequences support this finding, and while accuracy is certainly still scenedependent, the proposed classifier could be a useful preprocessing step to exploit temporal information in video.