No abstract

No abstract

We introduce a novel spatio-temporal deformable part model for offline detection of fine-grained interactions in video. One novelty of the model is that part detectors model the interacting individuals in a single graph that can contain different combinations of feature descriptors. This allows us to use both body pose and movement to model the coordination between two people in space and time. We evaluate the performance of our approach on novel and existing interaction datasets. When testing only on the target class, we achieve mean average precision scores of 0.82. When presented with distractor classes, the additional modelling of the motion of specific body parts significantly reduces the number of confusions. Cross-dataset tests demonstrate that our trained models generalize well to other settings.

The most popular optical flow algorithms rely on optimizing the energy function that integrates a data term and a smoothness term. In contrast to this traditional framework, we derive a new objective function that couples optical flow estimation and image restoration. Our method is inspired by the recent successes of edge-aware constraints (EAC) in preserving edges in general gradient domain image filtering. By incorporating an EAC image fidelity term (IFT) in the conventional variational model, the new energy function can simultaneously estimate optical flow and restore images with preserved edges, in a bidirectional manner. For the energy minimization, we rewrite the EAC into gradient form and optimize the IFT with Euler-Lagrange Equations. We can thus apply the image restoration by analytically solving a system of linear equations. Our EAC-combined IFT is easy to implement and can be seamlessly integrated into various optical flow functions suggested in literature. Extensive experiments on public optical flow benchmarks demonstrate that our method outperforms the current state-of-the-art in optical flow estimation and image restoration.Keywords: optical flow, image sequence restoration, edge preserving, efficient numerical solver. 15 varies smoothly over the flow field. In practice, however, these two constraints are often violated, which prevents the variational method to robustly treat image noise, handle illumination changes and large displacements, and to preserve flow discontinuities.Various extensions and improvements have been proposed to handle these 20 problems [10]. In general, these variations can be classified into three categories [12]: pre-processing of the input images, modification of the variational formulation and post-processing of the flow field.Pre-processing is often applied to remove image artifacts of the input images, including noise filtering [13, 14] and deblurring [15]. 25Modifications are the second category of variations of the traditional formulation. They are aimed at improving the data term to make the algorithm more 2 resistant to noise [16,17], more robust under illumination changes [18,19, 20], and more capable to deal with large displacements [10, 1,21]. In addition, refining the smoothness term to preserve motion discontinuity [22]. 30 One challenge for optical flow computation is to deal with large displacements. Recently, there has been an interest in using feature matching to assist the variational methods to address this problem. By matching image features, reliable correspondence information between two images can be obtained [21].The feature matching and variational methods are complementary, but com-35 bining them is not straightforward. Feature matching exploits a discrete space of admissible correspondences, whereas the variational model imposes linearization during the differential optimization procedure [23]. Brox et al. [10] adds a feature matching constraint into the variational framework. In this way, correspondences from descriptor matching are obtained, wh...

Median filtering the intermediate flow fields during optimization has been demonstrated to be very useful for improving the estimation accuracy. By formulating the median filtering heuristic as non-local term in the objective function, and modifying the new term to include flow and image information that according to spatial distance, color similarity as well as the occlusion state, a weighted non-local term (a practical weighted median filter) reduces errors that are produced by median filtering and better preserves motion details. However, the color similarity measure, which is the most powerful cue, can be easily perturbed by noisy pixels. To increase robustness of the weighted median filter to noise, we introduce the idea of non-local patch denoising method to compute the color similarity in terms of patch difference. Most importantly, we propose an improved color patch similarity measure (ICPSM) to modify the traditional patch manner based measure from three aspects. Comparative experimental results on different optical flow benchmarks show that our method can denoise the flow field more effectively and outperforms the state-of-the art methods, especially for heavy noise sequences.