Detection of Independently Moving Objects in Non-planar Scenes via Multi-Frame Monocular Epipolar Constraint

Abstract

Abstract. In this paper we present a novel approach for detection of independently moving foreground objects in non-planar scenes captured by a moving camera. We avoid the traditional assumptions that the stationary background of the scene is planar, or that it can be approximated by dominant single or multiple planes, or that the camera used to capture the video is orthographic. Instead we utilize a multiframe monocular epipolar constraint of camera motion derived for monocular moving cameras defined by an evolving epipolar plane between the moving camera center and 3D scene points. This constraint is parameterized as a polynomial function of time, and unlike repeated computations of inter-frame fundamental matrix, requires the estimation of fewer unknowns, and provides a more consistent separation between moving and static objects for different levels of noise. This constraint allows us to segment out moving objects in a general 3D scene where other approaches fail because their initial assumptions do not hold, and provides a natural way of fusing temporal information across multiple frames. We use a combination of optical flow and particle advection to capture all motion in the video across a number of frames, in the form of particle trajectories. We then apply the derived multi-frame epipolar constraint to these trajectories to determine which trajectories violate it, thus segmenting out the independently moving objects. We show superior results on a number of moving camera sequences observing non-planar scenes, where other methods fail.

Bio-Inspired Principles Applied to the Guidance, Navigation and Control of UAS

“…This can be thought of as an epipolar constraint. The epipolar constraint has also been utilised in [129] to detect independently moving objects in non-planar scenes. The study by Strydom, et al [128], describes a vision-based technique for the detection of moving objects by a UAS in motion.…”

“…As with most vision-based algorithms, the epipolar constraint algorithm can occasionally be deceived. One instance in which a moving object can escape detection if it moves in such a direction as to create an optic flow vector that has the same direction as the flow created by the stationary background behind it, that is the motion of the object is along the epipolar constraint line [128,129]. This shortcoming is overcome by another approach, termed the Triangle Closure Method, described below.…”

Abandoned object detection using operator-space pursuit

“…However, there are several challenges involved in using a moving camera, as the need of temporal alignment and geometric registration to compensate the movement. Several works in the literature [6][7][8][9][10] address this kind of problem. The challenges are even greater when the environment of interest is visually complex, as is the case of cluttered backgrounds.…”

A Hierarchical Association Framework for Multi-Object Tracking in Airborne Videos

“…Early approaches adopted optical flow [33] or feature points [5,21] to detect and estimate the trajectories of moving objects. Yu and Medioni, in [21], estimated the motion flow in each frame based on a cross-correlation method, and then, a tensor voting approach was used to analyze the optical flow to segment moving objects.…”

“…MOT is challenging due to the uncertain motion of airborne vehicles, the vibration of non-stationary cameras and the partial occlusions of objects [5]. Studies have focused on DATmethods [6] along with the improvement of object detection methods, which provide reliable detection even in complex scenarios.…”