Three-dimensional motion estimation of objects for video coding

Abstract

Abstract-In this work, three-dimensional (3-D) motion estimation is applied to the problem of motion compensation for video coding. We suppose that the video sequence consists of the perspective projections of a collection of rigid bodies which undergo a rototranslational motion. Motion compensation can be performed on the sequence once the shape of the objects and the motion parameters are determined. We show that the motion equations of a rigid body can be formulated as a nonlinear dynamic system whose state is represented by the motion parameters and by the scaled depths of the object feature points. An extended Kalman filter is used to estimate both the motion and the object shape parameters simultaneously. The inclusion of the shape parameters in the estimation procedure adds a set of constraints to the filter equations that appear to be essential for reliable motion estimation. Our experiments show that the proposed approach gives two advantages. First, the filter can give more reliable estimates in the presence of measurement noise in comparison with other motion estimators that separately compute motion and structure. Second, the filter can efficiently track abrupt motion changes. Moreover, the structure imposed by the model implies that the reconstructed motion is very natural as opposed to more common block-based schemes. Also, the parameterization of the model allows for a very efficient coding of motion information.

Model-Based Global and Local Motion Estimation for Videoconference Sequences

“…For features, we can write the corresponding system of equations in compact form as (4) The dependence from the control point coordinates was not made explicit, since these must be determined only once, when the features are chosen in the first frame, and do not change over time. Remember that once the position of the CANDIDE model in 3-D space at a given time instant is known, the 3-D coordinates of a control point can be computed from the 2-D coordinates of the corresponding feature by inverting the projection equations (3). Anyway, in the following section we will see that new features are added from time to time to replace lost ones.…”

“…With this assumption, the face has six degrees of freedom, so its position in 3-D space can be defined with six parameters, e.g., the components of the translation vector and of the rotation vector . In this case, the equations relating the 3-D coordinates of a control point to the 2-D coordinates of the corresponding feature on the image plane are given by (3) where is the projection operator defined in (2) and is a 3 3 matrix whose elements are the sines and cosines of the rotation angles, as in [2]. With this choice of the matrix, rotation around the axis is applied first, then rotation around the axis, and finally rotation around the axis.…”

“…A simplified but effective approach for head global motion consists in tracking the position of a set of characteristic points, called features, from one Manuscript frame to the next [1]- [3]. An extended Kalman filter (EKF) can then be used to determine the motion parameters from the feature positions, which are modeled as the projection onto the camera plane of 3-D points of the head, considered as a rigid body.…”

Multi-objective retrieval of object pose from video

“…Works [15,3,5, 161 addressing problems of "Shapefrom-Motion" and "Structure-from-Motion" in computer vision give an estimate of object pose in each of input images as a mid-way result. The main problem with this type of algorithms is that they rely on feature point correspondence which is prone to error in spite of large volumes of research [18].…”