This paper presents a joint head pose and facial landmark regression method with input from depth images for realtime application. Our main contributions are: firstly, a joint optimization method to estimate head pose and facial landmarks, i.e., the pose regression result provides supervised initialization for cascaded facial landmark regression, while the regression result for the facial landmarks can also help to further refine the head pose at each stage. Secondly, we classify the head pose space into 9 sub-spaces, and then use a cascaded random forest with a global shape constraint for training facial landmarks in each specific space. This classification-guided method can effectively handle the problem of large pose changes and occlusion. Lastly, we have built a 3D face database containing 73 subjects, each with 14 expressions in various head poses. Experiments on challenging databases show our method achieves state-of-the-art performance on both head pose estimation and facial landmark regression.

No abstract

No abstract

A real-time digital video stabilization system is proposed to remove unwanted camera shakes and jitters. Firstly, SIFT algorithm is improved to extract and match features between the reference frame and current frame reliably, and then global motion parameters are obtained based on the geometric constraint consistency between feature matches through random sample consensus algorithm. Secondly, multiple evaluation criteria are fused by an adaptive lowpass filter to smooth global motion for obtaining correction vector, which is used to compensate the current frame. Finally, stabilized video is obtained after each frame is completed by combining the texture synthesis method and the spatio-temporal information of video. The objective experiments demonstrate the system can increase the average peak signal-to-noise ratio of jittered videos around 6.12 dB, The subjective experiments demonstrate the system can increase the identification ability and perceptive comfort on video content.

No abstract