Content-Aware Unsupervised Deep Homography Estimation

Zhang, Jirong; Wang, Chuan; Liu, Shuaicheng; Jia, Lanpeng; Ye, Nianjin; Wang, Jue; Zhou, Ji; Sun, Jian

doi:10.1007/978-3-030-58452-8_38

Cited by 136 publications

(196 citation statements)

References 35 publications

Supporting

Mentioning

196

Contrasting

Order By: Relevance

“…For gesture recognition, we propose a motion-based approach which processes image sequences at two temporal levels (Figure 1). First, to analyze global motion between two consecutive video frames, we explore a convolutional network for homography estimation [28], which we refer to as CAUDHE (Content-Aware Unsupervised Deep Homography Estimation). Two key features of this recently proposed network are (1) its unsupervised nature, which facilitates training since no true homography values are required for each input frame pair, and (2) its robustness to independently moving objects since the object regions are masked out through an outlier rejector.…”

Section: Network Architecture and Trainingmentioning

confidence: 99%

Head and eye egocentric gesture recognition for human-robot interaction using eyewear cameras

Marina-Miranda,

Traver

2022

Preprint

View full text Add to dashboard Cite

Non-verbal communication plays a particularly important role in a wide range of scenarios in Human-Robot Interaction (HRI). Accordingly, this work addresses the problem of human gesture recognition. In particular, we focus on head and eye gestures, and adopt an egocentric (first-person) perspective using eyewear cameras. We argue that this egocentric view offers a number of conceptual and technical benefits over sceneor robot-centric perspectives.A motion-based recognition approach is proposed, which operates at two temporal granularities. Locally, frame-to-frame homographies are estimated with a convolutional neural network (CNN). The output of this CNN is input to a long shortterm memory (LSTM) to capture longer-term temporal visual relationships, which are relevant to characterize gestures.Regarding the configuration of the network architecture, one particularly interesting finding is that using the output of an internal layer of the homography CNN increases the recognition rate with respect to using the homography matrix itself. While this work focuses on action recognition, and no robot or user study has been conducted yet, the system has been designed to meet real-time constraints. The encouraging results suggest that the proposed egocentric perspective is viable, and this proof-of-concept work provides novel and useful contributions to the exciting area of HRI.

show abstract

Section: Network Architecture and Trainingmentioning

confidence: 99%

Head and eye egocentric gesture recognition for human-robot interaction using eyewear cameras

Marina-Miranda,

Traver

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…After that, calculate inliers and save it where SSD ( )<thr. Lastly, use inliers to re-calculate least squares H if the inliers grow over a specific threshold [26][27][28][29][30].…”

Section: Image Registrationmentioning

confidence: 99%

Generation of high dynamic range for enhancing the panorama environment

et al. 2021

View full text Add to dashboard Cite

This paper presents a methodology for enhancement of panorama images environment by calculating high dynamic range. Panorama is constructing by merge of several photographs that are capturing by traditional cameras at different exposure times. Traditional cameras usually have much lower dynamic range compared to the high dynamic range in the real panorama environment, where the images are captured with traditional cameras will have regions that are too bright or too dark. A more details will be visible in bright regions with a lower exposure time and more details will be visible in dark regions with a higher exposure time. Since the details in both bright and dark regions cannot preserve in the images that are creating using traditional cameras, the proposed system have to calculate one using the images that traditional camera can actually produce. The proposed systems start by get LDR panorama image from multiple LDR images using SIFT features technology and then convert this LDR panorama image to the HDR panorama image using inverted local patterns. The results in this paper explained that the HDR panorama images that resulting from the proposed method is more realistic image and appears as it is a real panorama environment.

show abstract

“…This can cause jitter. This issue may be tackled by estimating the motion on a more global (holistic) scale, as in [ 15 ] or more recently by using deep-learning, regression-based approaches, such as in [ 16 ] or [ 17 ]. Holistic approaches tend to be computationally expensive as compared to feature-point approaches and still do not prevent the occurrence of noise, as theorized in Section 3.1 .…”

Section: Prior Workmentioning

confidence: 99%

Registering Unmanned Aerial Vehicle Videos in the Long Term

Lemaire

Crispim-Junior

Robinault³

et al. 2021

Sensors

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) have become a very popular way of acquiring video within contexts such as remote data acquisition or surveillance. Unfortunately, their viewpoint is often unstable, which tends to impact the automatic processing of their video flux negatively. To counteract the effects of an inconsistent viewpoint, two video processing strategies are classically adopted, namely registration and stabilization, which tend to be affected by distinct issues, namely jitter and drifting. Following our prior work, we suggest that the motion estimators used in both situations can be modeled to take into account their inherent errors. By acknowledging that drifting and jittery errors are of a different nature, we propose a combination that is able to limit their influence and build a hybrid solution for jitter-free video registration. In this work, however, our modeling was restricted to 2D-rigid transforms, which are rather limited in the case of airborne videos. In the present paper, we extend and refine the theoretical ground of our previous work. This addition allows us to show how to practically adapt our previous work to perspective transforms, which our study shows to be much more accurate for this problem. A lightweight implementation enables us to automatically register stationary UAV videos in real time. Our evaluation includes traffic surveillance recordings of up to 2 h and shows the potential of the proposed approach when paired with background subtraction tasks.

show abstract

Content-Aware Unsupervised Deep Homography Estimation

Cited by 136 publications

References 35 publications

Head and eye egocentric gesture recognition for human-robot interaction using eyewear cameras

Head and eye egocentric gesture recognition for human-robot interaction using eyewear cameras

Generation of high dynamic range for enhancing the panorama environment

Registering Unmanned Aerial Vehicle Videos in the Long Term

Contact Info

Product

Resources

About