Event-based, 6-DOF pose tracking for high-speed maneuvers

Abstract

Abstract-In the last few years, we have witnessed impressive demonstrations of aggressive flights and acrobatics using quadrotors. However, those robots are actually blind. They do not see by themselves, but through the "eyes" of an external motion capture system. Flight maneuvers using onboard sensors are still slow compared to those attainable with motion capture systems. At the current state, the agility of a robot is limited by the latency of its perception pipeline. To obtain more agile robots, we need to use faster sensors. In this paper, we present the first onboard perception system for 6-DOF localization during high-speed maneuvers using a Dynamic Vision Sensor (DVS). Unlike a standard CMOS camera, a DVS does not wastefully send full image frames at a fixed frame rate. Conversely, similar to the human eye, it only transmits pixel-level brightness changes at the time they occur with microsecond resolution, thus, offering the possibility to create a perception pipeline whose latency is negligible compared to the dynamics of the robot. We exploit these characteristics to estimate the pose of a quadrotor with respect to a known pattern during high-speed maneuvers, such as flips, with rotational speeds up to 1,200• /s. Additionally, we provide a versatile method to capture ground-truth data using a DVS.

Continuous-Time Trajectory Estimation for Event-based Vision Sensors

“…For the experiments, we used the datasets from [23] and compare the results of our continuous-time approach with those achieved by the event-based reprojection error minimization algorithm in [23]. The first experiment uses data captured in a sensor-in-the-loop simulation.…”

“…In our previous work [23], we demonstrated robot localization in 3-D (with arbitrary 6-DOF motions) using a DVS, with no additional sensing, during high-speed maneuvers, where rotational speeds of up to 1,200…”

“…3) Trajectory Initialization and Extrapolation: We assume the first DVS pose to be known: considering that the map consists of line segments (7), the position of the DVS can be computed by integrating the events caused by the segments and using the Hough transform to detect the corresponding lines (see [23]). Due to the local convergence property of the iterative solving strategy, the control poses must be initialized in the basin of attraction of the optimal value.…”

“…Control poses were placed every 0.1 s. The position and orientation errors are shown in Fig. 4, summarized in Table I, and compared to the results of [23]. Among the plots for the six degrees of freedom, those corresponding to x and y positions and roll angle are the most relevant.…”

“…Eventbased adaptations of iterative closest points [24] and optical flow [5] have already been proposed. Recently, event-based visual odometry [9,17], tracking [28,23], and Simultaneous Localization And Mapping (SLAM) [29] algorithms have also been presented. The design goal of such algorithms is that each incoming event can asynchronously change the estimated state of the system, thus, preserving the event-based nature of the sensor and allowing the design of highly-reactive systems, such as pen balancing [10] or particle tracking in fluids [12].…”