MSL-RAPTOR: A 6DoF Relative Pose Tracker for Onboard Robotic Perception

Abstract

Determining the relative position and orientation of objects in an environment is a fundamental building block for a wide range of robotics applications. To accomplish this task efficiently in practical settings, a method must be fast, use common sensors, and generalize easily to new objects and environments. We present MSL-RAPTOR, a two-stage algorithm for tracking a rigid body with a monocular camera. The image is first processed by an efficient neural network-based front-end to detect new objects and track 2D bounding boxes between frames. The class label and bounding box is passed to the back-end that updates the object's pose using an unscented Kalman filter (UKF). The measurement posterior is fed back to the 2D tracker to improve robustness. The object's class is identified so a class-specific UKF can be used if custom dynamics and constraints are known. Adapting to track the pose of new classes only requires providing a trained 2D object detector or labeled 2D bounding box data, as well as the approximate size of the objects. The performance of MSL-RAPTOR is first verified on the NOCS-REAL275 dataset, achieving results comparable to RGB-D approaches despite not using depth measurements. When tracking a flying drone from onboard another drone, it outperforms the fastest comparable method in speed by a factor of 3, while giving lower translation and rotation median errors by 66% and 23% respectively.

Object Tracking with the Drone: Systems Analysis

“…4. Basic data flow [17] In [18] the researcher relies on a more advanced embedded system. NVIDIA Jetson tx2 is an Embedded system produced by NVIDIA company that contains a graphical processing unit to achieve a higher speed in executing tracking algorithms.…”

“…5. System overview [18] In [19] the researcher uses tracking algorithms designed using deep neural networks. And its implementation on more than one type of embedded system that contains a GPU and that does not contain a GPU.…”

NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challenge

“…The relative location can also be computed without depth measurements, as an additional back-up, and on systems without depth cameras or lidar (such as drones). In this case, we use a monocular based tracking approach over an image sequence (detailed in (Ramtoula et al, 2020)).…”