Real-Time, Flight-Ready, Non-Cooperative Spacecraft Pose Estimation Using Monocular Imagery

Abstract

A key requirement for autonomous on-orbit proximity operations is the estimation of a target spacecraft's relative pose (position and orientation). It is desirable to employ monocular cameras for this problem due to their low cost, weight, and power requirements. This work presents a novel convolutional neural network (CNN)-based monocular pose estimation system that achieves state-of-the-art accuracy with low computational demand. In combination with a Blender-based synthetic data generation scheme, the system demonstrates the ability to generalize from purely synthetic training data to real in-space imagery of the Northrop Grumman Enhanced Cygnus spacecraft. Additionally, the system achieves realtime performance on low-power flight-like hardware.

LSPnet: A 2D Localization-oriented Spacecraft Pose Estimation Neural Network

“…Deep Learning keypoints regression combined with PnP solving. The works of [4,15,1] all perform a first step of zooming in into a ROI yielded by an Object detection neural network. Afterwards, [4,1] regress a set of manually selected keypoints while [15] regresses the corners of the target spacecraft in an ordered manner to avoid additional matching computations.…”

“…The works of [4,15,1] all perform a first step of zooming in into a ROI yielded by an Object detection neural network. Afterwards, [4,1] regress a set of manually selected keypoints while [15] regresses the corners of the target spacecraft in an ordered manner to avoid additional matching computations. Keypoint-based pose estimation solutions are generally robust and accurate provided that high quality 2D-3D correspondences can be obtained beforehand.…”

CubeSat-CDT: A Cross-Domain Dataset for 6-DoF Trajectory Estimation of a Symmetric Spacecraft

“…As a consequence, existing spaceborne images captured from previous missions are sometimes combined with synthetic data. In particular, the Cygnus dataset [4] contains 540 pictures of the Cygnus spacecraft in orbit in conjunction with 20k synthetic images generated with Blender [1]. However, the main limitation of spaceborne images is SPARK [20] SPEED [15] SPEED+ [21] URSO [26] SwissCube [13] Cygnus [4] Prisma12K [22] Prisma25 [7] the lack of accurate pose labels and their limited diversity in terms of pose distribution.…”

“…In particular, the Cygnus dataset [4] contains 540 pictures of the Cygnus spacecraft in orbit in conjunction with 20k synthetic images generated with Blender [1]. However, the main limitation of spaceborne images is SPARK [20] SPEED [15] SPEED+ [21] URSO [26] SwissCube [13] Cygnus [4] Prisma12K [22] Prisma25 [7] the lack of accurate pose labels and their limited diversity in terms of pose distribution. To overcome these difficulties, laboratory setups trying to mimic space conditions currently represent the de facto target domain for spacecraft pose estimation algorithms [21].…”

Vision-based Neural Scene Representations for Spacecraft

“…The efficient and automated determination of poses is actively studied in similar contexts and in [10], for example, the spacecraft pose is determined to a high degree of precision from a single two-dimensional image assuming a detailed knowledge of the spacecraft model. In [20,6] pose reconstruction was carried out from a large dataset of two dimensional images without explicit use of a model. Space Situational Awareness [11], in-orbit servicing, manufacturing and even recycling are also relevant and active fields of aerospace research where the acquisition of information about a generic target structural integrity, function and pose is pursued.…”