3D Reconstruction and alignment by consumer RGB-D sensors and fiducial planar markers for patient positioning in radiation therapy

Abstract

Background and Objective: Patient positioning is a crucial step in radiation therapy, for which non-invasive methods have been developed based on surface reconstruction using optical 3D imaging. However, most solutions need expensive specialized hardware and a careful calibration procedure that must be repeated over time.This paper proposes a fast and cheap patient positioning method based on inexpensive consumer level RGB-D sensors.Methods: The proposed method relies on a 3D reconstruction approach that fuses, in real-time, artificial and natural visual landmarks recorded from a hand-held RGB-D sensor. The video sequence is transformed into a set of keyframes with known poses, that are later refined to obtain a realistic 3D reconstruction of the patient. The use of artificial landmarks allows our method to automatically align the reconstruction to a reference one, without the need of calibrating the system with respect to the linear accelerator coordinate system. Results:The experiments conducted show that our method obtains a median of 1 cm in translational error, and 1°of rotational error with respect to reference pose. Additionally, the proposed method shows as visual output overlayed poses (from the reference and the current scene) and an error map that can be used to correct the patient's current pose to match the reference pose.Conclusions: A novel approach to obtain 3D body reconstructions for patient positioning without requiring expensive hardware or dedicated graphic cards is proposed. The method can be used to align in real time the patient's current pose to a preview pose, which is a relevant step in radiation therapy.

Fractal Markers: A New Approach for Long-Range Marker Pose Estimation Under Occlusion

“…For these reasons, their use has spread in a wide variety of fields, such as surgery [9]- [11], distributed autonomous 3D printing [12], human-robot interaction [13], autonomous aerial vehicle landing [14], [15], patient positioning in radiotherapy treatments [16], study animal behaviour [17], human cognitive processes [18], 3D body scanning [19], [20], robotic grasping [21], underwater manipulation [22], etc.…”

Joint Scene and Object Tracking for Cost-Effective Augmented Reality Assisted Patient Positioning in Radiation Therapy

“…A new novel approach to computer-assisted patient positioning is presented in [23]. They take advantage of a heightmap data structure reconstructed using their Global ICP algorithm and an RGB-D sensor.…”

Detection of Binary Square Fiducial Markers Using an Event Camera

“…Square binary markers are fiducial markers that can be easily detected in images and their pose can be efficiently determine with respect to the camera given the size of the square marker. One of their most important applications is augmented reality [40,45,37] however they are also used in other fundamental computer vision problems such as camera calibration [2,14], 3D reconstruction [8,35,39], and SLAM [30,32]. Additionally, they are applied to unmanned aerial vehicles [36,3] medicine [39,40], and swarm robotics [44,11] amongst other applications.…”

“…One of their most important applications is augmented reality [40,45,37] however they are also used in other fundamental computer vision problems such as camera calibration [2,14], 3D reconstruction [8,35,39], and SLAM [30,32]. Additionally, they are applied to unmanned aerial vehicles [36,3] medicine [39,40], and swarm robotics [44,11] amongst other applications. One of the first works to design a marker with binary squares for coding and a black edge for easy detection is Matrix [34].…”

“…There are many examples of them such as IGD [49], Matrix [34], binARyID [15], ARToolKitPlus [48], and ARTag [13]. One of the most recent and most widely used versions is the ArUco marker [19] which has been employed in important applications such as medicine and robotics [9,39,43,40,3,36]. The ArUco markers are also used in more fundamental computer vision problems such as camera calibration, environment mapping, and SLAM [38,31,32].…”