Computed Tomography Data Colouring Based on Photogrammetric Images

Abstract

Abstract. Nowadays various methods and sensors are available for 3D reconstruction tasks; however, it is still necessary to integrate advantages of different technologies for optimizing the quality 3D models. Computed tomography (CT) is an imaging technique which takes a large number of radiographic measurements from different angles, in order to generate slices of the object, however, without colour information. The aim of this study is to put forward a framework to extract colour information from photogrammetric images for corresponding Computed Tomography (CT) surface data with high precision. The 3D models of the same object from CT and photogrammetry methods are generated respectively, and a transformation matrix is determined to align the extracted CT surface to the photogrammetric point cloud through a coarse-to-fine registration process. The estimated pose information of images to the photogrammetric point clouds, which can be obtained from the standard image alignment procedure, also applies to the aligned CT surface data. For each camera pose, a depth image of CT data is calculated by projecting all the CT points to the image plane. The depth image is in principle should agree with the corresponding photogrammetric image. The points, which cannot be seen from the pose, but are also projected on the depth image, are excluded from the colouring process. This is realized by comparing the range values of neighbouring pixels and finding the corresponding 3D points with larger range values. The same procedure is implemented for all the image poses to obtain the coloured CT surface. Thus, by using photogrammetric images, we achieve a coloured CT dataset with high precision, which combines the advantages from both methods. Rather than simply stitching different data, we deep-dive into the photogrammetric 3D reconstruction process and optimize the CT data with colour information. This process can also provide an initial route and more options for other data fusion processes.

Making Historical Gyroscopes Alive—2D and 3D Preservations by Sensor Fusion and Open Data Access

“…The second step is a transformation of the processed voxel cloud to a point cloud (see Figure 4 b). We have proven the two-step voxel-to-point cloud processing in [ 23 ] when coloring voxel clouds using photogrammetric hull textures. For the merging of two point clouds, the iterative closest point (ICP) algorithm [ 24 ] implemented in Open Source point cloud libraries, such as CloudCompare, MeshLab, PCL, and Open3D, is considered to be the classic approach.…”

Photogrammetry and Computed Tomography Point Cloud Registration Using Virtual Control Points

“…Though having the ability to recover the internal information in a non-destructive way, CT 3D reconstruction suffers from the noise and beam hardening artefacts and scattering. More details of 3D reconstruction as well as the CT surface extraction process could be referred to (Zhan et al, 2020).…”

“…The sensors used for CT scanning and photogrammetry can be referred to (Zhan et al, 2020). As for photogrammetric 3D reconstructions, Figure 4 shows the camera poses represented by circular white rectangles via SfM (Structure-from-Motion) and the watertight 3D surface model from DIM (Dense Image Matching) and texturing procedure.…”

“…We apply a ball query method for each point to extract the points with less neighbouring points within a threshold range, which are interpreted as the points on the surface of the 3D CT model. An alternative method finding the convex hull could be found in (Zhan et al, 2020). Figure 6(a) shows the surface extraction result.…”

“…In the 3D digitization field of Tech Heritage (TH), the Gyrolog project (Fritsch et al, 2018;Fritsch et al, 2021) funded by the Federal Ministry of Education and Research of Germany, has innovatively introduced different methodologies, such as Photogrammetry, Computed Tomography (CT) as well as Endoscopy for the 3D digitization of the gyroscopic instrument collection of the University of Stuttgart. The combination of photogrammetry and CT has been discussed frequently in the medical field (Bolandzadeh et al, 2013), however, has not yet been applied often in TH digitization applications (Zhan et al, 2020). This paper mainly discusses a new registration method of photogrammetric and CT point clouds in such TH applications, where complete models are required.…”