Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation

Baghaie, Ahmadreza; Tafti, Ahmad P.; Owen, Heather A.; D’Souza, Roshan; Yu, Zeyun

doi:10.1371/journal.pone.0175078

Cited by 16 publications

(17 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hybrid approaches combine the advantages from both single-and multi-view approaches, but they are yet to be fully designed and validated (Tafti et al 2015). Single-view setups are a well-studied and the most adopted approaches in the literature (Baghaie et al 2017).…”

Section: Microphotogrammetrymentioning

confidence: 99%

“…As in optical stereophotogrammetry, they created 3D models from series of overlapping images of the sample, in this case a rabbit kidney glomerulus. These images were produced by rotating the sample under the fixed detector to acquire data from multiple angles, a variant of multi-view techniques (Baghaie et al 2017). The result showed an enhanced 3D reconstruction quality and better preservation of the original specimen (Eulitz and Reiss 2015).…”

Section: Microphotogrammetrymentioning

confidence: 99%

See 1 more Smart Citation

Application of Photogrammetry in Biomedical Science

Struck

Cordoni

Aliotta

et al. 2019

Advances in Experimental Medicine and Biology

View full text Add to dashboard Cite

Photogrammetry is an upcoming technology in biomedical science as it provides a non-invasive and cost-effective alternative to established 3D imaging techniques such as computed tomography. This review introduces the photogrammetry approaches currently used for digital 3D reconstruction in biomedical science and discusses their suitability for different applications. It aims to offer the reader a better understanding of photogrammetry as a 3D reconstruction technique and to provide some guidance on how to choose the appropriate photogrammetry approach for their research area (including single-versus multi-camera setups, structure-from-motion versus conventional photogrammetry and macro-versus microphotogrammetry) as well as guidance on how to obtain high-quality data. This review highlights some key advantages of photogrammetry for a variety of applications in biomedical science, but it also discusses the limitations of this technique and the importance of taking steps to obtain high-quality images for accurate 3D reconstruction.

show abstract

Section: Microphotogrammetrymentioning

confidence: 99%

Section: Microphotogrammetrymentioning

confidence: 99%

Application of Photogrammetry in Biomedical Science

Struck

Cordoni

Aliotta

et al. 2019

Advances in Experimental Medicine and Biology

View full text Add to dashboard Cite

show abstract

“…To eliminate anomalous of the grid model of the micro surface, an adaptive median filter was used [Baghaie et al, 2017]. Before filtering, the grid model was converted to an image, where the elevations were displayed in proportion to the brightness values at the range [0,1].…”

Section: Finding the Corresponding Point Coordinates In The Right Imamentioning

confidence: 99%

“…Methods of SEM photogrammetry allow creation of digital models of micro surfaces of experimental objects from their SEM stereo images and to obtain their quantitative parameters (surface profiles, the magnitude of its deformation, microcrack sizes, micro surface area, etc.) [Baghaie et al, 2017;Balamucki et al, 2006;Cornille et al, 2003;Ivanchuk, 2019;Kudryavtsev, 2017;Marturi et al, 2013, Melnik et al, 1999, 2009Nicolls, 2004;Shostak, 2012;Tafti, 2016;Voloshin, 2004;Zhu et al, 2011]. The use of digital models made it possible to represent the structure of micro surfaces by changing angle, viewpoint, projection, scale visually.…”

Section: Introductionmentioning

confidence: 99%

Geodesy, Cartography and Aerial Photography

Ivanchuk¹,

Tumska²

2019

ISTCGCAP

View full text Add to dashboard Cite

Purpose. The goal of this work was the development and research of a method of automatically constructing a digital model of the micro surface of an object from SEM stereo pair of digital images taking into account the specifics of the survey SEM and evaluating the accuracy of digital modeling. Methods. The developed method consists, firstly, in generating a dense set of input points in the left SEM image of a stereo pair in regions with local features and using an iterative process in accordance with the levels of the image pyramid. Secondly, the search for the corresponding points in the right SEM image of the stereo pair is carried out on the basis of sequentially shifting the points (centers of the search windows) by a shift parameter from the possible parallax's range using the correlation method. For research, we have used two stereo pairs of digital SEM images. Digital images of the deformed surface chrome steel specimen were acquired with the JSM 7100F (JEOL) with magnification 750x. Images of loess soil were acquired with the SEM "Hitachi" S-800 with magnification 1000x. When calculating the spatial coordinates of the points of the surface micro relief, the values of geometric distortion inherent in the SEM image were taken into account. To eliminate some anomalous values of the heights of the 3D model, an adaptive median filtering procedure was applied. To evaluate the accuracy of micro surface simulation test models were created by manually measuring coordinate feature points of the digital stereo pairs for both specimens. Results. The proposed method for shifting parameters reduces the search area and the probability of mismatch and, in addition, speeds up the matching procedure in a pair of images. Formulas are obtained for calculating the coordinates of the center of the search window and the corresponding point in the right image at the k-th step of the shift process. To estimate the accuracy, the differences between the heights of the test model and the heights interpolated at the same points using the created models were computed. For the chrome steel specimen micro surface about 79 % of the points, and for the micro surface specimen of the loess soil about 70 % of the points were within tolerance ΔZ ≤ ± 2 μm. Scientific novelty. For the first time in Ukraine, a method was developed for an automatic search of corresponding points based on a shift of parameters taking into account the features of SEM survey. The proposed technological reconstruction automation scheme of a digital model of an object's micro surface from SEM stereo pair, and the creation of this authoring software show its efficiency and expediency. The practical significance. The ability to reproduce the surface micro relief of an object automatically using a stereo pair of SEM digital images was established in accordance with the requirements of both the accuracy of determining the spatial coordinates of points and the structure of the micro surface of the object.

show abstract

“…Subsequently, the structure is triangulated from sparse feature correspondences and a surface mesh is generated. Baghaie et al [ 10 , 11 ] determine a dense reconstruction applying algorithms that are partly linked to the perspective and orthographic projection model. The images are rectified based on the general pinhole camera model to perform a row-wise search for dense correspondences.…”

Section: Introductionmentioning

confidence: 99%

Quantitative 3D Reconstruction from Scanning Electron Microscope Images Based on Affine Camera Models

Töberg

Reithmeier

2020

Sensors

View full text Add to dashboard Cite

Scanning electron microscopes (SEMs) are versatile imaging devices for the micro- and nanoscale that find application in various disciplines such as the characterization of biological, mineral or mechanical specimen. Even though the specimen’s two-dimensional (2D) properties are provided by the acquired images, detailed morphological characterizations require knowledge about the three-dimensional (3D) surface structure. To overcome this limitation, a reconstruction routine is presented that allows the quantitative depth reconstruction from SEM image sequences. Based on the SEM’s imaging properties that can be well described by an affine camera, the proposed algorithms rely on the use of affine epipolar geometry, self-calibration via factorization and triangulation from dense correspondences. To yield the highest robustness and accuracy, different sub-models of the affine camera are applied to the SEM images and the obtained results are directly compared to confocal laser scanning microscope (CLSM) measurements to identify the ideal parametrization and underlying algorithms. To solve the rectification problem for stereo-pair images of an affine camera so that dense matching algorithms can be applied, existing approaches are adapted and extended to further enhance the yielded results. The evaluations of this study allow to specify the applicability of the affine camera models to SEM images and what accuracies can be expected for reconstruction routines based on self-calibration and dense matching algorithms.

show abstract

Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation

Cited by 16 publications

References 69 publications

Application of Photogrammetry in Biomedical Science

Application of Photogrammetry in Biomedical Science

Geodesy, Cartography and Aerial Photography

Quantitative 3D Reconstruction from Scanning Electron Microscope Images Based on Affine Camera Models

Contact Info

Product

Resources

About