2D–3D reconstruction of distal forearm bone from actual X-ray images of the wrist using convolutional neural networks

Shiode, Ryoya; Mototaka, Kabashima; Hiasa, Yuta; Oka, Kunihiro; Murase, Tsuyoshi; Sato, Yoshinobu; Otake, Yoshito

doi:10.1038/s41598-021-94634-2

Cited by 30 publications

(8 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the effort, facilities, time and, therefore, costs required to acquire and annotate a dataset of even this size are substantial due to the nature of the data. Further, we note that using a few hundred images, as we do for the hip-imaging X-ray tasks, is a typical size in the literature 5,12,[71][72][73][74][75][76] , and most of the existing work on developing machine learning solutions for intra-operative X-ray analysis tasks, such as 2D/3D registration, do not develop nor test their methods on any real data 13 . In summary, while datasets of the size reported here may not accurately reflect all of the variability one may expect during image-based surgery, the models trained on our datasets performed well on held-out data, using both leave-one-subject-out cross-validations and an independent test set, and performed comparably to previous studies on larger datasets 5,77 .…”

Section: Discussionmentioning

confidence: 99%

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

Gao

Killeen

et al. 2023

Nat Mach Intell

View full text Add to dashboard Cite

Artificial intelligence (AI) now enables automated interpretation of medical images. However, AI's potential use for interventional image analysis remains largely untapped. This is because the post hoc analysis of data collected during live procedures has fundamental and practical limitations, including ethical considerations, expense, scalability, data integrity and a lack of ground truth. Here we demonstrate that creating realistic simulated images from human models is a viable alternative and complement to large-scale in situ data collection. We show that training AI image analysis models on realistically synthesized data, combined with contemporary domain generalization techniques, results in machine learning models that on real data perform comparably to models trained on a precisely matched real data training set. We find that our model transfer paradigm for X-ray image analysis, which we refer to as SyntheX, can even outperform real-data-trained models due to the effectiveness of training on a larger dataset. SyntheX provides an opportunity to markedly accelerate the conception, design and evaluation of X-ray-based intelligent systems. In addition, SyntheX provides the opportunity to test novel instrumentation, design complementary surgical approaches, and envision novel techniques that improve outcomes, save time or mitigate human error, free from the ethical and practical considerations of live human data collection.Advances in robotics and artificial intelligence (AI) are bringing autonomous surgical systems closer to reality. However, developing the AI backbones of such systems currently depends on collecting training data during routine surgeries. This remains one of the largest barriers to widespread use of AI systems in interventional clinical settings, versus triage or diagnostic settings, as the acquisition and annotation of interventional data is time intensive and costly. Furthermore, while this approach can contribute to the automation or streamlining of existing surgical workflows, robotic and autonomous systems promise even more substantial advances: novel and super-human techniques that improve outcomes, save time or mitigate human error. This is perhaps the most exciting frontier of computer-assisted intervention research.Conventional approaches for curating data for AI development (that is, sourcing it retroactively from clinical practice) are insufficient for training AI models that benefit interventions that use novel instrumentation, different access points or more flexible imaging. This is because they are, by definition, incompatible with contemporary clinical practices and such data do not emerge from routine care. Furthermore, these novel systems are not readily approved, and thus not easily or quickly introduced into clinical practice. Ex vivo experimentation

show abstract

Section: Discussionmentioning

confidence: 99%

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

Gao

Killeen

et al. 2023

Nat Mach Intell

View full text Add to dashboard Cite

show abstract

“…Although stemming from real patient CT scans, the X-ray images used for training and testing the X23D algorithm were simulated by using a custom-made software to streamline the data collection, annotation and curation in light of the practical hurdles along the collection of paired real CT and X-ray datasets. Prior work, such as [ 71 , 72 ], show that clinically feasible simulations of X-rays can be achieved by more advanced simulation methods. Such X-ray simulation algorithms will be implemented into the X23D pipeline in the future phases of this project.…”

Section: Discussionmentioning

confidence: 99%

X23D—Intraoperative 3D Lumbar Spine Shape Reconstruction Based on Sparse Multi-View X-ray Data

et al. 2022

View full text Add to dashboard Cite

Visual assessment based on intraoperative 2D X-rays remains the predominant aid for intraoperative decision-making, surgical guidance, and error prevention. However, correctly assessing the 3D shape of complex anatomies, such as the spine, based on planar fluoroscopic images remains a challenge even for experienced surgeons. This work proposes a novel deep learning-based method to intraoperatively estimate the 3D shape of patients’ lumbar vertebrae directly from sparse, multi-view X-ray data. High-quality and accurate 3D reconstructions were achieved with a learned multi-view stereo machine approach capable of incorporating the X-ray calibration parameters in the neural network. This strategy allowed a priori knowledge of the spinal shape to be acquired while preserving patient specificity and achieving a higher accuracy compared to the state of the art. Our method was trained and evaluated on 17,420 fluoroscopy images that were digitally reconstructed from the public CTSpine1K dataset. As evaluated by unseen data, we achieved an 88% average F1 score and a 71% surface score. Furthermore, by utilizing the calibration parameters of the input X-rays, our method outperformed a counterpart method in the state of the art by 22% in terms of surface score. This increase in accuracy opens new possibilities for surgical navigation and intraoperative decision-making solely based on intraoperative data, especially in surgical applications where the acquisition of 3D image data is not part of the standard clinical workflow.

show abstract

“…They perform well with the average of accuracy taken as 93.67% for the fivefold cross-validation using learning models based on ResNet101 and Inception-V3. Furthermore, another author developed a three-dimensional bone model system which is based on employment of x-ray images for distal forearm engaging the convolutional neural networks [27]. The deep learning framework is employed in estimating and to construct a high accuracy for three-dimensional model of bones.…”

Section: Ensemble Machine Learningmentioning

confidence: 99%

Ensemble Machine Learning Algorithms for Prediction and Classification of Medical Images

S.¹,

Daramola²

2021

Artificial Intelligence

View full text Add to dashboard Cite

The employment of machine learning algorithms in disease classification has evolved as a precision medicine for scientific innovation. The geometric growth in various machine learning systems has paved the way for more research in the medical imaging process. This research aims to promote the development of machine learning algorithms for the classification of medical images. Automated classification of medical images is a fascinating application of machine learning and they have the possibility of higher predictability and accuracy. The technological advancement in the processing of medical imaging will help to reduce the complexities of diseases and some existing constraints will be greatly minimized. This research exposes the main ensemble learning techniques as it covers the theoretical background of machine learning, applications, comparison of machine learning and deep learning, ensemble learning with reviews of state-of the art literature, framework, and analysis. The work extends to medical image types, applications, benefits, and operations. We proposed the application of the ensemble machine learning approach in the classification of medical images for better performance and accuracy. The integration of advanced technology in clinical imaging will help in the prompt classification, prediction, early detection, and a better interpretation of medical images, this will, in turn, improves the quality of life and expands the clinical bearing for machine learning applications.

show abstract

2D–3D reconstruction of distal forearm bone from actual X-ray images of the wrist using convolutional neural networks

Cited by 30 publications

References 34 publications

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

X23D—Intraoperative 3D Lumbar Spine Shape Reconstruction Based on Sparse Multi-View X-ray Data

Ensemble Machine Learning Algorithms for Prediction and Classification of Medical Images

Contact Info

Product

Resources

About