Multimodal medical image fusion using convolutional neural network and extreme learning machine

Kong, Weiwei; Li, Chi; Lei, Yang

doi:10.3389/fnbot.2022.1050981

Cited by 13 publications

(5 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different image data types can provide more comprehensive and accurate information [ 26 ]. Bi et al [ 98 ] designed a robotic assist system for prostate intervention to improve the accuracy and real-time puncture.…”

Section: Optimization Methods Of Medical Image Navigation Technology ...mentioning

confidence: 99%

“…A single type of medical imaging information cannot provide comprehensive physiological and anatomical information [ 25 ]. Due to various factors, single-modality medical imaging technology has limitations [ 26 ]. The penetration ability of ultrasound is limited, and sometimes it is challenging to obtain image information of tissues and organs with greater depth or complex structures.…”

Section: Overview Of Medical Imaging Technologymentioning

confidence: 99%

See 1 more Smart Citation

Application of Medical Image Navigation Technology in Minimally Invasive Puncture Robot

Lü

Zhu

et al. 2023

Sensors

View full text Add to dashboard Cite

Microneedle puncture is a standard minimally invasive treatment and surgical method, which is widely used in extracting blood, tissues, and their secretions for pathological examination, needle-puncture-directed drug therapy, local anaesthesia, microwave ablation needle therapy, radiotherapy, and other procedures. The use of robots for microneedle puncture has become a worldwide research hotspot, and medical imaging navigation technology plays an essential role in preoperative robotic puncture path planning, intraoperative assisted puncture, and surgical efficacy detection. This paper introduces medical imaging technology and minimally invasive puncture robots, reviews the current status of research on the application of medical imaging navigation technology in minimally invasive puncture robots, and points out its future development trends and challenges.

show abstract

Section: Optimization Methods Of Medical Image Navigation Technology ...mentioning

confidence: 99%

Section: Overview Of Medical Imaging Technologymentioning

confidence: 99%

Application of Medical Image Navigation Technology in Minimally Invasive Puncture Robot

Lü

Zhu

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Features extracted from different modalities were fused so that only the lower-level, middle-level, and higher-level image contents are extracted 29 . The combination of extreme learning machines and convolutional neural network have been proposed for feature extraction and fusion on multimodal images to support the classification accuracy and localization of medical images 60 . Another approach to multimodality is the consideration of multicolor imaging for the purpose of extracting features which reveal sufficient symptoms to arrive at the detection of diseases.…”

Section: Related Workmentioning

confidence: 99%

A twin convolutional neural network with hybrid binary optimizer for multimodal breast cancer digital image classification

Oyelade,

Irunokhai,

Wang

2024

Sci Rep

View full text Add to dashboard Cite

There is a wide application of deep learning technique to unimodal medical image analysis with significant classification accuracy performance observed. However, real-world diagnosis of some chronic diseases such as breast cancer often require multimodal data streams with different modalities of visual and textual content. Mammography, magnetic resonance imaging (MRI) and image-guided breast biopsy represent a few of multimodal visual streams considered by physicians in isolating cases of breast cancer. Unfortunately, most studies applying deep learning techniques to solving classification problems in digital breast images have often narrowed their study to unimodal samples. This is understood considering the challenging nature of multimodal image abnormality classification where the fusion of high dimension heterogeneous features learned needs to be projected into a common representation space. This paper presents a novel deep learning approach combining a dual/twin convolutional neural network (TwinCNN) framework to address the challenge of breast cancer image classification from multi-modalities. First, modality-based feature learning was achieved by extracting both low and high levels features using the networks embedded with TwinCNN. Secondly, to address the notorious problem of high dimensionality associated with the extracted features, binary optimization method is adapted to effectively eliminate non-discriminant features in the search space. Furthermore, a novel method for feature fusion is applied to computationally leverage the ground-truth and predicted labels for each sample to enable multimodality classification. To evaluate the proposed method, digital mammography images and digital histopathology breast biopsy samples from benchmark datasets namely MIAS and BreakHis respectively. Experimental results obtained showed that the classification accuracy and area under the curve (AUC) for the single modalities yielded 0.755 and 0.861871 for histology, and 0.791 and 0.638 for mammography. Furthermore, the study investigated classification accuracy resulting from the fused feature method, and the result obtained showed that 0.977, 0.913, and 0.667 for histology, mammography, and multimodality respectively. The findings from the study confirmed that multimodal image classification based on combination of image features and predicted label improves performance. In addition, the contribution of the study shows that feature dimensionality reduction based on binary optimizer supports the elimination of non-discriminant features capable of bottle-necking the classifier.

show abstract

“…For the enhancement of image fusion efficiency, methodologies grounded in machine learning were employed [20][21][22]. Overlaps amidst images were initially pinpointed using feature point detection.…”

Section: Application Of Image Fusion Techniques In Tree Measurementmentioning

confidence: 99%

Estimation of Forest Diameter-at-Breast-Height: A Fusion of Machine Learning and 3D Image Processing Innovations

Wang,

Sun,

Wang

2023

View full text Add to dashboard Cite

As technology persistently advances, breakthroughs in machine learning and image processing have been harnessed for the meticulous measurement and analysis of natural resources. In the pursuit of addressing the imperative task of feature extraction and measurement within forestry, an integration of convolutional neural networks (CNNs), traditional machine learning, and image processing techniques has been devised. Highresolution 3D image data were procured using the D435i depth camera, targeting the detailed representation of tree structures. Upon acquisition, refined strategies encompassing passthrough filtering and K-means clustering were utilised for noise mitigation and segmentation. For feature discernment, CNNs were synergised with other machine learning models, facilitating comprehensive and automated extraction of the tree's structural and morphological nuances. The Random Sample Consensus (RANSAC) algorithm was subsequently invoked for meticulous cylindrical shape fitting, culminating in precise estimations of tree diameter-at-breast-height. Rigorous experimental validation revealed not only eminent accuracy but also unparalleled robustness across a gamut of scenarios and environments. When juxtaposed with conventional forestry measurement techniques, this methodology unmistakably signals a promising trajectory for forthcoming forestry applications.

show abstract

Multimodal medical image fusion using convolutional neural network and extreme learning machine

Cited by 13 publications

References 58 publications

Application of Medical Image Navigation Technology in Minimally Invasive Puncture Robot

Application of Medical Image Navigation Technology in Minimally Invasive Puncture Robot

A twin convolutional neural network with hybrid binary optimizer for multimodal breast cancer digital image classification

Estimation of Forest Diameter-at-Breast-Height: A Fusion of Machine Learning and 3D Image Processing Innovations

Contact Info

Product

Resources

About