Improving needle visibility in LED-based photoacoustic imaging using deep learning with semi-synthetic datasets

Shi, Miao; Zhao, Tianrui; West, Simeon J.; Desjardins, Adrien E.; Vercauteren, Tom; Xia, Wenfeng

doi:10.1016/j.pacs.2022.100351

Cited by 16 publications

(10 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This shows that it is possible to achieve a real-time frame rate of around 10 Hz with DAS+LCF beamformer when reconstructing an image with a pixel size of 512 × 512, making it a good choice for any real-time PA imaging systems. In recent years, LED-based PA imaging has been used for multiple applications including cancer imaging, 44 monitoring tumor progression, 41 needle-based biopsy, 45 and several other pre-clinical and clinical applications. 46,47 We believe that our proposed method can be used for enhancing the image quality in these applications and also potentially opens up new clinical imaging applications requiring higher SNR and resolution at realtime frame rates.…”

Section: Discussionmentioning

confidence: 99%

Improvement of LED‐based photoacoustic imaging using lag‐coherence factor (LCF) beamforming

Paul,

Mulani,

Singh

et al. 2023

Medical Physics

View full text Add to dashboard Cite

BackgroundOwing to its portability, affordability, and energy‐efficiency, LED‐based photoacoustic (PA) imaging is increasingly becoming popular when compared to its laser‐based alternative, mainly for superficial vascular imaging applications. However, this technique suffers from low SNR and thereby limited imaging depth. As a result, visual image quality of LED‐based PA imaging is not optimal, especially in sub‐surface vascular imaging applications.PurposeCombination of linear ultrasound (US) probes and LED arrays are the most common implementation in LED‐based PA imaging, which is currently being explored for different clinical imaging applications. Traditional delay‐and‐sum (DAS) is the most common beamforming algorithm in linear array‐based PA detection. Side‐lobes and reconstruction‐related artifacts make the DAS performance unsatisfactory and poor for a clinical‐implementation. In this work, we explored a new weighting‐based image processing technique for LED‐based PAs to yield improved image quality when compared to the traditional methods.MethodsWe are proposing a lag‐coherence factor (LCF), which is fundamentally based on the combination of the spatial auto‐correlation of the detected PA signals. In LCF, the numerator contains lag‐delay‐multiply‐and‐sum (DMAS) beamformer instead of a conventional DAS beamformer. A spatial auto‐correlation operation is performed between the detected US array signals before using DMAS beamformer. We evaluated the new method on both tissue‐mimicking phantom (2D) and human volunteer imaging (3D) data acquired using a commercial LED‐based PA imaging system.ResultsOur novel correlation‐based weighting technique showed LED‐based PA image quality improvement when it is combined with conventional DAS beamformer. Both phantom and human volunteer imaging results gave a direct confirmation that by introducing LCF, image quality was improved and this method could reduce side‐lobes and artifacts when compared to the DAS and coherence‐factor (CF) approaches. Signal‐to‐noise ratio, generalized contrast‐to‐noise ratio, contrast ratio and spatial resolution were evaluated and compared with conventional beamformers to assess the reconstruction performance in a quantitative way. Results show that our approach offered image quality enhancement with an average signal‐to‐noise ratio and spatial resolution improvement of around 20% and 25% respectively, when compared with conventional CF based DAS algorithm.ConclusionsOur results demonstrate that the proposed LCF based algorithm performs better than the conventional DAS and CF algorithms by improving signal‐to‐noise ratio and spatial resolution. Therefore, our new weighting technique could be a promising tool to improve the performance of LED‐based PA imaging and thus accelerate its clinical translation.

show abstract

Section: Discussionmentioning

confidence: 99%

Improvement of LED‐based photoacoustic imaging using lag‐coherence factor (LCF) beamforming

Paul,

Mulani,

Singh

et al. 2023

Medical Physics

View full text Add to dashboard Cite

show abstract

“…The goal of distinguishing point sources with high accuracy may benefit from photoacoustic imaging, where the absorption of optical or radio-frequency electromagnetic excitations cause tissue to generate acoustic waves [172]. Combining in silico and in vivo data for training, [173,174] explore the use of light-emitting diodes as excitation sources for photoacoustic visualization of clinical needles in MIS, developing a DNN-based system to enhance the visualization, achieving 4.3-times higher signal-to-noise ratio compared to conventional reconstructions. Their semi-synthetic approach allows for complete knowledge of the desired ground truth while reducing the sim-to-real gap.…”

Section: K-wavementioning

confidence: 99%

In silico simulation: a key enabling technology for next-generation intelligent surgical systems

et al. 2023

View full text Add to dashboard Cite

To mitigate the challenges of operating through narrow incisions under image guidance, there is a desire to develop intelligent systems that assist decision making and spatial reasoning in minimally invasive surgery (MIS). In this context, machine learning-based systems for interventional image analysis are receiving considerable attention because of their flexibility and the opportunity to provide immediate, informative feedback to clinicians. It is further believed that learning-based image analysis may eventually form the foundation for semior fully automated delivery of surgical treatments. A significant bottleneck in developing such systems is the availability of annotated images with sufficient variability to train generalizable models, particularly the most recently favored deep convolutional neural networks or transformer architectures. A popular alternative to acquiring and manually annotating data from the clinical practice is the simulation of these data from human-based models. Simulation has many advantages, including the avoidance of ethical issues, precisely controlled environments, and the scalability of data collection. Here, we survey recent work that relies on in silico training of learning-based MIS systems, in which data are generated via computational simulation. For each imaging modality, we review available simulation tools in terms of compute requirements, image quality, and usability, as well as their applications for training intelligent systems. We further discuss open challenges for simulation-based development of MIS systems, such as the need for integrated imaging and physical modeling for non-optical modalities, as well as generative patient models not dependent on underlying CT, MRI, or other patient data. In conclusion, as the capabilities of in silico training mature, with respect to sim-to-real transfer, computational efficiency, and degree of control, they are contributing toward the next generation of intelligent surgical systems.

show abstract

“…Compared with US imaging which limited needle visibility, PA imaging offers high-contrast needle visualization due to the strong laser absorption by metallic needle. Deep learning techniques can further enhance PA imaging's needle tracking capabilities (27). However, the use of nonclinical US transducers and immobile system design hinders clinical adoption.…”

Section: Introductionmentioning

confidence: 99%

“…However, the use of nonclinical US transducers and immobile system design hinders clinical adoption. Dual US/PA imaging can provide mutually beneficial information into tissue structure and precise identification of surgical devices like needle (27)(28)(29). While dual-mode US/PA systems are prevalent in research, FDA clearance for clinical use is limited, although a recent pre-market approval (PMA) was granted for an opto-acoustic ultrasound system in diagnostic breast cancer imaging (30).…”

Section: Introductionmentioning

confidence: 99%

Advancing ultrasound-guided needle visibility: deep learning empowered by photoacoustic imaging

Hui,

Rajendran,

Ling

et al. 2024

Photons Plus Ultrasound: Imaging and Sensing 2024

View full text Add to dashboard Cite

Needle insertion is a vital procedure in both clinical diagnosis and therapeutical treatment. To ensure the accurate placement of needle, ultrasound (US) imaging is generally used to guide the needle insertion. However, due to depthdependent attenuation and angular dependency, US imaging always face the challenge in consistently and precisely visualizing the needle, necessitating the development of reliable methods to track the needle. Deep learning, an advanced tool that has proven effective and efficient in addressing imaging challenges, has shown promise in enhancing needle visibility in US images. But the existing approaches often rely on manual annotation or simulated data as ground truth, leading to heavy human workload and bias or difficulties in generalizing to real US images. Recently, photoacoustic (PA) imaging has shown the capability of high-contrast needle visualization. In this study, we explore the potential of PA imaging as reliable ground truth for training deep learning networks, eliminating the need for expert annotation. Our network, trained on ex vivo image datasets, demonstrated the abilities of precise needle localization in US images. This research represents a significant advancement in the application of deep learning and PA imaging in clinical settings, with the potential to enhance the accuracy and safety of needle-based procedures.

show abstract

Improving needle visibility in LED-based photoacoustic imaging using deep learning with semi-synthetic datasets

Cited by 16 publications

References 60 publications

Improvement of LED‐based photoacoustic imaging using lag‐coherence factor (LCF) beamforming

Improvement of LED‐based photoacoustic imaging using lag‐coherence factor (LCF) beamforming

In silico simulation: a key enabling technology for next-generation intelligent surgical systems

Advancing ultrasound-guided needle visibility: deep learning empowered by photoacoustic imaging

Contact Info

Product

Resources

About