Mobile ultrafast ultrasound imaging system based on smartphone and tablet devices

Hewener, Holger; Tretbar, Steffen

doi:10.1109/ultsym.2015.0520

Cited by 11 publications

(7 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It runs in real-time at 92 FPS on an NVIDIA P100 and uses 2000x fewer FLOPS than models such as MobileNetV2, which was designed for less capable hardware such as mobile phone CPUs. This runtime is relevant since more ultrasound systems are being developed for mobile phone viewing [38]. Additionally, the last row of Table III named "ver", is a 7.76M parameter model trained only on Verasonics Vantage data with MAE distance metric optimization while achieving similar metrics to training on the full dataset.…”

Section: Discussionmentioning

confidence: 99%

MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints

Huang

Long

Bottenus

et al. 2019

2019 IEEE International Ultrasonics Symposium (IUS)

View full text Add to dashboard Cite

Image post-processing is used in clinical-grade ultrasound scanners to improve image quality (e.g., reduce speckle noise and enhance contrast). These post-processing techniques vary across manufacturers and are generally kept proprietary, which presents a challenge for researchers looking to match current clinical-grade workflows. We introduce a deep learning framework, MimickNet, that transforms raw conventional delayand-summed (DAS) beams into the approximate post-processed images found on clinical-grade scanners. Training MimickNet only requires post-processed image samples from a scanner of interest without the need for explicit pairing to raw DAS data. This flexibility allows it to hypothetically approximate any manufacturer's post-processing without access to the preprocessed data. MimickNet generates images with an average similarity index measurement (SSIM) of 0.930±0.0892 on a 300 cineloop test set, and it generalizes to cardiac cineloops outside of our train-test distribution achieving an SSIM of 0.967±0.002. We also explore the theoretical SSIM achievable by evaluating MimickNet performance when trained under graybox constraints (i.e., when both pre-processed and post-processed images are available). To our knowledge, this is the first work to establish deep learning models that closely approximate current clinical-grade ultrasound post-processing under realistic blackbox constraints where before and after post-processing data is unavailable. MimickNet serves as a clinical post-processing baseline for future works in ultrasound image formation to compare against. To this end, we have made the MimickNet software open source.

show abstract

Section: Discussionmentioning

confidence: 99%

MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints

Huang

Long

Bottenus

et al. 2019

2019 IEEE International Ultrasonics Symposium (IUS)

View full text Add to dashboard Cite

show abstract

“…It runs in realtime at 142 FPS on a P100 (Nvidia, Santa Clara, USA), has been used to process DAS beams from a Verasonics C5-2v in real-time at 65 FPS, and has been shown to run on a Mate 10 (Huawei, Shenzhen, China) mobile phone at 8 FPS. This runtime is relevant since more ultrasound systems are being developed for mobile phone viewing [50]. Future work will investigate inverted residuals and linear bottlenecks [51] to increase computational speed on mobile devices.…”

Section: Mimicknet Mobile Runtimementioning

confidence: 99%

MimickNet, Mimicking Clinical Image Post- Processing Under Black-Box Constraints

Huang

Long

Bottenus

et al. 2020

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Image post-processing is used in clinical-grade ultrasound scanners to improve image quality (e.g., reduce speckle noise and enhance contrast). These post-processing techniques vary across manufacturers and are generally kept proprietary, which presents a challenge for researchers looking to match current clinical-grade workflows. We introduce a deep learning framework, MimickNet, that transforms conventional delay-and-summed (DAS) beams into the approximate Dynamic Tissue Contrast Enhanced (DTCE™) post-processed images found on Siemens clinicalgrade scanners. Training MimickNet only requires post-processed image samples from a scanner of interest without the need for explicit pairing to DAS data. This flexibility allows MimickNet to hypothetically approximate any manufacturer's post-processing without access to the preprocessed data. MimickNet post-processing achieves a 0.940±0.018 structural similarity index measurement (SSIM) compared to clinical-grade post-processing on a 400 cine-loop test set, 0.937±0.025 SSIM on a prospectively acquired dataset, and 0.928±0.003 SSIM on an out-ofdistribution cardiac cine-loop after gain adjustment. To our knowledge, this is the first work to establish deep learning models that closely approximate ultrasound post-processing found in current medical practice. MimickNet serves as a clinical post-processing baseline for future works in ultrasound image formation to compare against. Additionally, it can be used as a pretrained model for fine-tuning towards different post-processing techniques. To this end, we have made the MimickNet software, phantom data, and permitted in vivo data open-source at https://github.com/ ouwen/MimickNet.

show abstract

“…Previously, the reconstruction process is done on a desktop/laptop/workstation since they own powerful computational ability. However, mobile phones also possess high computation power with the upgrade every year, and they can be applied in many different fields, such as biomedical fields (20)(21)(22)(23)(24)(25)(26)(27)(28)(29). Bright-field microscopy (23) and fluorescent microscopy (24,25) can be combined with mobile phones to achieve portable imaging, sensing, and diagnostics, while real-time ultrasound image formation can be accomplished on iPhone or iPad by utilizing wireless transmission of raw data from the ultrasound transducer to the mobile platforms (20).…”

Section: Introductionmentioning

confidence: 99%

Photoacoustic image reconstruction on a mobile platform

Xie¹,

Rajendran²,

Zulkifli³

et al. 2023

Photons Plus Ultrasound: Imaging and Sensing 2023

View full text Add to dashboard Cite

Photoacoustic tomography (PAT) is a non-invasive imaging modality showing great potential in medical diagnosis and research due to its high optical contrast and high-resolution deep imaging. After laser irradiation on the tissue surface, energy absorption leads to the generation of acoustic waves (also known as PA waves), which can be collected by ultrasound detectors such as single-element ultrasound transducers (SUTs). A variety of image reconstruction algorithms can be employed to obtain the initial pressure distribution map. Previously, desktops or workstations are widely used for performing image-forming processes owing to their high computation power. But with the upgrade of mobile phones, they possess more and more powerful CPU or GPU, sometimes comparable to desktop computers. The capability of PAT can be further enhanced with the use of the mobile platform. In this work, we explored the usage of mobile platforms to reconstruct PAT images without sacrificing image quality. A mobile application was developed based on Python, implementing a simple delay-and-sum (DAS) beamformer for generating PAT images. HUAWEI P20 was employed to test the application performance, which spent less than 30 seconds to form a well-reconstructed PAT image with the SNR value more than 40 dB. Downsampling process can be performed, leading to much less reconstruction time while the photoacoustic target structure was still reconstructed properly, especially for two-fold downsampling operation. These results indicated that mobile platforms could support fast PAT image reconstruction and at the same time support good image quality.

show abstract

Mobile ultrafast ultrasound imaging system based on smartphone and tablet devices

Cited by 11 publications

References 4 publications

MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints

MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints

MimickNet, Mimicking Clinical Image Post- Processing Under Black-Box Constraints

Photoacoustic image reconstruction on a mobile platform

Contact Info

Product

Resources

About