Autonomous Robotic Point-of-Care Ultrasound Imaging for Monitoring of COVID-19–Induced Pulmonary Diseases

Abstract

The COVID-19 pandemic has emerged as a serious global health crisis, with the predominant morbidity and mortality linked to pulmonary involvement. Point-of-Care ultrasound (POCUS) scanning, becoming one of the primary determinative methods for its diagnosis and staging, requires, however, close contact of healthcare workers with patients, therefore increasing the risk of infection. This work thus proposes an autonomous robotic solution that enables POCUS scanning of COVID-19 patients’ lungs for diagnosis and staging. An algorithm was developed for approximating the optimal position of an ultrasound probe on a patient from prior CT scans to reach predefined lung infiltrates. In the absence of prior CT scans, a deep learning method was developed for predicting 3D landmark positions of a human ribcage given a torso surface model. The landmarks, combined with the surface model, are subsequently used for estimating optimal ultrasound probe position on the patient for imaging infiltrates. These algorithms, combined with a force–displacement profile collection methodology, enabled the system to successfully image all points of interest in a simulated experimental setup with an average accuracy of 20.6 ± 14.7 mm using prior CT scans, and 19.8 ± 16.9 mm using only ribcage landmark estimation. A study on a full torso ultrasound phantom showed that autonomously acquired ultrasound images were 100% interpretable when using force feedback with prior CT and 88% with landmark estimation, compared to 75 and 58% without force feedback, respectively. This demonstrates the preliminary feasibility of the system, and its potential for offering a solution to help mitigate the spread of COVID-19 in vulnerable environments.

Ultrasound-Based AI for COVID-19 Detection: A Comprehensive Review of Public and Private Lung Ultrasound Datasets and Studies

“…We also describe different types of AI models, used by state-of-the-art US COVID-19 studies in the following sections. [59] SqueezeNet, MobileNetV2 ✗ Al-Jumaili et al [68] ResNet-18, RestNet-50, NASNetMobile, GoogleNet, SVM Al-Zogbi et al [70] DenseNet ✗ Almeida et al [71] MobileNet ✗ Arntfield et al [38] Xception ✗ Awasthi et al [72] MiniCOVIDNet ✗ Azimi et al [73] InceptionV3, RNN ✗ Barros et al [69] Xception-LSTM ✗ Born et al [12] VGG-16 ✗ Born et al [74] VGG-16 ✗ Born et al [13] VGG-16 ✗ Carrer et al [16] Hidden Markov Model, Viterbi Algorithm, SVM ✗ Che et al [17] Multi-scale Residual CNN ✗ Chen et al [40] 2-layer NN, SVM, Decision tree Diaz-Escobar et al [67] InceptionV3, VGG-19, ResNet-50, Xception ✗ Dastider et al [18] Autoencoder-based Hybrid CNN-LSTM ✗ Durrani et al [35] Reg-STN ✗ Ebadi et al [52] Kinetics-I3D ✗ Frank et al [19] ResNet-18, MobileNetV2, DeepLabV3++ ✗ Gare et al [15] Reverse Transfer Learning on UNet ✗ Hou et al [75] Saab transform-based SSL, CNN ✗ Huang et al [41] Non-local channel attention ResNet ✗ Karar et al [53] MobileNet, ShuffleNet, MENet, MnasNet ✗ Karar et al [56] A semi-supervised GAN, a modified AC-GAN ✗ Karnes et al [54] Few-shot learning using MobileNet ✗ Khan et al [76] CNN ✗ La Salvia et al [42] ResNet-18, ResNet-50 ✗ Liu et al [48] Multi-symptom multi-label (MSML) network ✗ MacLean et al [77] COVID-Net US ✗ MacLean et al [78] ResNet ✗ Mento et al [44] STN, U-Net, DeepLabV3+ ✗ Muhammad and Hossain [58] CNN ✗ Nabalamba [49] VGG-16, VGG-19, ResNet ✗ Panicker et al [36] LUSNet (a U-Net like network for ultrasound...…”

“…where y true and y predict are the ground truth and predicted continuous values, respectively. Al-Zogbi et al [70] used this loss function to train their deep model to predict landmarks for optimal ultrasound scanning.…”

“…These features are then fed to an SVM classifier to classify the images into COVID-19, CAP, and healthy classes. A regression task was performed by Al-Zogbi et al [70], who employed DenseNet to approximate the position of the ultrasound probe in the desired scanning areas of the torso. Almeida et al [71] investigated a lightweight neural network, MobileNets, in the context of computer-aided diagnostics and classified ultrasound videos among abnormal, B-lines, mild B-lines, severe B-lines, consolidations, and pleural thickening classes.…”

Review of Machine Learning in Lung Ultrasound in COVID-19 Pandemic

“…Nowadays, lightweight, portable ultrasound and telehealth are widely explored during the COVID-19 pandemic. Some studies use AI-robotics to perform tele-examination of patients [ 113 , 114 ], including a telerobotic system to scan LUS on a COVID patient [ 115 ]. These robot-assisted systems can increase the distance between sonographers and patients, thus minimizing the transmission risk [ 116 ].…”

Ultrasound-Based AI for COVID-19 Detection: A Comprehensive Review of Public and Private Lung Ultrasound Datasets and Studies

“…These features are then fed to an SVM classifier to classify the images into COVID-19, CAP, and healthy classes. A regression task was performed byAl-Zogbi et al (2021), who employed DenseNet to approximate the position of the ultrasound probe in the desired scanning areas of the torso. Almeida et al (2020) investigated a lightweight neural network, MobileNets, in the context of computer-aided diagnostics and classified ultrasound videos among abnormal, B-lines, mild B-lines, severe Blines, consolidations, and pleural thickening classes Awasthi et al (2021).…”