Follow the Curve: Robotic-Ultrasound Navigation with Learning Based Localization of Spinous Processes for Scoliosis Assessment

Abstract

The scoliosis progression in adolescents requires close monitoring to timely take treatment measures. Ultrasound imaging is a radiation-free and low-cost alternative in scoliosis assessment to X-rays, which are typically used in clinical practice. However, ultrasound images are prone to speckle noises, making it challenging for sonographers to detect bony features and follow the spine's curvature. This paper introduces a robotic-ultrasound approach for spinal curvature tracking and automatic navigation. A fully connected network with deconvolutional heads is developed to locate the spinous process efficiently with real-time ultrasound images. We use this machine learning-based method to guide the motion of the robot-held ultrasound probe and follow the spinal curvature while capturing ultrasound images and correspondent position. We developed a new force-driven controller that automatically adjusts the probe's pose relative to the skin surface to ensure a good acoustic coupling between the probe and skin. After the scanning, the acquired data is used to reconstruct the coronal spinal image, where the deformity of the scoliosis spine can be assessed and measured. To evaluate the performance of our methodology, we conducted an experimental study with human subjects where the deviations from the image center during the robotized procedure are compared to that obtained from manual scanning. The angles of spinal deformity measured on spinal reconstruction images were similar for both methods, implying that they equally reflect human anatomy.

Ultrasound-Guided Assistive Robots for Scoliosis Assessment with Optimization-based Control and Variable Impedance

“…To this end, we previously developed a novel fully connected network built upon ResNet that is suitable for spinous localization. The proposed in [18] neural network takes as input the raw ultrasound image and outputs a spatial heatmap that indicates the location of the spinous process as well as the confidence probability. The distance between the detected spinous location and the image center is then used to guide the movement of x-direction of the body frame such that the ultrasound probe will move in a way that the vertebrae is always kept in the center of the ultrasound image, as shown in Fig.…”

“…For the probe's orientation control, the desired rotation matrix is determined such that the probe is normally pointing towards the human back surface. The reference force profile is empirically set based on the subject's body mass index to ensure tight contact with skin [18].…”

“…Previously, we devised a novel algorithm for trajectory planning based on spinous process localization in ultrasound images [18]. Although the preliminary results have shown that a robotic approach is promising for spinal image reconstruction, there is a lack of a principled control theoretic framework for prioritizing different tasks, such as contact force, end-effector pose, robot configuration, etc, under external contact constraints.…”

Reliability of Robotic Ultrasound Scanning for Scoliosis Assessment in Comparison with Manual Scanning

“…The robot adjusts the ultrasound probe rotation according the force sensing and smoothly scans through the patient's back, maintaining constant contact. The robot uses a machine-learning algorithm to follow the spinal curvature and detect the spinal region for rotation adjustments, described in earlier stage of the current work [16]. The fully connected network model was designed to detect the spinous process location to indicate the spine center at each ultrasound B-mode frame in real-time.…”

“…The human scanning was performed for 120 seconds capturing 150 ultrasound frames per each. The robotic ultrasound system used in this study (Figure 1b) was designed for scoliosis assessment and was initially presented in [12] and extended in [16]. It uses a DL-based navigation control to detect the center of the spine, which is defined as spinous process tip, in real-time at each B-mode ultrasound frame; thus, there is no need for additional external sensors, such as a camera.…”