Monocular Vision- and IMU-Based System for Prosthesis Pose Estimation During Total Hip Replacement Surgery

Abstract

The average age of population increases worldwide, so does the number of total hip replacement surgeries. Total hip replacement, however, often involves a risk of dislocation and prosthetic impingement. To minimize the risk after surgery, we propose an instrumented hip prosthesis that estimates the relative pose between prostheses intraoperatively and ensures the placement of prostheses within a safe zone. We create a model of the hip prosthesis as a ball and socket joint, which has four degrees of freedom (DOFs), including 3-DOF rotation and 1-DOF translation. We mount a camera and an inertial measurement unit (IMU) inside the hollow ball, or "femoral head prosthesis," while printing customized patterns on the internal surface of the socket, or "acetabular cup." Since the sensors were rigidly fixed to the femoral head prosthesis, measuring its motions poses a sensor ego-motion estimation problem. By matching feature points in images of the reference patterns, we propose a monocular vision based method with a relative error of less than 7% in the 3-DOF rotation and 8% in the 1-DOF translation. Further, to reduce system power consumption, we apply the IMU with its data fused by an extended Kalman filter to replace the camera in the 3-DOF rotation estimation, which yields a less than 4.8% relative error and a 21.6% decrease in power consumption. Experimental results show that the best approach to prosthesis pose estimation is a combination of monocular vision-based translation estimation and IMU-based rotation estimation, and we have verified the feasibility and validity of this system in prosthesis pose estimation.

CMOS Image Sensor Design and Image Processing Algorithm Implementation for Total Hip Arthroplasty Surgery

“…7, the image sensor measures around 3.5 mm × 3.5 mm with pad frame, each unit pixel size measures around 15 µm × 15 µm big enough to detect low light signal variations [17]. Camera used in [13] is too bulky and it is hard to mount onto the femoral head and integrate with the other modules consuming most of the system power. The sensor size proposed in this paper is more compact and is designed to be integrated with an on-chip image processing ASIC for the SoC development.…”

“…Later, to deal with the problems in mounting enough contact sensors onto the femoral head prosthesis. We proposed a design of monocular vision and IMU-based visual aided system [13], the system consists of a miniature camera inside the femoral head prosthesis from Omni Vision (OV7660) to capture images inside the acetabular cup, and an inertial measurement unit IMU for more precise pose estimation. The commercial camera module used in our previous works has a package dimension of 4.2 mm × 3.9 mm with an optical size 1/5 inch and active power consumption by the module is 40 mW without loading.…”

“…An efficient image-processing algorithm is developed to detect the features and coordinates of each pattern from the blood-covered images. The method is compared with the top-hat algorithm [13]. Finally, the algorithms are implemented on FPGA for the ASIC development to provide an on-chip image processing.…”

State-of-the-Art Mobile Intelligence: Enabling Robots to Move Like Humans by Estimating Mobility with Artificial Intelligence

“…Sensors [18,49,50] GPS, IMU [19,51,52] Camera, IMU [36,53] Wi-Fi, IMU [54] GPS, Digital Compass, IMU [37,55,56] UWB, IMU [57,58] RFID, IMU…”

“…This facilitates acceleration of the localization algorithms, either by providing an initial guess of the attitude, or in closed form as direct contribution to the localization. The first part, shown by black arrows in the figure, depicts that the attitude extracted from the IMU helps the camera catch the more accurate feature points, which can result in the higher performance of the trajectory estimation [51,52]. The methods of feature detection include Integral Channel Image Patches (ICIMGP) [69], Scale Invariant Feature Transform (SIFT) [70], Features from Accelerated Segment Test (FAST) [71] and Speed Up Robust Features (SURF) [72].…”