Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness System

Abstract

In the wake of COVID-19, the digital fitness market combining health equipment and ICT technologies is experiencing unexpected high growth. A smart trampoline fitness system is a new representative home exercise equipment for muscle strengthening and rehabilitation exercises. Recognizing the motions of the user and evaluating user activity is critical for implementing its self-guided exercising system. This study aimed to estimate the three-dimensional positions of the user’s foot using deep learning-based image processing algorithms for footprint shadow images acquired from the system. The proposed system comprises a jumping fitness trampoline; an upward-looking camera with a wide-angle and fish-eye lens; and an embedded board to process deep learning algorithms. Compared with our previous approach, which suffered from a geometric calibration process, a camera calibration method for highly distorted images, and algorithmic sensitivity to environmental changes such as illumination conditions, the proposed deep learning algorithm utilizes end-to-end learning without calibration. The network is configured with a modified Fast-RCNN based on ResNet-50, where the region proposal network is modified to process location regression different from box regression. To verify the effectiveness and accuracy of the proposed algorithm, a series of experiments are performed using a prototype system with a robotic manipulator to handle a foot mockup. The three root mean square errors corresponding to X, Y, and Z directions were revealed to be 8.32, 15.14, and 4.05 mm, respectively. Thus, the system can be utilized for motion recognition and performance evaluation of jumping exercises.

Trampoline Stiffness Estimation by Using Robotic System for Quantitative Evaluation of Jumping Exercises

“…In previous research, we utilized a deep-learning-based image-processing algorithm to estimate the three-dimensional position of the user's feet on the trampoline using shadow images of the feet [15,16]. However, estimating the physiological indicators of trampoline exercise based on the shadow images of the user's feet proved challenging.…”

“…Sensor-based quantitative assessment systems for trampoline use have limitations, such as the need to attach sensors to the user’s body or install them on the trampoline. In previous research, we utilized a deep-learning-based image-processing algorithm to estimate the three-dimensional position of the user’s feet on the trampoline using shadow images of the feet [ 15 , 16 ]. However, estimating the physiological indicators of trampoline exercise based on the shadow images of the user’s feet proved challenging.…”

Absolute IOP/EOP Estimation Models without Initial Information of Various Smart City Sensors

“…Furthermore, some studies performed positioning estimation of various sensors using deep learning [ 28 , 29 , 30 , 31 ], but it is difficult to accept that it is close to the true value from a surveying standpoint.…”