Joint Position Estimation for Body Pressure Images during Sleep: An Extension for CPM Using Body Area and Posture Estimation Mashups

Abstract

Estimating sleeping postures with body joint positions is critical for identifying potential sleeping problems and the risk of pressure ulcers. Many methods have estimated postures with body joint positions from camera images for general purposes. However, visual monitoring of sleeping contexts suffers from privacy and occlusion issues due to blankets, pillows, etc. An approach to solve those issues is the use of body pressure images obtained from bed surfaces. We have developed a textile-based sheet-type pressure sensor to avoid such issues. Unfortunately, its use raises other issues that are absent from camera images such as low resolution and noise caused by the wrinkling of sensor sheets. In this paper, we extend DNN-based joint estimation, called Convolutional Pose Machine (CPM), using body area and posture estimation mashups to improve the accuracy of joint estimation. The following are our evaluation results with cross-validation with 16 joints in six sleeping postures of 12 subjects: 7.15 cm accuracy in mean absolute error (MAE), which is a 33.7% improvement from the standard CPM, and 8.52 cm accuracy in MAE, which is a 37.4% improvement from CPM with camera images in situations using a pillow and a blanket.