Camera Selection for Occlusion-Less Surgery Recording via Training With an Egocentric Camera

Abstract

Recording surgery is an important technique for education and the evaluation of medical treatments. However, capturing targets such as the surgical field, surgical tools, and the surgeon's hands, is almost impossible since these targets are heavily occluded by the surgeon's head and body during a surgery. We used a recording system in which multiple cameras are installed in the surgical lump, supposing at least one camera would capture the target without occlusion. As this system records multiple video sequences, we address the task to select a best view camera automatically. Recently, learning-based approaches in a fully supervised manner have been proposed for this task, but these previous approaches completely rely on manual annotation of the training data. In this paper, we focus on the eye tracker mounted on the surgeon's head, which can capture the recording targets without occlusion. Employing this first-person-view video synchronized with multiple videos of the surgical lump, we propose a novel camera selection approach using a self-supervised learning framework. In experiments, we created a dataset composed of four different breast surgery. Our extended experiments showed that our approach successfully switched to the best camera view without manual annotation and achieved competitive accuracy compared with conventional supervised methods. Also, our approach yielded effective visual representations comparable to state-of-the-art selfsupervised learning frameworks.

Surgical Tool Detection in Open Surgery Videos

“…Hachiuma et al [39] improved this camera selection algorithm using CNNs. Saito et al [40] presented selfsupervised learning for camera selection by taking advantage of a first-person view. Yoshida et al [41] estimated the incision scenes in long-duration open surgery videos using learning gaze speed, hand movements, number of hands, and background movements in egocentric surgical videos.…”

Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test

“…An egocentric video provides detailed information on hand posture and movements and the object or environment in which the hand interacts. The advantage of egocentric vision is reduced user privacy concerns [15], the ability to monitor people when they are mobile [16], and finding attention during activities [17]. Currently, egocentric vision research has begun to be developed for hand rehabilitation [6].…”