A Portable Image Overlay Projection Device for Computer-Aided Open Liver Surgery

Abstract

Image overlay projection is a form of augmented reality that allows surgeons to view underlying anatomical structures directly on the patient surface. It improves intuitiveness of computer-aided surgery by removing the need for sight diversion between the patient and a display screen and has been reported to assist in 3-D understanding of anatomical structures and the identification of target and critical structures. Challenges in the development of image overlay technologies for surgery remain in the projection setup. Calibration, patient registration, view direction, and projection obstruction remain unsolved limitations to image overlay techniques. In this paper, we propose a novel, portable, and handheld-navigated image overlay device based on miniature laser projection technology that allows images of 3-D patient-specific models to be projected directly onto the organ surface intraoperatively without the need for intrusive hardware around the surgical site. The device can be integrated into a navigation system, thereby exploiting existing patient registration and model generation solutions. The position of the device is tracked by the navigation system's position sensor and used to project geometrically correct images from any position within the workspace of the navigation system. The projector was calibrated using modified camera calibration techniques and images for projection are rendered using a virtual camera defined by the projectors extrinsic parameters. Verification of the device's projection accuracy concluded a mean projection error of 1.3 mm. Visibility testing of the projection performed on pig liver tissue found the device suitable for the display of anatomical structures on the organ surface. The feasibility of use within the surgical workflow was assessed during open liver surgery. We show that the device could be quickly and unobtrusively deployed within the sterile environment.

Image overlay solution based on threshold detection for a compact near infrared fluorescence goggle system

“…This instrument is currently undergoing clinical trials and we have employed this procedure to ensure sterility of the instrument. As an alternative solution, the concept of a projection-based tracking pod used in surgical navigation systems 24 is expected to be explored in our future research.…”

An Analytic Solution to Projector Pose Estimation Problem

“…The proposed solution can be the theoretic basis of more practical applications [1], [2]. For example, an interesting future work will be planning multiple projectors to cover a complex indoor area.…”

“…It is a fundamental problem in spatial augmented reality [1] and projector-camera systems [2] in which the primary display device is a video projector.…”

Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies

“…As the image overlay device relies on the registration and tracking capabilities of the navigation system into which it is integrated, the accuracy of the device also depends on the accuracy of the navigation system. In a previous study [14], the image overlay device was found to have a mean projection accuracy of 1.3 mm when integrated into a Liver Surgical Navigation System [16] with navigation camera accuracy of approximately 0.3 mm. Tests were performed on both planar and 3D rigid models.…”

“…A portable projection image overlay device (IOD) [14] developed within our institute displays medical data directly on the patient and overcomes many of the deficiencies of previously described AR approaches. The device is minimally intrusive, non-impeding to the view of the surgeon, and efficient in set-up and movement handling.…”