3D reconstruction is a useful tool for surgical planning and guidance. However, the lack of available medical data stunts research and development in this field, as supervised deep learning methods for accurate disparity estimation rely heavily on large datasets containing ground truth information. Alternative approaches to supervision have been explored, such as self-supervision, which can reduce or remove entirely the need for ground truth. However, no proposed alternatives have demonstrated performance capabilities close to what would be expected from a supervised setup. This work aims to alleviate this issue. In this paper, we investigate the learning of structured light projections to enhance the development of direct disparity estimation networks. We show for the first time that it is possible to accurately learn the projection of structured light on a scene, implicitly learning disparity. Secondly, we explore the use of a multi task learning (MTL) framework for the joint training of structured light and disparity. We present results which show that MTL with structured light improves disparity training; without increasing the number of model parameters. Our MTL setup outperformed the single task learning (STL) network in every validation test. Notably, in the medical generalisation test, the STL error was 1.4 times worse than that of the best MTL performance. The benefit of using MTL is emphasised when the training data is limited. A dataset containing stereoscopic images, disparity maps and structured light projections on medical phantoms and ex vivo tissue was created for evaluation together with virtual scenes. This dataset will be made publicly available in the future.

The main goal of surgical oncology is to achieve complete resection of cancerous tissue with minimal iatrogenic injury to the adjacent healthy structures. Brain tumour surgery is particularly demanding due to the eloquence of the tissue involved. There is evidence that increasing the extent of tumour resection substantially improves overall and progression-free survival. Realtime intraoperative tools which inform of residual disease are invaluable. Intraoperative Ultrasound (iUS) has been established as an efficient tool for tissue characterisation during brain tumour resection in neurosurgery [1]. The integration of iUS into the operating theatre is characterised by significant challenges related to the interpretation and quality of the US data. The capturing of high-quality US images requires substantial experi- ence and visuo-tactile skills during manual operation. Recently, robotically-controlled US scanning systems have been proposed (see e.g. [2]) but the scanning of brain tissue poses major challenges to robotic systems because of the safety-critical nature of the procedure, the very low and precise contact forces required, the narrow access space and the large variety of tissue properties (hard scull, soft brain structure). The aim of this paper is to introduce a robotic platform for autonomous iUS tissue scanning to optimise intraop- erative diagnosis and improve surgical resection during robot-assisted operations. To guide anatomy specific robotic scanning and generate a representation of the robot task space, fast and accurate techniques for the recovery of 3D morphological structures of the surgical cavity are developed. The prototypic DLR MIRO surgi- cal robotic arm [3] is used to control the applied force and the in-plane motion of the US transducer. A key application of the proposed platform is the scanning of brain tissue to guide tumour resection.

No abstract