Stereoscopic visualization of laparoscope image using depth information from 3D model

SLAM-based dense surface reconstruction in monocular Minimally Invasive Surgery and its application to Augmented Reality

“…Using stereoscopic views is a possibility and we will investigate this in future work. One possible solution to accurately simulate and track the deformation is to to use real-time deformation model [54] and feature-based tracking [22] to recovery the movement of tissue. Although the accuracy and speed of our framework are acceptable, we will continue developing a dense SLAM system to be used in MIS reconstruction and extend the current reconstruction framework.…”

“…The luminance changes dramatically and an endoscope can move rapidly during insertion and extrusion. Traditional tracking methods for AR in MIS usually involve feature points based tracking such as Scale-Invariant Feature Transform (SIFT) [18], Speeded Up Robust Features (SURF) [22], Optical Flow tracking [38] or other approaches specifically designed to work with soft tissues that account for changes in scale, rotation and brightness [31]. As these invariant descriptors are designed for 2D tracking, the information regarding the depth within a scene has not been recovered and selected feature points extracted from vision algorithms must be within the field of view, resulting in the lack of global information in AR augmentations.…”

Interactive Instrument-Driven Image Display in Laparoscopic Surgery

“…1 These include threedimensional, high-definition, and digital imaging. 11,12 There have even been attempts at manipulating the image as a whole, such as mirror-image and augmented reality (superimposition of preacquired data), or the monitors themselves (optimal position and viewing angle of the display). 13,14 However, all members of the surgical team have always had to rely on the same captured image-unlike conventional open surgery, whereby all team members determine their own point of view.…”

Real‐time geometry‐aware augmented reality in minimally invasive surgery

“…The traditional MIS AR approaches usually employ feature points tracking methods for information overlay. Feature-based 2D tracking methods such as Kanade–Lucas–Tomasi features [ 6 , 7 ], scale-invariant feature transform (SIFT) [ 1 ], speeded up robust features (SURF) [ 15 ], even those methods specifically designed to cater for the scale, rotation and brightness of soft tissue [ 8 ] have a major drawback for AR, because selected feature points extracted from vision algorithms must be within the FOV. Therefore, traditional feature tracking methods can severely affect the precision of procedure guidance, especially in surgical scenes where the accuracy is paramount.…”