A Distributed Framework for Relaying Stereo Vision for Telerobotics

Abstract

Stereo vision is one critical tool in minimally invasive surgery (MIS) for enhancing perception of depth of organs which greatly improves the operation efficiency. Augmented stereo views results from superimposing 3D anatomical models with real organ views. A reliable distributed framework for relaying stereoscopic visual feedbacks between a telerobotic server and a client station is proposed. The distributed components are based on DirectX, Visual C#, and Window sockets. We used a multi-threaded execution to promote concurrency in grabbing, transmitting, receiving, processing, and displaying image data using head-mounted display (HMD) technology. The client station provides components that support augmented reality (AR), i.e. superimposing animated graphic model with real views from the operating site. Design and performance issues of proposed multi-threaded execution for streaming of stereo data in a distributed and modular framework is presented.

Human Machine Interface with Augmented Reality for the Network Based Mobile Robot

“…Display device such as HMD (head mounted display) is convenient to use as interface with many system because it can easily build virtual or augmented reality environment. Many researchers have been studies the possibility of HMD device as human-machine interface [1] [7]. In Fig 1, by displaying all information through HMD, user can interact with the mobile robot more easily and intelligently.…”