ARTEMIS: A Collaborative Mixed-Reality System for Immersive Surgical Telementoring

Opportunities and Challenges of Smartglass-Assisted Interactive Telementoring

“…Google Glass launched in 2013 and demonstrated a viable use of OST HMDs through various situated applications in information visualization and augmented reality using images and instructions. Since then, commercial smartglasses from Microsoft [13][14][15][16][17][18][19][20][21], Google [22,23], HTC [7,8], Vuzix [24], Epson [25,26] and Brother [27] have integrated a wearable display with a computationally sufficient computing unit that provides connectivity and a multi-modal user interface. For example, the latest OST HMDs such as HoloLens 2 and MagicLeap One provide high-resolution displays (i.e., up to 2048 × 1080), a wide FoV (i.e., 52 and 50 degrees), and tracking and control capabilities.…”

“…Kim et al [33] No No Nursing Education Klinker et al [13] No No Medical Kopetz et al [22] No No Nursing Education Rochlen et al [25] No No Medical Huang et al [27] No No Medical Aebersold et al [34] No No Medical Hanna et al [14] Yes Yes Medical Yoon et al [23] Yes Yes Medical Lacy et al [35] Yes Yes Medical Rio et al [26] Yes Yes Medical Greenberg et al [32] Yes Yes Medical Liu et al [15] Yes Yes Medical Meijer et al [36] Yes Yes Medical Andersen et al [37,38] Yes Yes Medical STAR [16][17][18][19] Yes Yes Medical Mitsuno et al [20] Yes Yes Medical Mentis et al [39] Yes Yes Medical, Industrial Lin et al [40] Yes Yes Medical Gasques et al [21] Yes Yes Medical Hassan et al [41] Yes Yes Medical Giuseppe et al [42] Yes Yes Industrial Semsar et al [31] Yes Yes Medical Wang et al [7] Yes Yes Industrial Wang et al [8] Yes Yes Physical Task…”

XRmas: Extended Reality Multi-Agency Spaces for a Magical Remote Christmas

“…Such systems (e.g., [16,17,26,27]) present a realtime panoramic view of the local space, but often do not provide depth for virtual objects to interact with. Another approach is to recreate the local space as a pre-created 3D model (e.g., [25]) or realtime captured point cloud or 3D model (e.g., [8,28,31,32]). This method can add benefit by providing depth for the VR user to view the space from different angles.…”

“…Manipulating virtual objects is also achieved by many AVT systems, including via spawning, moving, and deleting virtual objects (e.g., [28,29]) and annotating in the air (e.g., [8,28,31]). However, only a few systems enable smart virtual objects that understand the physical environment (e.g., [29]).…”

“…First, since AVT systems provide a remote-collaboration solution for activities that are conducted in a local space (e.g., remote surgery [8]), one of the main challenges is to increase the remote user's agency to manipulate and influence the local space. To fully explore such systems' advantage of letting the remote and local users feel they are in the same space, both local and remote users should be able to manipulate virtual and physical objects, and virtual objects should understand the physical environment instead of being isolated.…”