LumiPath – Towards Real-Time Physically-Based Rendering on Embedded Devices

Abstract

With the increasing computational power of today's workstations, real-time physically-based rendering is within reach, rapidly gaining attention across a variety of domains. These have expeditiously applied to medicine, where it is a powerful tool for intuitive 3D data visualization. Embedded devices such as optical see-through head-mounted displays (OST HMDs) have been a trend for medical augmented reality. However, leveraging the obvious benefits of physically-based rendering remains challenging on these devices because of limited computational power, memory usage, and power consumption. We navigate the compromise between device limitations and image quality to achieve reasonable rendering results by introducing a novel light field that can be sampled in real-time on embedded devices. We demonstrate its applications in medicine and discuss limitations of the proposed method. An open-source version of this project is available at https://github.com/lorafib/LumiPath which provides full insight on implementation and exemplary demonstrational material.

VET: Visual Error Tomography for Point Cloud Completion and High-Quality Neural Rendering

“…To better represent the environment, previous methods usually use environment maps Zhang et al 2020], recently in the form of multi-sphere images [Fridovich-Keil et al 2022]. In the same spirit, we spawn points on four layers of nested spherical environment maps using spherical Fibonacci sampling [Fink et al 2019;Keinert et al 2015] of 0.5M points. These points are optimized and cleaned identically to the original scene points.…”