WebGL-based streaming and presentation of objects with bidirectional texture functions

Abstract

Museums and Cultural Heritage institutions have a growing interest in presenting their collections to a broader community via the Internet. The photo-realistic presentation of interactively inspectable virtual surrogates is one of the most challenging problems in this field. For this purpose, we seek to employ not only a 3D geometry but also a powerful material representation capable of reproducing the full visual appeal of an object. In this paper, we propose a WebGL-based presentation framework in which reflectance information is represented via Bidirectional Texture Functions. Our approach works out-of-the-box in modern web browsers and allows for the progressive transmission and interactive rendering of digitized artifacts consisting of 3D geometry and reflectance information. We handle the huge amount of data needed for this representation by employing a novel progressive streaming approach for BTFs which allows for the smooth interactive inspection of a steadily improving version during the download. We demonstrate an interesting use-case of this technique at a cross-section of Cultural Heritage, medical education and research and provide an evaluation of the capabilities of our framework in the scope of BTF compression and transmission.

Animation on the web

“…One approach to reduce the vertex buffer switches, is to have all the mesh data in a single image (either a texture atlas or map) and create one VBO on the JavaScript layer and bind it with this single image geometry and have all the other data directly uploaded as textures in the GPU. This is similar to the technique used in [Behr et al 2012], [Schwartz et al 2011], [Englert et al 2014] and [Stein et al 2014]. This allows the GPU to store more mesh data, and calculate all the decoding and vertex coordination on the fly.…”

“…Similarly, Normal maps and displacement maps give the illusion of more structured details. For 3D data transmission, textures can be used by using the quantization threshold [Rodríguez et al 2013;Maglo et al 2012;Schwartz et al 2011] which represents a factor by which to quantize the geometry, which can be transmitted separately in a JSON file. Once the client side application knows of the factor, the decompression of the PNG takes place to retrieve the chunk of relevant data into a WebGL texture.…”

Extensions to bidirectional texture function compression with multi-level vector quantization in OpenCL

“…Silva et al [20] proposed an extension of SPMM using rasterization instead of ray tracing, where the fragment shader of GLSL in the modern OpenGL pipeline is used to achieve interactivity. Schwartz et al [21,22] presented two approaches for a BTF data driven compression scheme that provides the data in a hierarchical level-of-detail approach for real-time rendering and streaming the data over the network.…”

Interactive high fidelity visualization of complex materials on the GPU

“…The proposed approach achieves high compression ratios, supports mip-mapping and importance sampling for Monte Carlo based renderers, and the authors report rendering rates of up to 170 fps on the GPU for point based lighting. Schwartz et al [18] demonstrate that BTFs can be used for interactive progressive photorealistic visualization of cultural heritage artifacts over the web. They use SVD factorization together with a wavelet based compression and stream individual components sorted by order of perceptual relevance to a WebGL visualizer, being able to present high-quality previews of the BTF within a few seconds.…”