Point-based rendering enhancement via deep learning

Bui, Giang; Le, Tu; Morago, Brittany; Duan, Ye

doi:10.1007/s00371-018-1550-6

Cited by 30 publications

(20 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While 3D modelling approaches solely based on spatial attributes can leverage both the At the point level (lowest level), the geometry is sparse and defines the lowest possible geometric description. While this is convenient for point-based rendering [108] that can be enhanced through deep learning [109], leading to simpler and more efficient algorithms [110], geometric clustering covers applications identified in Section 1. At the patch level, subsets of points are grouped to form small spatial conglomerates.…”

Section: Knowledge-base Structurationmentioning

confidence: 99%

3D Point Cloud Semantic Modelling: Integrated Framework for Indoor Spaces and Furniture

et al. 2018

View full text Add to dashboard Cite

3D models derived from point clouds are useful in various shapes to optimize the trade-off between precision and geometric complexity. They are defined at different granularity levels according to each indoor situation. In this article, we present an integrated 3D semantic reconstruction framework that leverages segmented point cloud data and domain ontologies. Our approach follows a part-to-whole conception which models a point cloud in parametric elements usable per instance and aggregated to obtain a global 3D model. We first extract analytic features, object relationships and contextual information to permit better object characterization. Then, we propose a multi-representation modelling mechanism augmented by automatic recognition and fitting from the 3D library ModelNet10 to provide the best candidates for several 3D scans of furniture. Finally, we combine every element to obtain a consistent indoor hybrid 3D model. The method allows a wide range of applications from interior navigation to virtual stores.

show abstract

Section: Knowledge-base Structurationmentioning

confidence: 99%

3D Point Cloud Semantic Modelling: Integrated Framework for Indoor Spaces and Furniture

et al. 2018

View full text Add to dashboard Cite

show abstract

“…We would contend that it is relatively low, given the significant computing power required to achieve ultra-high frame rate. Despite this, rapid progression in graphics technology (e.g., deep-learning supported 3D graphics processing; Bui et al, 2018;Nalbach et al, 2017) means the capability for high-resolution, high frame rate rendering will soon be widely available, such that even small enhancements in vection will be easily attainable. However, the lack of improvement in subjective vection for high FPS displays suggests that values of around 15-45 FPS, around the economical frame rates of our study and others (Fujii et al, 2018), are optimal for inducing compelling self-motion, at least for the impoverished stimuli evaluated in these experiments.…”

Section: Discussionmentioning

confidence: 99%

Limits of subjective and objective vection for ultra-high frame rate visual displays

et al. 2020

View full text Add to dashboard Cite

Large-field optic flow generates the illusory percept of self-motion, termed 'vection'. Smoother visual motion displays generate a more compelling subjective sense of vection and objective postural responses, as well as a greater sense of immersiveness for the user observing the visual display. Research suggests that the function linking frame rate and vection asymptotes at 60 frames per second (FPS), but previous studies have used only moderate frame rates that do not approach the limits of human perception. Here, we measure vection using subjective and objective (mean frequency and path length of postural centre-ofpressure (COP) excursions) responses following the presentation of high-contrast optic flow stimuli at slow and fast speeds and low and ultra-high frame rates. We achieve this using a novel rendering method implemented with a projector capable of sub-millisecond temporal resolution in order to simulate refresh rates ranging from very low (15 FPS) to ultra-high frame rates (480 FPS). The results suggest that subjective vection was experienced most strongly at 60 FPS. Below and above 60 FPS, subjective vection is generally weaker, shorter, and starts later, although this pattern varied slightly according to the speed of stimuli. For objective measures, while the frequency of postural sway was unaffected by frame rate, COP path length was greatest for 480 FPS stimuli. Together, our results support diminishing returns for vection above 60 FPS and provide insight into the use of high frame rate for enhancing the user experience in visual displays.

show abstract

“…A neural network solution for upsampling and filling a splatted point cloud was published in [106]. Instead of using a rendering network to learn a mapping from a point cloud to final image, it used a GAN-based network to learn a post-processing step from an incomplete and low-frequency splatted rendering to a high quality final frame.…”

Section: Point-based Neural Renderingmentioning

confidence: 99%

Real-Time Rendering of Point Clouds With Photorealistic Effects: A Survey

et al. 2022

View full text Add to dashboard Cite

Readily available RGB-D cameras in smart phones and improving 3D scanning technologies have made it possible to produce detailed point cloud and point-based models of real world objects even in real time. Rendering such models in high quality and at satisfactory frame rates is needed for realistic extended reality (XR) applications. This publication reviews real-time photorealistic point cloud rendering methods which directly ray trace or rasterize point cloud models, with an emphasis on ray tracing and realtime performance. We found that real-time direct point cloud ray tracing research has been focused on static non-animated content, and thus, open research possibilities include adapting modern dedicated ray tracing hardware for increased performance for animated and live captured scenes, and adding path tracing techniques to increase photorealistic effects in the rendering result. A categorization and discussion on the capabilities of state-of-the-art photorealistic point cloud rendering methods is presented by surveying both real-time and offline methods, which are assumed to become real-time capable with the advances in near-future hardware. Challenges and future trends are derived by comparing different rasterization and ray tracing methods as well as acceleration structures for point clouds in terms of produced rendering effects and speed.

show abstract

Point-based rendering enhancement via deep learning

Cited by 30 publications

References 34 publications

3D Point Cloud Semantic Modelling: Integrated Framework for Indoor Spaces and Furniture

3D Point Cloud Semantic Modelling: Integrated Framework for Indoor Spaces and Furniture

Limits of subjective and objective vection for ultra-high frame rate visual displays

Real-Time Rendering of Point Clouds With Photorealistic Effects: A Survey

Contact Info

Product

Resources

About