Figure 1. Rendered images using our proposed pipeline, providing high quality visual effects between virtual and real objects. Occlusion, reflections, refractions, soft shadows, custom shaders and lens effects are supported in real time.Abstract-This paper introduces a novel graphics rendering pipeline applied to augmented reality, based on a real time ray tracing paradigm. Ray tracing techniques process pixels independently from each other, allowing an easy integration with image-based tracking techniques, contrary to traditional projection-based rasterization graphics systems, e.g. OpenGL. Therefore, by associating our highly optimized ray tracer with an augmented reality framework, the proposed pipeline is capable to provide high quality rendering with real time interaction between virtual and real objects, such as occlusions, soft shadows, custom shaders, reflections and self-reflections, some of these features only available in our rendering pipeline. As proof of concept, we present a case study with the ARToolKitPlus library and the Microsoft Kinect hardware, both integrated in our pipeline. Furthermore, we show the performance and visual results in high definition of the novel pipeline on modern graphics cards, presenting occlusion and recursive reflection effects between virtual and real objects without the latter ones needing to be previously modeled when using Kinect. Furthermore, an adaptive soft shadow sampling algorithm for ray tracing is presented, generating high quality shadows in real time for most scenes.

In this work in progress we address the problem of interacting with augmented objects. A bare hand tracking technique is developed, which allied to gesture recognition heuristics, enables interaction with augmented objects in an intuitive way. The tracking algorithm uses a flock of features approach that tracks both hands in real time. The interaction occurs by the execution of grasp and release gestures. Physics simulation and photorealistic rendering are added to the pipeline. This way, the tool provides more coherent feedback in order to make the virtual objects look and respond more likely real ones. The pipeline was tested through specific tasks, designed to analyze its performance regarding the easiness of use, precision and response time.In this work we address the problem of direct interaction with the augmented content on a scene, which provides to the user natural means to manipulate virtual objects. In tabletop AR applications, the augmented objects are Extended

No abstract