Semantic 3D object manipulation using object ontology in multimodal interaction framework

Abstract

This paper presents a multimodal interaction framework for semantic 3D object manipulation in the virtual reality. In our framework, interaction devices such as keyboard, mouse, joystick, tracker, can be combined with speech utterance to give a command to the system. We define an object ontology based on common sense knowledge which defines relationships between virtual objects. By taking into account the current user context and the object ontology, semantic integration component integrates the interpretation result from input manager, and then sends the result to the interaction manager. That result will be mapped into a proper object manipulation. Thus, the system can understand the user intention and assist him for achieving his goal in the handling process, instead of relying entirely on the user's control upon the interaction device and the object, avoiding nonsensical manipulations.

Interactive visual analytics of molecular data in immersive environments via a semantic definition of the content and the context

“…The semantic description of content through ontologies in immersive environments has been reported as a very efficient way to setup artificial intelligent systems [21], to design new VR applications [13] or to implement multimodal interactions [12]. Although these developments allowed for a very interesting usage of semantic descriptions in immersive environments, they aim for a generic definition of either the content or the context.…”

3D Interaction Assistance in Virtual Reality: A Semantic Reasoning Engine for Context-Awareness

“…As tasks and environments become more and more complex, these techniques can no longer be the same for every applications. A solution can be to adapt the interaction [5] to the needs and the context in order to improve usability, for example to: -choose other techniques ("specificity") or make techniques variations ("flavor") [18]; -add or manage modalities [12][4] [18]; -perform automatically parts of the task [7].…”

3D INTERACTION ASSISTANCE THROUGH CONTEXT-AWARENESS - A Semantic Reasoning Engine for Classic Virtual Environment

“…• Choose other techniques (Octavia et al, 2010) as the specificity idea; • Make techniques variations (Octavia et al, 2010) as the flavor idea; • Add/manage modalities (Irawati et al, 2005) (Bouyer et al, 2007) (Octavia et al, 2010); • Perform automatically parts of the task (Celentano and Nodari, 2004).…”