A new approach to human-computer interaction---synchronous modelling in real and virtual spaces

Abstract

Three-dimensional computer-aided modelling of dynamic processes supported by virtual reality techniques like 3D-stereo vision does not reach the usability (ease, concreteness, intuitiveness, directness) we experience in modelling with real physical objects. We propose an interface that aims at coupling two previously separated model worldsthe the real space of physical objects and the virtual space of signs and images. The basic issues of this Real Reality concept are discussed and some applications are presented. Keywords real reality, data glove, graspable user interfaces, grasp recognition, modelling and simulation, programming by demonstration INTRODUCTIONSeveral projects with industrial partners as well as our own practice in the design of simulation models indicated that physical models play an important role for cognition and communication. They are used 8s prototypes for new products, design studies, and for the illustration of complex tasks and processes by making use of their medial qualities (Bruns, 1997;Brauer, 1996). Especially in heterogeneously qualified teams physical models allow to work in a very problem oriented way, without the need to concentrate on user interfaces and software functionality as it is the case with purely computer based tools. Nevertheless the advantages of abstract systems are their capabilities of quantitative analysis, modification, and automatic variation of symbolically represented virtual models. Therefore the idea arose to combine both previously separated model worlds the physical and the virtual one, thus preserving all their advantages.

Tangible User Interfaces: Past, Present, and Future Directions

“…The notion of control cubes has been popular in research particularly for end-user programming, where the faces of a cube usually serve to program, e.g., home automation or as a remote control for consumer electronics [21,28,62]. Tangible programming by demonstration was utilized to program event-based simulations of material flow on conveyor belts in plants [208]. Users can define simple rules (e.g., "if machine A is occupied and machine B is free then route pallets to B") by placing tags on a physical model and moving tokens.…”

Direct manipulation of user interfaces for migration

“…The migration is partial, presentation oriented, system initiated and the meta-UI is totally invisible. [22,23] do the same for a collaborative virtual environment. The Stanford Interactive Mural system [13] is similar except that wall screens could be hung up on different walls.…”

“…In most of research/development projects involving some form of migration [1,2,3,4,5,9,10,12,13,19,20,21,22,23,24], the meta-UI is implemented in very different ways with different manifestations. It is not made explicit whether the meta-UI is system initiated (the system initiates the migration), user-initiated (the user initiates the migration), or mixed-initiated (the user and the system collaborate to perform the migration).…”

Brick-Building Interface Support for Cocreative Communication

“…Before using this system, we calculated a parameter F b of rotation and translation of a virtual brick so that the relationship between position and posture of a physical hand and the physical brick, as a person grasps the physical brick, should agree with the relationship between position and posture of a virtual hand and the virtual brick. This method of representing bodily interactions with physical objects in a virtual space is also applied in the Real Reality Modeling Interface (Schäfer, Brauer, & Bruns, 1997). The interface system can recognize an interaction of grasping, transferring, and putting on an object located at the preregistered position by combining the data glove's open-closed condition with the position and posture of the hand.…”