A multi-user Virtual World has been implemented combining a flexible-object simulator with a multisensory user interface, including hand motion and gestures, speech input and output, sound output, and 3-D stereoscopic graphics with head-motion parallax, The implementation is based on a distributed clientherver architecture with a centralized Dialogue Manager. The simulator is inserted into the Virtual World as a server. A discipline for writing interaction dialogues provides a clear conceptual hierarchy and the encapsulation of state. This hierarchy facilitates the creation of alternative interaction scenarios and shared muhiuser environments.KEYWORDS: user interface management system, dialog manager, virtual worlds, virtual reality, interactive lation. BACKGROUND A Virtuat World is an interactive, multisensory, dimensional environment where human-computer

This paper presents a system architecture for creating interactive, multisensory, three-dimensional environments called virtual worlds. The architecture specifically addresses the requirements of virtual worlds for high performance, flexibility, and coordination of concurrent events. Performance is enhanced by a distributed client/server system structure and by efficient overlap of processing time and input/output delay. All processes communicate via asynchronous messages. The functional partitioning of a virtual world requires relatively low bandwidth among the individual processes and the system can be implemented over a conventional local-area network. A key element of this architecture is a central, event-driven dialogue manager that coordinates concurrent input and output events. The dialogue manager provides a clear separation of the interaction techniques from the content of the virtual world as defined by the application. The system is flexible and easily reconfigurable. An interaction technique can be readily changed or replaced because each interaction device is modularized into a separate server and each interaction modality into a separate subdialogue. Subdialogues can be loaded and dropped dynamically, enabling input/output device remapping and the selection of interaction techniques while a virtual world is running. As an initial test of this architecture we have implemented a virtual world for interacting with data from a computational fluid dynamics simulation.

No abstract

We describe a so~tware architecture for virtual worlds, built on a base of multiple proccsscs conl-

In recent years, a number of research group s have implemented various versions of virtual worl d concept [2, 4 . 6 . 7] . A common thread amon g these virtual worlds is a direct manipulation use r interface paradigm based on a glove device with th e position and orientation of the hand registered by a tracking device . To explore this paradigm, a ne w project at IBM Research was started in 1989 to buil d a virtual laboratory for scientists and engineers . Ou r first step is to integrate the glove and space trackin g devices with the real time graphics on a graphic s superworkstation . A simple bouncing ball virtua l world has been created to test underlying softwar e and fine tune interactive performance .Our initial emphasis is placed on understandin g the limitations of various system components an d getting the best interactive performance from th e system . With current state of technology . the glov e and tracking devices can generate much more dat a than the graphics update process can utilize . Bot h the rendering process and the processes handlin g the device serial ports are CPU intensive . Our firs t design problem is how to distribute the processin g and match the incoming data rates of input device s with the update rate of the graphics . After a ne w position from the tracker is received by the graphics, it is displayed only at the next frame updat e time giving the appearance that the hand image always lags behind the motion of the real hand . Ou r second design problem is to use techniques t o compensate for this inherent lag time . This abstract describes the specific approaches we use to solv e these problems and some useful insight gained i n Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association fo r Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission . © 1990 ACM 089791-351-5/90/0003/003951 .50 experimenting with lag time reduction by positio n prediction . System Overvie wOur virtual laboratory is built on commerciall y available components . We choose a two processo r version Ardent/Titan graphics supercomputer a s our graphics platform . [3] . With the Dore' rendering package [1], each processor is capable of rendering a maximum of 20 .000 smoothly shaded smal l polygons/seconds . A DataGlove TM (VPL Research ) device [8] senses finger gestures and a 3space TM IS OT RACK T`I (Polhemus Navigation Sciences ) tracker [5] is attached to the back of the glove t o track position . The glove and track devices are attached to two IBM PC-RT workstations respectivel y via RS-232c serial ports . The track server and glov e server software for the RTs are developed at IB M Research based on vendor supplied device specifications . Both servers and the graphics platform ar e all connected together via an ethern...