Collaborative virtual environment has been limited on static or rigid 3D models, due to the difficulties of real-time streaming of large amounts of data that is required to describe motions of 3D deformable models. Streaming shape deformations of complex 3D models arising from a remote user's manipulations is a challenging task. In this paper, we present a framework based on spectral transformation that encodes surface deformations in a frequency format to successfully meet the challenge, and demonstrate its use in a distributed virtual environment. Our research contributions through this framework include: i) we reduce the data size to be streamed for surface deformations since we stream only the transformed spectral coefficients and not the deformed model; ii) we propose a mapping method to allow models with multi-resolutions to have the same deformations simultaneously; iii) our streaming strategy can tolerate loss without the need for special handling of packet loss. Our system guarantees real-time transmission of shape deformations and ensures the smooth motions of 3D models. Moreover, we achieve very effective performance over real Internet conditions as well as a local LAN. Experimental results show that we get low distortion and small delays even when surface deformations of large and complicated 3D models are streamed over lossy networks.
KEYWORDS:Collaborative Interaction, Distributed Virtual Reality, 3D Shape Deformation.
INDEX TERMS: H.5.1 [Information Interfaces and Presentation]:Multimedia Information Systems -Artificial, augmented, and virtual realities.
INTRODUCTIONWith the rapid advances of networks, computer-assisted collaborative virtual environment has been profoundly involved into many aspects of our daily life. Traditional collaborative virtual environments mainly employ video, audio, or text to communicate, and extensive research has helped technologies to transmit these media to remote sites in real-time. Compared with traditional media, 3D models can represent objects in a more realistic and comprehensive manner. Therefore, they have received increasing attentions in collaborative virtual environments. However, the current state-of-the-art technologies cannot facilitate a real-time collaborative virtual environment where large and complicated 3D models and their related shape deformations are involved. Prior researchers have proposed algorithms to effectively transfer large and complex 3D models over network under different conditions, nevertheless these methods focus on static/rigid 3D models only. Some researchers generate 3D models' shape deformations offline, and stream them as animations. But these methods work off-line instead of real-time.Real-time streaming of 3D models' deformable motions faces two grand challenges, namely high computational cost and requirement of transmitting a large amount of data in real-time. For a deformable 3D model, the number of DOFs could be as large as the size of the 3D model. Therefore describing the shape deformations of a 3D deformable mod...