MIVAS: A Multi-Modal Immersive Virtual Assembly System

Abstract

Evaluation and planning of assembly processes in virtual environments have become an active research area in engineering community. However, planning of complex assemblies in virtual environments, especially large-scale virtual environments, is still hindered by limitations like unnatural user interaction, insufficient frame rates, and deficiencies in processing of assembly constraints. In this paper, we present MIVAS, a Multi-modal Immersive Virtual Assembly System. By viewing the virtual assembly system as a finite state machine, we incorporate tracked devices, force feedback dataglove, voice commands, human sounds, fully immersive 4-sided CAVE, together with optimization techniques for both complex assembly models and assembly operations to provide for engineers an intuitive and natural way of assembly evaluation and planning. Testing scenarios on disassembling different components of an intelligent hydraulic excavator are described. Special attention is paid upon such technical issues as interface between CAD packages and the CAVE virtual environment, natural and intuitive user interaction including realistic virtual hand interaction and force feedback, intelligent navigation for assembly operations, and real-time display of complex assemblies.

A new methodology to evaluate the performance of physics simulation engines in haptic virtual assembly

“…Immersive environment; realistic hand interaction; documentation of assembly plans (Wan et al, 2004) Cybergrasp SHARP Portable system; network module for collaborative assembly; subassemblies capability (Seth et al, 2006) Dual Phantom desktop HAMMS Test bed to investigate user interactions and response while performing various engineering tasks, physics-based (Ritchie et al, 2008a, b) Single Phantom Omni HIVEx Designed for training; imitates real scenarios; pleasurable to use; HMD visualization (Bhatti et al, 2009) Phantom Omni and data glove VEDAP-II Multiple DOF force feedback; focused on modelling the dynamic behaviour of parts Cybergrasp MRA Mixed reality application for the assessment of manual assembly; double hand assembly; demonstrates assembly procedures (Bordegoni et al, 2009) Virtuose 6D35-45 and CyberGlove…”

Virtual reality for assembly methods prototyping: a review

“…In addition, when combined with accurate models (e.g., parametric surface representations, or B-Rep solids) constraint-based approaches allow users to manipulate and position parts in an assembly with very high fidelity. However, some of these applications required special CAD toolkits to extract relevant CAD metadata ( Figure 10) that was required for preparing an assembly scenario [65,74,123]. These special data requirements and their dependence on specific CAD systems prevented widespread acceptance of these applications.…”

“…The resulting lack of part shape accuracy of such approaches presents problems when detailed contact information is necessary. Simulating complex part interactions such as grasping is also demanding as it requires the simulation to detect collisions and generate contact forces accurately for each individual finger [74,75,87,115]. Maintaining update rates for haptic interaction (~1 KHz) while performing highly accurate collision/physics computations in complex interactive simulations such assembly remains a challenge for the community.…”

“…Various approaches for providing haptic feedback for assembly have been presented in the past which focused on developing new methods for providing tactile [58,65,74,87,113], collision [100][101][102] and gravitational force feedback [108,114]. The high update rate (~1KHz) requirement for effective haptics has always been a challenge in integrating this technology.…”

Training of Procedural Tasks Through the Use of Virtual Reality and Direct Aids

“…• MIVAS: A Multi-modal Immersive Virtual Assembly System (Wan et al (2004)) MIVAS is a multi-modal immersive virtual assembly system developed at the State Key Lab of CAD&CG, Zhejiang University. This system provides an intuitive and natural way of assembly evaluation and planning using tracked devices for tracking both hands and head motions, dataglove with force feedback for simulating the realistic operation of human hand and providing force feedback, voice input for issuing commands, sound feedback for prompts, fully immersive 4-sided CAVE as working space.…”