A voice-based annotation system for collaborative computer-aided design

Abstract

In this paper, we describe a voice-based interaction mechanism to annotate 3D models directly from a computer-aided design (CAD) modeling environment. The audio signal is captured and automatically transcribed to a textual 3D note, which is attached to the geometry and made available to other product information and business processes across the enterprise via a product data management system. Our approach provides a more natural and intuitive method to capture design and engineering knowledge that is particularly effective when large amounts of information need to be communicated. We discuss the rationale of the software architecture and the value of this modality for capturing knowledge in a collaborative engineering context. Finally, we examine the results of an experiment to validate our proposal. Our results show that 3D annotations are an effective mechanism to communicate design knowledge, which suggests the need for further developments in the areas of multimodal interaction methods and interfaces for CAD and collaborative tools.

Multimodal Natural Human–Computer Interfaces for Computer-Aided Design: A Review Paper

“…This frees users from the need to memorize a host of predefined voice commands [24]. Whereas it is plain to see how this would work for simple commands, recognition methods should be further advanced for a speech-based CAD system to deal flexibly and intelligently with complex verbal commands [24,55].…”