We introduce Reality-based User Interface System (RUIS), a platform that even novice programmers can easily use for prototyping novel user interfaces. We implemented a puzzle application with a 3D user interface employing modified Nintendo Wii Remotes as 6-DOF controllers to showcase some of the features that are possible with our platform: two-handed gestural interaction, natural interaction metaphors, and haptic feedback. We conducted a basic user study with 16 participants to evaluate the puzzle application and its 3D user interface. The results show how test subjects using the puzzle application tended to rate the 3D user interface to be more fun, intuitive, and realistic than a traditional mouse interface.
INTRODUCTION3D user interfaces, gestural interfaces, and other novel interaction styles used to be available only for experimental UI researchers confined in their laboratories. This has changed with the advent of modern game controllers like Nintendo Wii Remote (Wiimote) that have been embraced by consumers and researchers alike [1,3, 5].Recently released Microsoft Kinect has created a wave of excitement in research and hacker communities that are eager to explore the possibilities of new hardware [2]. We believe that amateur software developers have considerable potential for not only developing useful applications, but also making major contributions in the way we interact with computers. Currently however, the number of hobbyists using novel interaction devices is restricted to tech-savvy computer users who know how to utilize hacked drivers and programming libraries.We have developed Reality-based User Interface System (RUIS) to ease the adoption of new interaction technology. In this paper we briefly introduce RUIS and show how it can be used for prototyping novel user interfaces by presenting an example of a 3D puzzle application running in a virtual reality environment. We also present results of a basic user study that was conducted to evaluate the application and its 3D user interface.
SYSTEMRUIS runs on top of Processing, a development environment that is widely used by artists and designers among others. The idea of RUIS is to present an easy programming interface for Processing users to take advantage of novel interaction devices like Wiimote.
2.1Hardware Currently we run RUIS applications in a CAVE-like setup with two 3-by-3 meter stereographic walls perpendicular to each other [4]. For head tracking, we use two cameras placed above the walls and a color LED attached to stereographic glasses worn by user.Main interaction devices in RUIS are two Wiimotes that have been modified to work as 6-DOF controllers. Both Wiimotes have a MotionPlus extension that includes additional gyros for accurately tracking orientation along three axes. Each Wiimote is equipped with a color LED whose 3D location is tracked with the aforementioned camera-based head tracking system. Simple haptic feedback is achieved via Wiimotes' vibration motors.RUIS applications can also run on a desktop setup using just a mouse ...