Sensor fusion approaches for observation of user actions in programming by demonstration

Ehrenmann, M.; Zöllner, R.; Knoop, Steffen; Dillmann, Rüdiger

doi:10.1109/mfi.2001.1013539

Cited by 11 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For a complete review we refer to Wimmer [24] who describes different aspects of grasp sensing and a variety of sensing technologies that can be employed. We built onto work by Ehrenmann et al [5] who were among the first to use a pressure sensing data glove and a camerasystem to track hand posture, position and applied forces while users are manipulating objects. Since then, pressure sensing and motion capture approaches have been used to build more detailed models of how humans interact with objects (Saidon et al [18]) including the creation of large gesture databases with a variety of manipulation tasks involving objects ranging from kitchen utensils, to tools, mugs, jars, and toys (Verdier et al [22]).…”

Section: Grasp Sensing For Interactive Systemsmentioning

confidence: 99%

Integrating Electronic Components into Deformable Objects Based on User Interaction Data

Worgan

Reuss

Mueller

2019

Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction

View full text Add to dashboard Cite

A key challenge when designing deformable user interfaces is the integration of rigid electronic components with the soft deformable device. In this paper, we propose to place electronic components based on how the user is interacting with the device, i.e., in which way the device is being deformed when the user performs gestures. To identify optimum locations for placing electronic components, we developed a design tool that takes as input a 3D model of the deformable device and a set of captured user gestures. It then visualizes the stress distribution resulting from the gestures applied to the deformable device and suggests where not to place components because the location is highly deformed when users interact (e.g., a rigid battery that would constraint interaction); or alternatively where to place components to sense deformation more accurately (e.g., a bend sensor to detect a specific gesture) and efficiently (e.g., an energy harvesting component). We evaluated our approach by collecting interaction data from 12 users across three deformable devices (a watch, a camera, and a mouse) and applied the resulting stress distributions to the placement of selected electronic components.

show abstract

Section: Grasp Sensing For Interactive Systemsmentioning

confidence: 99%

Integrating Electronic Components into Deformable Objects Based on User Interaction Data

Worgan

Reuss

Mueller

2019

Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction

View full text Add to dashboard Cite

show abstract

“…Several articles by Ikeuchi and his coworkers [60,61,62,85,86], work by Tung and Kak [129] and contributions by Dillmann and colleagues [31,47] extend the use of vision in assembly from quality and execution control to process understanding and learning by demonstration. They all introduce systems that analyze assembly tasks demonstrated by a human and derive corresponding plans and manipulator trajectories.…”

Section: Assembly Modeling Within the Sfb 360mentioning

confidence: 99%

“…1.3(a) on page 7), 2D vision is sufficient to estimate their position and orientation. Dillmann and his coworkers [31,47] also combine different sensing strategies in their implementations of skill acquiring systems. Like Tung and Kak they process data provided by a data glove and a camera.…”

Section: Assembly Modeling Within the Sfb 360mentioning

confidence: 99%

A Structural Framework for Assembly Modeling and Recognition

Bauckhage

Kummert

Sagerer

2003

Computer Analysis of Images and Patterns

View full text Add to dashboard Cite

“…Previous research has integrated two vision based systems for the purpose of high fidelity hand motion data acquisition [ 20 ]. Furthermore, various studies have integrated vision and contact based systems with the aim of aiding in the tracking of the location of a grasped object within a hand [ 21 , 22 , 23 , 24 ] or for improving the recognition of sign language and hand gestures [ 25 , 26 , 27 ]. These multi-sensor techniques supplement each other, where the separate sensors measure different aspects of the motions of the arm and hands, after which their combined data is used for higher-level feature extraction for gesture recognition [ 28 ].…”

Section: Introductionmentioning

confidence: 99%

Robust Hand Motion Tracking through Data Fusion of 5DT Data Glove and Nimble VR Kinect Camera Measurements

Arkenbout

Winter

Breedveld

2015

Sensors

View full text Add to dashboard Cite

Vision based interfaces for human computer interaction have gained increasing attention over the past decade. This study presents a data fusion approach of the Nimble VR vision based system, using the Kinect camera, with the contact based 5DT Data Glove. Data fusion was achieved through a Kalman filter. The Nimble VR and filter output were compared using measurements performed on (1) a wooden hand model placed in various static postures and orientations; and (2) three differently sized human hands during active finger flexions. Precision and accuracy of joint angle estimates as a function of hand posture and orientation were determined. Moreover, in light of possible self-occlusions of the fingers in the Kinect camera images, data completeness was assessed. Results showed that the integration of the Data Glove through the Kalman filter provided for the proximal interphalangeal (PIP) joints of the fingers a substantial improvement of 79% in precision, from 2.2 deg to 0.9 deg. Moreover, a moderate improvement of 31% in accuracy (being the mean angular deviation from the true joint angle) was established, from 24 deg to 17 deg. The metacarpophalangeal (MCP) joint was relatively unaffected by the Kalman filter. Moreover, the Data Glove increased data completeness, thus providing a substantial advantage over the sole use of the Nimble VR system.

show abstract

Sensor fusion approaches for observation of user actions in programming by demonstration

Cited by 11 publications

References 12 publications

Integrating Electronic Components into Deformable Objects Based on User Interaction Data

Integrating Electronic Components into Deformable Objects Based on User Interaction Data

A Structural Framework for Assembly Modeling and Recognition

Robust Hand Motion Tracking through Data Fusion of 5DT Data Glove and Nimble VR Kinect Camera Measurements

Contact Info

Product

Resources

About