Learning task structure from video examples for workflow tracking and authoring

Petersen, Nils; Stricker, Didier

doi:10.1109/ismar.2012.6402562

Cited by 40 publications

(15 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [13,12], a method that uses image similarities captures and monitors workflows. The approach there compares incoming images with those stored for the same task and overlays them on a HMD.…”

Section: Guidance and Authoringmentioning

confidence: 99%

See 1 more Smart Citation

Automated capture and delivery of assistive task guidance with an eyewear computer

Leelasawassuk

Damen

Mayol-Cuevas

2017

Proceedings of the 8th Augmented Human International Conference

View full text Add to dashboard Cite

General rightsThis document is made available in accordance with publisher policies. Please cite only the published version using the reference above. ABSTRACTIn this paper we describe and evaluate an assistive mixed reality system that aims to augment users in tasks by combining automated and unsupervised information collection with minimally invasive video guides. The result is a fully self-contained system that we call GlaciAR (Glass-enabled Contextual Interactions for Augmented Reality). It operates by extracting contextual interactions from observing users performing actions. GlaciAR is able to i) automatically determine moments of relevance based on a head motion attention model, ii) automatically produce video guidance information, iii) trigger these guides based on an object detection method, iv) learn without supervision from observing multiple users and v) operate fully on-board a current eyewear computer (Google Glass). We describe the components of GlaciAR together with user evaluations on three tasks. We see this work as a first step toward scaling up the notoriously difficult authoring problem in guidance systems and an exploration of enhancing user natural abilities via minimally invasive visual cues.

show abstract

Section: Guidance and Authoringmentioning

confidence: 99%

“…These approaches aim to alleviate the authoring problem somehow by trying to automate the extraction of relevant information as much as possible. In the case of [13,12] this is done using 2D image metrics and the information to be displayed to users is an overlay on top of the object being part of the task.…”

Section: Guidance and Authoringmentioning

confidence: 99%

Automated capture and delivery of assistive task guidance with an eyewear computer

Leelasawassuk

Damen

Mayol-Cuevas

2017

Proceedings of the 8th Augmented Human International Conference

View full text Add to dashboard Cite

show abstract

“…An alternative approach is to build AR manuals from recorded videos [20]. This allows to author complicated manuals without additional knowledge about any programming language or animation tool.…”

Section: Related Workmentioning

confidence: 99%

Retargeting Technical Documentation to Augmented Reality

Mohr¹,

Kerbl²,

Donoser³

et al. 2015

Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

a) (c) (d) Figure 1. Retargeted 2D documentation. (a) The input documentation consists of an annotated explosion diagram and a sequence of images presenting disassembly instructions. (b)From analyzing the 2D explosion diagram, our system is able to generate a 3D explosion diagram presented in AR. Moreover, our system generates 3D annotations in AR based on the input 2D documentation. (c) In addition, our system is able to analyze image sequences in order to create 3D animations from it. This allows to present animated 3D documentations in AR. Here we show six key-frames from the resulting AR animation. ABSTRACTWe present a system which automatically transfers printed technical documentation, such as handbooks, to threedimensional Augmented Reality. Our system identifies the most frequent forms of instructions found in printed documentation, such as image sequences, explosion diagrams, textual annotations and arrows indicating motion. The analysis of the printed documentation works automatically, with minimal user input. The system only requires the documentation itself and a CAD model or 3D scan of the object described in the documentation. The output is a fully interactive Augmented Reality application, presenting the information from the printed documentation in 3D, registered to the real object.

show abstract

“…Other mixed reality applications emphasize the virtual content without connecting it to the real environment, i.e., Virtual Reality systems. The potential of both Augmented and Virtual Reality for learning purposes has been elaborated in previous work, i.e., COGNITO [3], VISTRA [4], AR handbook [5], and manual workstation [6]. For instance, using head mounted displays or glasses providing information at the time and place when and where it is needed can be a promising opportunity for learning on-the-job.…”

Section: Mixed Realitymentioning

confidence: 99%

In-Factory Learning – Qualification For The Factory Of The Future

Quint

Mura

Gorecky

2015

ACTA Universitatis Cibiniensis

View full text Add to dashboard Cite

The Industry 4.0 vision anticipates that internet technologies will find their way into future factories replacing traditional components by dynamic and intelligent cyber-physical systems (CPS) that combine IntroductionThe arising connection between the real physical world (like our homes and factories) and the digital world (support by internet technologies) brings more and more intelligent objects into our everyday lives and work contexts. The so-called Internet of Things facilitates, for instance, smart homes, which can adjust heat, light, and ventilation autonomously in order to optimize both energy consumption and comfort. The Internet of Things implies a revolution for the future factory environment, too. Following the Industry 4.0 vision [1] we expect the digitalisation to promote intelligent devices, which are able to learn from experiences, to communicate with each other, and to take decisions towards selfoptimization. Alongside this extensive automation, the human worker has been acknowledged as the most flexible entity in the production system, who plans, controls, manages, and trouble shoots. Thus, the demands for the employees' broad and interdisciplinary skills increase simultaneously.High complexity and various interdependencies result from the involved smart objects, which may act in a dynamic way instead of being static and predictable. Furthermore, production workers will initially have difficulties to keep track of the digital information behind the physical production environment. The challenge is that the digital mechanisms are invisible in nature. Professions in manufacturing or electronic engineering have to be enriched by interdisciplinary competences from computer science in order to make employees cope with their new requirements. Therefore, innovative and appropriate qualification is needed.In the light of the mentioned learning challenges within smart factories and other environments, we suggest a comprehensive approach combining current trends from human-machine interaction and insights from psychological and pedagogical science with the aim of developing interactive learning environments integrating real and virtual world.

show abstract

Learning task structure from video examples for workflow tracking and authoring

Cited by 40 publications

References 20 publications

Automated capture and delivery of assistive task guidance with an eyewear computer

Automated capture and delivery of assistive task guidance with an eyewear computer

Retargeting Technical Documentation to Augmented Reality

In-Factory Learning – Qualification For The Factory Of The Future

Contact Info

Product

Resources

About