The Object at Hand: Automated Editing for Mixed Reality Video Guidance from Hand-Object Interactions

Lu, Yao; Mayol-Cuevas, Walterio

doi:10.1109/ismar52148.2021.00023

Cited by 4 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dataset 100 Days of Hands (100DOH) [32] is composed of both Third Person Vision (TPV) and FPV images and is suitable to study object-class agnostic HOI detection. The authors of [24] labeled images collected from different FPV datasets [7,12,20] providing annotations for hands, objects and their relation. The MECCANO dataset [29] contains videos acquired in an industrial-like domain also annotated with bounding boxes around active objects, together with the related classes.…”

Section: Related Workmentioning

confidence: 99%

“…Previous FPV works focused on the detection of hands interacting with an object without recognizing it [1,25]. Other recent works focused on object-class agnostic EHOI detection [8,24]. The authors of [29] defined Egocentric Human-Object Interaction (EHOI) detection as the task of producing <verb, objects> pairs.…”

Section: Related Workmentioning

confidence: 99%

“…Such models are then used with the proposed data generation tool to automatically produce labeled images of EHOIs. Even though some works provide datasets to study HOI in general domains [1,6,7,24,32] and in industrial contexts [29] to the best of our knowledge, this is the first attempt to define a pipeline for the generation of a large-scale photo-realistic synthetic FPV dataset to study EHOIs with rich annotations of active and non-active objects in an industrial scenario. To assess the suitability of the generated synthetic data to tackle the EHOI detection task, we acquired and labeled 8 real egocentric videos in an industrial laboratory, in which subjects perform test and repair operations on electrical boards (see Figure 2).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Egocentric Human-Object Interaction Detection Exploiting Synthetic Data

Leonardi¹,

Ragusa²,

Furnari³

et al. 2022

Preprint

View full text Add to dashboard Cite

We consider the problem of detecting Egocentric Human-Object Interactions (EHOIs) in industrial contexts. Since collecting and labeling large amounts of real images is challenging, we propose a pipeline and a tool to generate photo-realistic synthetic First Person Vision (FPV) images automatically labeled for EHOI detection in a specific industrial scenario. To tackle the problem of EHOI detection, we propose a method that detects the hands, the objects in the scene, and determines which objects are currently involved in an interaction. We compare the performance of our method with a set of state-of-the-art baselines. Results show that using a synthetic dataset improves the performance of an EHOI detection system, especially when few real data are available. To encourage research on this topic, we publicly release the proposed dataset at the following url: https://iplab.dmi.unict.it/EHOI SYNTH/.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Egocentric Human-Object Interaction Detection Exploiting Synthetic Data

Leonardi¹,

Ragusa²,

Furnari³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Egocentric Human-Object Interaction Detection Exploiting Synthetic Data

Leonardi

Ragusa

Furnari

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Decision support for augmented reality-based assistance systems deployment in industrial settings

Bock,

Bohné,

Tadeja

2024

Multimed Tools Appl

View full text Add to dashboard Cite

The successful deployment of augmented reality (AR) in the industry for on-the-job guidance depends heavily on factors such as the availability of required expertise, existing digital content and other deployment-related criteria such as a task’s error-proneness or complexity. Particularly in idiosyncratic manufacturing situations involving customised products and diverse complex and non-complex products and its variants, the applicability and attractiveness of AR as a worker assistance system is often unclear and difficult to gauge for decision-makers. To address this gap, we developed a decision support tool to help prepare customised deployment strategies for AR-based assistance systems utilising manual assembly as the main example. Consequently, we report results from an interview study with sixteen domain experts. Furthermore, when analysing captured expert knowledge, we found significant differences in criteria weighting based on task complexity and other factors, such as the effort required to obtain data.

show abstract

The Object at Hand: Automated Editing for Mixed Reality Video Guidance from Hand-Object Interactions

Cited by 4 publications

References 26 publications

Egocentric Human-Object Interaction Detection Exploiting Synthetic Data

Egocentric Human-Object Interaction Detection Exploiting Synthetic Data

Egocentric Human-Object Interaction Detection Exploiting Synthetic Data

Decision support for augmented reality-based assistance systems deployment in industrial settings

Contact Info

Product

Resources

About