Video Annotation by Cascading Microtasks

Amorim, Marcello N. de; Segundo, Ricardo Mendes Costa; Santos, Celso A. S.; Tavares, Orivaldo de Lira

doi:10.1145/3126858.3126897

Cited by 4 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding the quality and productivity of video annotation, previous works [9], [30], [42], [43] have focused on impacting factors from the perspective of the annotators, ignoring the impact from the quality of the recorded video [18], [23], [44], [45]. In the design of ARiana, we have considered the video recording process and its impact on the quality and efficiency of video annotation.…”

Section: B Video Annotation Toolsmentioning

confidence: 99%

ARiana: Augmented Reality Based In-Situ Annotation of Assembly Videos

et al. 2022

View full text Add to dashboard Cite

Annotated videos are commonly produced for documenting assembly and maintenance processes in the manufacturing industry. However, according to a semi-structured interview we conducted with industrial experts, the current process of creating annotated assembly videos, in which the annotator annotates the video capturing the expert's demonstration of assembly and maintenance process, is cumbersome and time-consuming. The key challenges include three key problems in annotation: (1) unnecessary extra communications between field workers and annotators, (2) lack of suitable camera gear, and (3) wasting time in the manual removal of non-informative portions of captured videos. Because annotation always follows video capture, the problem 1 remains out of scope for state-of-the-art video annotation tools. And making the assumption of a perfect captured video, which covers no occlusion and contains only relevant assembly or maintenance information, causes problem 2 and 3. As a result, we have developed ARiana, a wearable augmented reality-based in-situ video annotation tool that guides field experts to create annotations efficiently while conducting the assembly or maintenance tasks. ARiana has three key features that include context-awareness enabled by hand-object interaction, multimodal interaction for annotation on the fly, and real-time audiovisual guidance enabled by edge offloading. We have implemented ARiana on Android-based smart glasses, equipped with first-person camera and microphone. In a usability test based on attempting to assemble a toy model and to annotate the recorded video simultaneously, ARiana demonstrated higher efficiency and effectiveness compared to one of the state-of-the-art video annotation tools, in which the assembling process is followed by the annotation process. In particular, ARiana helps users finish annotation tasks four times faster, and increase the annotation accuracy by 23%.

show abstract

Section: B Video Annotation Toolsmentioning

confidence: 99%

ARiana: Augmented Reality Based In-Situ Annotation of Assembly Videos

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Workers' contributions are processed and aggregated using different statistical-based algorithms to generate similar, equivalent, complementary results, or with acceptance similar to those that would be generated by specialists. This can be seen in the article [4], in which crowdsourcing was used to generate multimedia presentations based on video annotation.…”

Section: Introductionmentioning

confidence: 99%

Integrating Crowdsourcing and Human Computation for Complex Video Annotation Tasks

Amorim

Santos

Tavares

2020

Anais Estendidos Do XXVI Simpósio Brasileiro De Sistemas Multimídia E Web (WebMedia 2020)

View full text Add to dashboard Cite

Video annotation is an activity that aims to supplement this type of multimedia object with additional content or information about its context, nature, content, quality and other aspects. These annotations are the basis for building a variety of multimedia applications for various purposes ranging from entertainment to security. Manual annotation is a strategy that uses the intelligence and workforce of people in the annotation process and is an alternative to cases where automatic methods cannot be applied. However, manual video annotation can be a costly process because as the content to be annotated increases, so does the workload for annotating. Crowdsourcing appears as a viable solution strategy in this con- text because it relies on outsourcing the tasks to a multitude of workers, who perform specific parts of the work in a distributed way. However, as the complexity of required media annoyances increases, it becomes necessary to employ skilled labor, or willing to perform larger, more complicated, and more time-consuming tasks. This makes it challenging to use crowdsourcing, as experts demand higher pay, and recruiting tends to be a difficult activity. In order to overcome this problem, strategies based on the decom- position of the main problem into a set of simpler subtasks suitable for crowdsourcing processes have emerged. These smaller tasks are organized in a workflow so that the execution process can be formalized and controlled. In this sense, this thesis aims to present a new framework that allows the use of crowdsourcing to create applications that require complex video annotation tasks. The developed framework considers the whole process from the definition of the problem and the decomposition of the tasks, until the construction, execution, and management of the workflow. This framework, called CrowdWaterfall, contemplates the strengths of current proposals, incorporating new concepts, techniques, and resources to overcome some of its limitations.

show abstract