Automatic Generation of Two-Level Hierarchical Tutorials from Instructional Makeup Videos

Truong, Anh Minh; Chi, Peggy; Salesin, David; Essa, Irfan; Agrawala, Maneesh

doi:10.1145/3411764.3445721

Cited by 44 publications

(22 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cuts are performed only when speakers change. Automatic segmentations of videos are performed for tutorial videos [34], for example, a semi-automatic video editing system will be developed to support the production of concise tutorials [6]. Automatic camera control of a single camera is used especially in amateur sports, because production with many cameras and camera crew is expensive.…”

Section: Related Workmentioning

confidence: 99%

Modeling of an Automatic Vision Mixer With Human Characteristics for Multi-Camera Theater Recordings

et al. 2023

View full text Add to dashboard Cite

Laymen are not trained in camera control and how to use a vision mixing desk to switch between different cameras in a video production. Specific training and expert knowledge are required. In the non-professional environment, multi-camera recordings of theater performances or other stage performances are therefore difficult to realize. This can be remedied by a production process with highresolution cameras from which image sections of long shots or medium long shots are cropped manually or automatically in post-production. One approach to automate the generation of suitable image cuts could be to focus on speaker changes so that the person who is speaking is shown in the generated cut. However, these image cuts can appear static and robotic if they are set too precisely. Therefore, the characteristics and habits of professional vision mixers (persons who operate the vision mixing desk) during the editing process are investigated in more detail in order to incorporate them into an automation process. The characteristic features of five different vision mixers are examined, which were used under almost identical recording conditions for theatrical cuts in TV productions. The cuts are examined with regard to their temporal position in relation to pauses in speech, which take place during speaker changes on stage. It is shown that different professional vision mixers set the cuts individually differently before, in or after the pauses in speech. From the analysis of the image cuts, an approach for a model is developed in which the individual characteristics of a vision mixer can be set. With the help of this model, a more human appearance can be given to otherwise exact and robotic cuts, when automating image cuts. INDEX TERMS Automatic vision mixer, human charakteristics, multi-camera theatre recordingsPreliminary note:The term vision mixer is usually referred to both the device used to edit video and the person operating the device. The ambiguity is avoided in this paper by using the following terms:• Vision mixing desk = the device used to edit video.• Vision mixer (also abbreviated as VM) = the person who operates the vision mixing desk.

show abstract

Section: Related Workmentioning

confidence: 99%

Modeling of an Automatic Vision Mixer With Human Characteristics for Multi-Camera Theater Recordings

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Prior research has explored different approaches to reduce the effort of makeup steps through technology, such as providing makeup recommendations [44,64,75,76], recording and sharing makeup logs [71], improving makeup creativity [93], developing instructional makeup videos [94], enabling interactive makeup experiences [48] and supporting marginalized groups in makeup [20]. For example, Jain and Bhatti [44] developed a multimodal cosmetic advisory system that leveraged face recognition and color detection to provide recommendations based on skin tones.…”

Section: Existing Technology For Makeupmentioning

confidence: 99%

“…To support the creativity of makeup processes, Treepong et al [93] introduced an interactive face makeup system that combined 3D face modeling, tangible interfaces, projection mapping techniques, and a drawing system that allowed users to interactively design their makeup that enhanced creativity. As additional examples in learning makeup styles, Truong et al [94] showed the approach of combining computer vision with transcript text analysis to provide hierarchical tutorials from instructional makeup videos automatically, and Chang et al [17] used content-based voice navigation for how-to videos in makeup tutorials. Beyond supporting makeup just for the general public, Chong et al [20] created a makeup recommendation system for transgender individuals through automatic facial recognition systems.…”

Section: Existing Technology For Makeupmentioning

confidence: 99%

"It Feels Like Taking a Gamble": Exploring Perceptions, Practices, and Challenges of Using Makeup and Cosmetics for People with Visual Impairments

Li,

Spektor,

Xia

et al. 2022

Preprint

View full text Add to dashboard Cite

Makeup and cosmetics offer the potential for self-expression and the reshaping of social roles for visually impaired people. However, there exist barriers to conducting a beauty regime because of the reliance on visual information and color variances in makeup. We present a content analysis of 145 YouTube videos to demonstrate visually impaired individuals' unique practices before, during, and after doing makeup. Based on the makeup practices, we then conducted semi-structured interviews with 12 visually impaired people to discuss their perceptions of and challenges with the makeup process in more depth. Overall, through our findings and discussion, we present novel perceptions of makeup from visually impaired individuals (e.g., broader representations of blindness and beauty).The existing challenges provide opportunities for future research to address learning barriers, insufficient feedback, and physical and environmental barriers, making the experience of doing makeup more accessible to people with visual impairments. CCS Concepts: • Human-centered computing → Empirical studies in accessibility.

show abstract

“…Khan et al personalized the recommendations by utilizing web documentations and a user's command history [18]. Recent work has shown that by presenting its two-level hierarchy, a tutorial can be more easily consumed, especially for longer sequences [34].…”

Section: Context-aware Tutorial Followingmentioning

confidence: 99%

HelpViz: Automatic Generation of Contextual Visual Mobile Tutorials from Text-Based Instructions

Zhong

Chi

et al. 2021

The 34th Annual ACM Symposium on User Interface Software and Technology

Self Cite

View full text Add to dashboard Cite

We present HelpViz, a tool for generating contextual visual mobile tutorials from text-based instructions that are abundant on the web. HelpViz transforms text instructions to graphical tutorials in batch, by extracting a sequence of actions from each text instruction through an instruction parsing model, and executing the extracted actions on a simulation infrastructure that manages an array of Android emulators. The automatic execution of each instruction produces a set of graphical and structural assets, including images, videos, and metadata such as clicked elements for each step. HelpViz then synthesizes a tutorial by combining parsed text instructions with the generated assets, and contextualizes the tutorial to user interaction by tracking the user's progress and highlighting the next step. Our experiments with HelpViz indicate that our pipeline improved tutorial execution robustness and that participants preferred tutorials generated by HelpViz over text-based instructions. HelpViz promises a cost-effective approach for generating contextual visual tutorials for mobile interaction at scale. CCS CONCEPTS• Human-centered computing → Interactive systems and tools.

show abstract

Automatic Generation of Two-Level Hierarchical Tutorials from Instructional Makeup Videos

Cited by 44 publications

References 25 publications

Modeling of an Automatic Vision Mixer With Human Characteristics for Multi-Camera Theater Recordings

Modeling of an Automatic Vision Mixer With Human Characteristics for Multi-Camera Theater Recordings

"It Feels Like Taking a Gamble": Exploring Perceptions, Practices, and Challenges of Using Makeup and Cosmetics for People with Visual Impairments

HelpViz: Automatic Generation of Contextual Visual Mobile Tutorials from Text-Based Instructions

Contact Info

Product

Resources

About