Crosspower

Abstract

Despite the ubiquity of direct manipulation techniques available in computer-aided design applications, creating digital content remains a tedious and indirect task. This is because applications require users to perform numerous low-level editing operations rather than allowing them to directly indicate high-level design goals. Yet, the creation of graphic content, such as videos, animations, and presentations often begins with a description of design goals in natural language, such as screenplays, scripts, outlines. Therefore, there is an opportunity for languageoriented authoring, i.e., leveraging the information found in the structure of a language to facilitate the creation of graphic content. We present a systematic exploration of the identification, graphic description, and interaction with various linguistic structures to assist in the creation of visual content. The prototype system, Crosspower, and its proposed interaction techniques, enables content creators to indicate and customize their desired visual content in a flexible and direct manner.

LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing

“…Additionally, visual elements of raw footage such as thumbnails and audio waveforms might not always convey its semantics effectively. Language, on the other hand, offers an intuitive and efficient alternative to complex UI in video editing and has been investigated in video editing tool research [23,28,29,56,68,78,79]. One common approach treats language as a "Command", where users employ language to instruct tools for specific operations.…”

CrossData: Leveraging Text-Data Connections for Authoring Data Documents

“…For example, Rubin et al [45] and Troung et al [55] leveraged the linear temporal properties that are common across text, audio, and video to assist in the editing of media clips. Perhaps the most closely related work to the present research is Crosspower [59], which leveraged desired correspondences between linguistic structures and graphical structures to enable users to flexibly and quickly create and manipulate graphical elements, as well as their layouts and animations. The present research also seeks to support content creation.…”

Preserving Hand-Drawn Qualities in Audiovisual Performance Through Sketch-Based Interaction

“…One approach would be to develop mechanisms that could act on changing visual and sonic qualitiese.g. line style and colors or tones and timbers -or on organizing events in time to create movement and directionality established by a predetermined script as demonstrated by [56]. In Megafauna, features that act on the timelines show simple forms of how narrative can be supported by computational means.…”