Sketch-n-Sketch

mage: Fluid Moves Between Code and Graphical Work in Computational Notebooks

“…Aside from data table transformation work mentioned [6,15], prior work in flexible GUI/code work systems has focused on visual domains, such as drawing [11], UI design [10], data visualizations [1], and commercial 3D modeling software. As these systems use various strategies for combining GUI with code, here we outline key design dimensions where mage draws upon, improves, or departs in key ways from prior work.…”

“…User action in code come with the risk of there not being a GUI equivalent for that action. For instance, if the user writes mathematical constraints on their star shape, how should the GUI interface behave if it has no widget behavior built-in to represent constraints [11]?…”

“…Beyond simple parameterization, another approach to balance between code expressivity and ambiguity is the use of a domain-specific language (DSL) where the creators carefully author mappings between a DSL and GUI actions [1,15,11]. Thus, for any action that the user takes, there is an equal representation of that action in both the code and GUI.…”

“…However, every DSL is by definition designed for a specific task. [11,6,15,9]. PBD has the benefit that, when carefully designed, it can ensure high-quality generated code synthesized fast enough to work in real time interactive applications [6,9].…”

“…As a departure from prior research, the goal of mage is not building a single highly flexible GUI/code tool, but rather creating a platform for an ecosystem of tools. Unlike a single closed system, like Sketch-n-Sketch for SVG editing [11], we cannot precisely model how a widget built with mage will affect state. Sketch-n-Sketch contributed a complete and live 1-1 mapping between GUI and code.…”