From Lost to Found

Abstract

Design sharing sites provide UI designers with a platform to share their works and also an opportunity to get inspiration from others' designs. To facilitate management and search of millions of UI design images, many design sharing sites adopt collaborative tagging systems by distributing the work of categorization to the community. However, designers often do not know how to properly tag one design image with compact textual description, resulting in unclear, incomplete, and inconsistent tags for uploaded examples which impede retrieval, according to our empirical study and interview with four professional designers. Based on a deep neural network, we introduce a novel approach for encoding both the visual and textual information to recover the missing tags for existing UI examples so that they can be more easily found by text queries. We achieve 82.72% accuracy in the tag prediction. Through a simulation test of 5 queries, our system on average returns hundreds more results than the default Dribbble search, leading to better relatedness, diversity and satisfaction. CCS Concepts: • Human-centered computing → User interface design; User interface management systems; Graphical user interfaces.

MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling

“…Other research involves modeling UI screen retrieval to aid designers in data-driven searches, creating design examples by stimulating inspiration, generating new ideas, and examining design decisions from UI repositories. Numerous attempts have been made to enhance retrieval algorithms to obtain more relevant UI designs, utilizing deep learning [16,33,35,42,58], reverse engineering [17,28,29,57], and app information [47]. However, the significance of UI repositories has often been neglected.…”

Enhancement of GUI Display Error Detection Using Improved Faster R-CNN and Multi-Scale Attention Mechanism

“…Qian et al harnessed computer vision techniques for robotically operating applications through a touch screen, automating GUI testing [8]. Chen et al introduced a deep learning-based method encoding visual and text information to recover missing labels for the current UI, simplifying their retrieval via text queries [9]. They also examined the limitations and effective design of deep learningbased object detection methods for UI components, incorporating a new top-down strategy combined with a deep learning-based GUI text model [10].…”

“…Hence, this paper uses the AP metric to quantitatively analyze the algorithm's performance. The formula is seen in Equation (9).…”

A Practice Theory Perspective on Dribbble and the Evolving Design Industry

“…They showed that users did think something like virality did happen and that what spread were design trends, like a line quality or style or set of color combinations. More recent work (Chen et al, 2020) uses computational approaches to help designers tag their UI designs, in an effort to help designers more quickly find examples of others’ design solutions. None of the above studies examines the impact that Dribbble may bring to the design industry.…”