A Neural Representation of Sketch Drawings

Ha, David; Eck, Douglas

doi:10.48550/arxiv.1704.03477

Cited by 177 publications

(206 citation statements)

References 0 publications

Supporting

Mentioning

206

Contrasting

Order By: Relevance

“…In our lesson, we focused on intuition and applications instead. VAEs as a subgroup of generative models have demonstrated creativity in music, drawings, and 3D objects generation: MusicVAE (Roberts et al 2019) was trained on melodies and beats, the web-based tools Melody Blender and Beats Mixer leverage MusicVAE to let users explore melodies and beats interpolation, SketchRNN (Ha and Eck 2017) is able to complete the users' sketches, performing interpolations, and mimic users' drawings, PointFlow (Yang et al 2019) trained a VAE to interpolate between objects constructed by 3D points.…”

Section: Variational Autoencoders (Vaes)mentioning

confidence: 99%

“…The continuity of VAE's latent space enables interpolation, which provides a creative way to explore artistic ideas. Hence, it is no surprise that VAEs are increasingly being used for media generation like creating music (Roberts et al 2019), sketches (Ha and Eck 2017), or 3D shapes (Yang et al 2019). In this work, we explore ways to teach high school students about VAEs and their applications, with the aim for these approaches to be included in high school AI curricula.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Introducing Variational Autoencoders to High School Students

Lyu¹,

Ali²,

Breazeal³

2021

Preprint

View full text Add to dashboard Cite

Generative Artificial Intelligence (AI) models are a compelling way to introduce K-12 students to AI education using an artistic medium, and hence have drawn attention from K-12 AI educators. Previous Creative AI curricula mainly focus on Generative Adversarial Networks (GANs) while paying less attention to Autoregressive Models, Variational Autoencoders (VAEs), or other generative models, which have since become common in the field of generative AI. VAEs' latent-space structure and interpolation ability could effectively ground the interdisciplinary learning of AI, creative arts, and philosophy. Thus, we designed a lesson 1 to teach high school students about VAEs. We developed a web-based game and used Plato's cave, a philosophical metaphor, to introduce how VAEs work. We used a Google Colab notebook for students to re-train VAEs with their hand-written digits to consolidate their understandings. Finally, we guided the exploration of creative VAE tools such as SketchRNN and Mu-sicVAE to draw the connection between what they learned and real-world applications. This paper describes the lesson design and shares insights from the pilot studies with 22 students. We found that our approach was effective in teaching students about a novel AI concept.

show abstract

Section: Variational Autoencoders (Vaes)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Introducing Variational Autoencoders to High School Students

Lyu¹,

Ali²,

Breazeal³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…proposed a system to automatically generate large-scale Chinese handwriting fonts by learning styles of stroke shape and layout separately. Zhang et al [2017] and Ha et al [2017] utilized RNNs to generate handwritten characters or sketches. proposed to handle the task of generating large-scale Chinese fonts from a few input samples by using a modified RNN model with the monotonic attention mechanism.…”

Section: Vector Font Generationmentioning

confidence: 99%

“…Early machine-learning based methods, such as [Campbell and Kautz 2014;Suveeranont and Igarashi 2010], perform affine transformations or manifold learning on existing font data to synthesize vector fonts in new styles. More recently, several methods based on sequential generative models have been proposed to directly generate the draw-command sequences of sketches, handwritings or SVGs (Scalable Vector Graphics), such as [Carlier et al 2020;Ha and Eck 2017;Lopes et al 2019;Zhang et al 2017]. The sequential generative models are naturally suitable for vector font generation as they are capable of imitating the writing/drawing process of glyphs.…”

Section: Introductionmentioning

confidence: 99%

“…The first problem is that most of those previous works only consider one modality of glyphs when encoding features, i.e., using either the sequential representation ( [Carlier et al 2020;Ha and Eck 2017;Zhang et al 2017]) or the image representation ( [Lopes et al 2019]). In this paper, we investigate how to fully exploit the information contained in the two modalities of glyphs, i.e., the image modality and the sequence modality, respectively.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning

Wang,

Lian

2021

Preprint

View full text Add to dashboard Cite

nnnnnnn features of fonts to synthesize vector glyphs. Second, we provide a new generative paradigm to handle unstructured data (e.g., vector glyphs) by randomly sampling plausible synthesis results to get the optimal one which is further refined under the guidance of generated structured data (e.g., glyph images). Finally, qualitative and quantitative experiments conducted on a publicly-available dataset demonstrate that our method obtains highquality synthesis results in the applications of vector font generation and interpolation, significantly outperforming the state of the art.

show abstract

Geo‐Foundation Models

Mai

2024

International Encyclopedia of Geography

View full text Add to dashboard Cite

As a group of task‐agnostic pretrained large‐scale neural network models that can be later adapted to numerous downstream tasks, foundation models have made a significant impact on academia, industry, and society. Meanwhile, several efforts have been made to develop foundation models for the geoscience domain. They are known as geo‐foundation models (GeoFMs). The necessary steps for GeoFM development were taken in the context of the uniqueness of geographic data and a collaborative effort among academia, industry, and society is necessary to develop a reliable, sustainable, and ethically aware framework.

show abstract

A Neural Representation of Sketch Drawings

Cited by 177 publications

References 0 publications

Introducing Variational Autoencoders to High School Students

Introducing Variational Autoencoders to High School Students

DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning

Geo‐Foundation Models

Contact Info

Product

Resources

About