Natural language driven image generation

Adorni, Giovanni; Manzo, Mauro Di; Giunchiglia, Fausto

doi:10.3115/980431.980597

Cited by 19 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 the Mirai© 3D animation system from IZware, and uses 3D models from Viewpoint. 2 An individual semantic representation fragment as currently used in WordsEye may seem relatively simple when compared, say, with the PAR (("node2" (:ENTITY :3D-OBJECTS ("mr_happy") :LEXICAL-SOURCE "John" :SOURCE SELF)) ("node1" (:ACTION "say" :SUBJECT "node2" :DIRECT-OBJECT ("node5" "node4" "node7")...)) ("node5" (:ENTITY :3D-OBJECTS ("cat-vp2842"))) ("node4" (:STATIVE-RELATION "on" :FIGURE "node5"…”

Section: Linguistic Analysismentioning

confidence: 99%

“…Natural language input has been investigated in a number of 3D graphics systems including an early system by [2] and the oft-cited Put system [8]; the Put system shared our goal of making graphics creation easier, but was limited to spatial arrangements of existing objects. Also, input was restricted to an artificial subset of English consisting of expressions of the form Put (X P Y) ¡ , where X and Y are objects, and P is a spatial preposition.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

WordsEye

Coyne

Sproat

2001

Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques

271

View full text Add to dashboard Cite

Natural language is an easy and effective medium for describing visual ideas and mental images. Thus, we foresee the emergence of language-based 3D scene generation systems to let ordinary users quickly create 3D scenes without having to learn special software, acquire artistic skills, or even touch a desktop window-oriented interface. WordsEye is such a system for automatically converting text into representative 3D scenes. WordsEye relies on a large database of 3D models and poses to depict entities and actions. Every 3D model can have associated shape displacements, spatial tags, and functional properties to be used in the depiction process. We describe the linguistic analysis and depiction techniques used by WordsEye along with some general strategies by which more abstract concepts are made depictable.

show abstract

Section: Linguistic Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

WordsEye

Coyne

Sproat

2001

Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques

271

View full text Add to dashboard Cite

show abstract

“…In this case, intelligent systems facilitating cross-media references are helpful and worth developing. In this research area so far, it has been most conventional that conceptual contents conveyed by information media such as languages and pictures are represented in computable forms independent of each other and translated via 'transfer' processes so called which are often ad hoc and very specific to task domains [1][2][3].…”

Section: Introductionmentioning

confidence: 99%

Aware computing guided by L<inf>md</inf> expression and direct knowledge in spatial language understanding

Yokota

2011

2011 3rd International Conference on Awareness Science and Technology (iCAST)

View full text Add to dashboard Cite

Mental Image Directed Semantic Theory (MIDST) has proposed an omnisensory mental image model and its description language Lmd-This language can provide multimedia expressions with intermediate semantic descriptions in predicate logic. This paper describes systematic and efficient computing guided by Lmd expression and 3D map data, here so called direct knowledge of space, in cross-media operation between linguistic and pictorial expressions as spatial language understanding.

show abstract

“…The authors pointed out that there is a possible way to improve the visualization to be more dynamic. They suggested directly creating the scene rather than showing representative pictures; this can be done via text-toscene systems such as NALIG [1] and WordsEye [9], or text-to-animation systems such as animated pictures like text-to-picture Synthesis [21] and animations like Carsim [14], the latter of which converts narrative text about car accidents into 3D scenes using techniques for information extraction coupled with a planning and a visualization module. The CONFUCIUS system is also capable of converting single sentences into corresponding 3D animations [38].…”

Section: Introductionmentioning

confidence: 99%

Text-to-picture tools, systems, and approaches: a survey

Zakraoui

Saleh

Jaam

2019

Multimed Tools Appl

View full text Add to dashboard Cite

Text-to-picture systems attempt to facilitate high-level, user-friendly communication between humans and computers while promoting understanding of natural language. These systems interpret a natural language text and transform it into a visual format as pictures or images that are either static or dynamic. In this paper, we aim to identify current difficulties and the main problems faced by prior systems, and in particular, we seek to investigate the feasibility of automatic visualization of Arabic story text through multimedia. Hence, we analyzed a number of well-known text-to-picture systems, tools, and approaches. We showed their constituent steps, such as knowledge extraction, mapping, and image layout, as well as their performance and limitations. We also compared these systems based on a set of criteria, mainly natural language processing, natural language understanding, and input/output modalities. Our survey showed that currently emerging techniques in natural language processing tools and computer vision have made promising advances in analyzing general text and understanding images and videos. Furthermore, important remarks and findings have been deduced from these prior works, which would help in developing an effective text-to-picture system for learning and educational purposes.

show abstract

Natural language driven image generation

Cited by 19 publications

References 0 publications

WordsEye

WordsEye

Aware computing guided by L<inf>md</inf> expression and direct knowledge in spatial language understanding

Text-to-picture tools, systems, and approaches: a survey

Contact Info

Product

Resources

About