Fragmented Layers of Design Thinking: Limitations and Opportunities of Neural Language Model-assisted processes for Design Creativity

Abstract

This paper offers insights about the otherwise limited NLM-driven methodologies, supporting an examination of design creativity following the ‘process’ approach. [Abraham 2018] Recent application of AI models which rely on natural language processing (semantic references) is increasingly popular because of their directness and ease-of-use. Neural Language Models (NLMs) like VQGAN+CLIP, DALL-E, MidJourney) offer promising results, [Rodrigues, et al. 2021] seemingly bypassing the need for expensive datasets and technical expertise. Naturally, such models are limited because they cannot capture the multimodal complexity of architectural thinking and human cognition in general [Penrose 1989]. Alternative approaches propose the combination of NLMs with other artificial neural networks (ANNs) i.e. StyleGAN; CycleGAN which are custom-trained on domain-specific data. [Bolojan, Vermisso and Yousif 2022] Architects seek to expand their agency within such AI-assisted processes by controling the input encoding, so they can subsequently convert the generated outcomes to 3D models fairly directly. Still, AI models of computer vision like NLMs and GANs offer 2-dimensional output, which requires extensive decoding into 3-dimensional format. While this may seem severely constraining, it presents a silver lining when it comes to furthering design creativity. Designers are asked to scrutinize their methods from a cognitive standpoint, because these methodologies not only encourage, but demand thorough interrogation of the design intentionality, the design decision making factors and qualification criteria. Text-to-image correlation, on which NLMs rely, and their 2-dimensional output, ensure that certain important considerations are not circumvented. Instead of obtaining a 3D model, multiple possible -fragmented- versions of it are separately implied. Often, ‘fake’ images generated by the ANNs promote contradictory inferences of space, which require further examination. The hidden opportunity within the limited format of AI models echo Neil Spiller’s comments about the advantage of drawing over animation techniques twenty years ago: “Enigma is a creative tool that allows designers to see bifurcated outcomes in their sketches and drawings; it plays on the inability of drawings to faithfully record the distinct placement and extent of architectural elements”. [Spiller 2001] Comparing animations to static drawings, Spiller praised the drawing’s ability to hold “…an imagined past and an imagined future”. ‘Reading’ these results involves the (human) disentanglement of high and low-level features and consciously allocating their corresponding qualities for curation. The process of evaluating ‘parts-to-whole’ visual relationships is noteworthy because it depends on shifting our attention away from certain features, and an unconscious binding of visual elements. [Dehaene 2014] The philosopher Alain wrote that “The art of paying attention, the great art,…supposes the art of not paying attention…the royal art”. [Dehaene 2021]. According to neuroscientists, the brain uses attention as an amplifier and selective filter, during one of the three major attention systems (Alerting; Orienting; Executive Attention). [Dehaene 2021] Orienting our attention addresses what we focus on and what we don’t. Suppressing the unwanted information, through interfering electrical waves, is useful for processing the object of attention. Considering the ANNs’ results at ‘Gestalt’ level, we can structure the AI-assisted process to ensure low-level features (composition) is retained while enhancing high-level (detail) features (Fig.1a).