Resident's Satisfaction in Street Landscape Using the Immersive Virtual Environment-Based Eye-Tracking Technique and Deep Learning Model

Abstract

Abstract. Virtual reality technology provides a significant clue to understanding the human visual perception process by enabling the interaction between humans and computers. In addition, deep learning techniques in the visual field provide analysis methods for image classification, processing, and segmentation. This study reviewed the applicability of gaze movement and deep learning-based satisfaction evaluation on the landscape using an immersive virtual reality-based eye-tracking device. To this end, the following research procedures were established and analysed. First, the gaze movement of the test taker is measured using an immersive virtual environment-based eye tracker. The relationship between the gaze movement pattern of the test taker and the satisfaction evaluation result for the landscape image is analysed. Second, using the Convolutional Neural Networks (CNN)-based Class Activation Map (CAM) technique, a model for estimating the satisfaction evaluation result is constructed, and the gaze pattern of the test taker is derived. Third, we compare and analyse the similarity between the gaze heat map derived through the immersive virtual environment-based gaze tracker and the heat map generated by CAM. This study suggests the applicability of urban environment technology and deep learning methods to understand landscape planning factors that affect urban landscape satisfaction, resulting from the three-dimensional and immediate visual cognitive activity.

Research on the Application of Extended Reality in the Construction and Management of Landscape Engineering

“…There is no statement or description of the graphics engines or add-ons used to create XR application scenarios 29 [6][7][8][10][11][12]15,16,31,34,35,[37][38][39][40][41][42]45,47,48,[50][51][52]54,61,62,67,68,70] •…”

“…Fully immersive experiences, panoramic roaming, virtual simulation. 56 [3][4][5][6][7][8][9][10][11][12][13][14][15][45][46][47][48][49][50]53,54,[58][59][60][61][62][63][66][67][68][69][70]82,83] AR Environment perception integration of virtual and real, enhanced interactive experience. 15 [16][17][18][19]21,39,46,51,52,54,57,61,63,64,69] MR Virtual and real seamless integration, multimode real-time interaction.…”

“…XR, through human-computer interactions, provides crucial insights into understanding the human visual perception process [53]. Additionally, deep learning in the field of view offers analytical approaches for image classification, processing, and segmentation [50]. For instance, Kido et al [30] developed an MR-based future landscape assessment system capable of recognizing the surrounding environment and estimating landscape indices using deep learning semantic segmentation.…”

“…Computer vision methods automate the analysis of traditional landscape images, reducing the need for researcher intervention. Coupled with algorithms based on deep learning, these methods minimize time and human resource inputs, garnering attention in areas such as plant growth health monitoring, landscape preference assessment, and plant education learning [21,30,35,50,63,67].…”

“…Landscape aesthetic preference and cognition Among the literature included in the analysis, there are five articles [38,39,42,49,50] on the application of XR in landscape aesthetic preference and cognition. Understanding preferences for natural landscapes is crucial for creating attractive green spaces and promoting the benefits individuals derive from nature.…”