Spatio-Temporal parsing in spatial hypermedia

Abstract

Since 2011 Thomas Schedel has been working as a research assistant at the Institute of Information Systems (iisys) at Hof University. In 2016 he successfully finished the Ph.D. program "Computer Science and Engineering" (CSE) at Aalborg University in Denmark. In his thesis - supervised by Henrik Legind Larsen (Aalborg University) and Claus Atzenbeck (Institute of Information Systems, Hof University) - he claims that considering temporal aspects in addition to spatial and visual properties in spatial parser design will lead to significant increase in parsing accuracy, detection of richer structures and thus higher parser performance. For the purpose of providing evidence, parsers for recognizing spatial, visual and temporal object relations have been implemented and tested in a series of user surveys. It turned out, that in none of the test cases pure spatial or visual parser could outperform the spatio-temporal parser. Instead, the spatio-temporal parser was able to compensate limitations of conventional parsers. Furthermore, differences in parsing accuracy were successfully tested for statistical significance. The results indicate a non-trivial effect that is recognizable by humans. We have shown that the addition of a temporal parser shifts machine detected structures significantly closer to what knowledge workers intend to express. Due to its significant contributions to the research area of spatial hypermedia the thesis has been awarded with the Informatikpreis 2016 (given by the German Fachbereichstag Informatik, FBTI).

Synthesising augmentation and automation

“…For example, spatial hypertext uses visual cues such as spatial distance or arrangements for expressing associations. Some users may not expect such "fuzzy" connections, ambiguities or different interpretations to be provided by a computer, as they are more difficult for machines to handle (Schedel & Atzenbeck, 2016).…”

“…The most distinguished feature in Mother is its spatial parsers that are capable of computing associations between nodes placed on a 2D canvas. This is based on their visual cues, spatial arrangements or distance (Schedel & Atzenbeck, 2016). Figure 2 shows Mother's architecture.…”

“…HEIMDALL's intelligent UI bridges the gap between the acquired knowledge and users in doubt about which Cubble to use: The spatial structure service in Mother's software layer Asgard uses spatio-temporal parsers (Schedel & Atzenbeck, 2016) that analyse relations between components that the user has dropped onto the BDE. From that, the system acquires related Cubbles from the knowledge base and presents them to the user.…”

“…Typical hypertextual structure are complex by nature, such that a naïve conversion to a sequential, spoken text is not possible. Transforming navigational or even spatial hypertext to a sequential text is heavily related to structure interpretation, as provided, for example, by spatial parsers (Schedel & Atzenbeck, 2016).…”

Visualization of the Relevance

“…With CAOS [25] it was shown, that it is advantageous to keep the machine's interpretation in sync with what the user is doing, because it allows instant visual feedback and further applications like collaboration support. Most recently Schedel enhanced spatial parsing with a temporal inspection of user interaction events [28].…”