Web Interface Interpretation Using Graph Grammars

Abstract

Abstract-With the advent of the Internet, it is desirable to interpret and extract useful information from the Web. One major challenge in Web interface interpretation is to discover the semantic structure underlying a Web interface. Many heuristic approaches have been developed to discover and group semantically related interface objects. However, those approaches cannot solve the problem of nonuniformity satisfactorily and are not able to tag the semantic role of each object. Distinct from existing approaches, this paper develops a robust and formal approach to recovering interface semantics using graph grammars. Because of the distinct capability of spatial specifications in the abstract syntax, the spatial graph grammar (SGG) is selected to perform the semantic grouping and interpretation of segmented screen objects. Instead of analyzing HTML source codes, we apply an efficient imageprocessing technology to recognize atomic interface objects from the screenshot of an interface and produce a spatial graph, which records significant spatial relations among recognized objects. A spatial graph is more concise than its corresponding document object model structure and, thus, facilitates interface analysis and interpretation. Based on the spatial graph, the SGG parser recovers the hierarchical relations among interface objects.

An enhanced grammatical approach for graph drawing

“…SGG uses qualitative analysis methods to describe the spatial relationships between any two nodes, including 6 topological relationships, 4 directional relationships, sequential distance relationships, and 7 alignment relationships. Based on the qualitative analysis method, SGG can intuitively and flexibly analyze the syntax structures and the spatial models of various visual languages and has achieved good application results in the field of web page description and pattern verification [21,22]. However, SGG also has shortcomings.…”