An End-User Pipeline for Scraping and Visualizing Semi-Structured Data over the Web

Abstract

The Web is a vast source of semi-structured data sets that are made readily available to support the construction of new knowledge. Information visualization techniques have been demonstrated a suitable alternative for allowing users to analyze and understand a large amount of data. However, the steps required for visualizing semi-structured data obtained from the Web is not straightforward, and it requires proper treatment before information visualization techniques could be applied. In this work, we present a visualization pipeline for describing the fundamental operations required for visualizing semi-structured data over the Web. For that, we employ Web Scrapping and Web Augmentation techniques for supporting interactive visualizations and solving tasks without changing the context of use of the data. Our approach is duly supported by a framework including scrapping, augmenting and visualization tools and it has been applied to different kinds of websites to demonstrate its validity and feasibility. Our ultimate goal is to expand the limits of our technology for improving the user interaction with websites and creating new experiences for better understanding large data sets.