WebFrame: In Pursuit of Computationally and Cognitively Efficient Web Mining

Abstract

Abstract. The goal of web mining is relatively simple: provide both computationally and cognitively efficient methods for improving the value of information to users of the WWW. The need for computational efficiency is well-recognized by the data mining community, which sprung from the database community concern for efficient manipulation of large datasets. The motivation for cognitive efficiency is more elusive but at least as important. In as much as cognitive efficiency can be informally construed as ease of understanding, then what is important is any tool or technique that presents cognitively manageable abstractions of large datasets. We present our initial development of a framework for gathering, analyzing, and redeploying web data. Not dissimilar to conventional data mining, the general idea is that good use of web data first requires the careful selection of data (both usage and content data), the deployment of appropriate learning methods, and the evaluation of the results of applying the results of learning in a web application. Our framework includes tools for building, using, and visualizing web abstractions. We present an example of the deployment of our framework to navigation improvement. The abstractions we develop are called Navigation Compression Models (NCMs), and we show a method for creating them, using them, and visualizing them to aid in their understanding.

A taxonomy of sequential pattern mining algorithms

“…This entails having a model that can infer semantic relationships from domain knowledge obtained through an ontology or as annotations in the sequence and avoid support counting if a candidate sequence is semantically not likely to occur. A number of Ph.D. dissertations propose a web usage-mining framework which integrates usage, structure and content into the mining process for more directed constrained mining (e.g., Zheng [2004], Tanasa [2005], and Jin [2006]), but do not utilize domain knowledge in the mining process.…”

“…We think that sampling should be given more attention in pattern-growth and early-pruning algorithms as a way to reduce search space and processing time for mining. An open research area is in investigating ways to sample sequences and partition the search space based on the Fibonacci Sequence [Zill 1998] as a guide for sampling or partitioning of sequences. Another way to reduce the search space while mining sequential patterns is to focus on concise representations of sequences, such as mining maximal or closed sequences.…”

Visual Data Mining of Web Navigational Data

“…We apply the methods of our previous work [21] to generate the usage data record and user sessions from web access logs. As a default, WebViz uses node size to represent the page visit count, node colour to indicate the average page view time, edge thickness to show the hyperlink usage count, and edge colour to represent the usage count percentage of a hyperlink (out of total count of the hyperlinks that share the same start page).…”

“…The data preparation [21] consists of crawling the website structure, cleaning irrelevant records, and breaking the access log into user sessions. In order to cut usage records into sessions, we identify users by their authentication or cookie value, then exclude all other…”

“…In our work on the development of web navigation tools (e.g., Navigation Compression Models [21]), and the development of algebras to manipulate and interpret dynamic usage data [4], we have found that the volume of dynamic data is so large that we had to develop methods that not just provide visual representation of compressed data, but also provide insight into the machine learning methods that we use to extract structure from that data. Our visualization system, called WebViz, combines a number of visualization and visual manipulation techniques from a broad range of existing web mining and visualization research.…”