An Architecture for Streaming Coclustering in High Speed Hardware

Abstract

We seek to learn the semantics of a data stream at optical line speed. 1,2 We focus on text data, but the techniques developed should apply to broad modalities of network data wherever appropriate features can be computed rapidly enough. We consider a custom hardware system designed to categorize documents based on feature clusters and document clusters that have been learned offline on standard general-purpose computers, and we present a technique for extending this system to permit online learning from arbitrarily large data sets.

Streaming Hierarchical Clustering for Concept Mining

“…As an extension to our prior Aerospace conference papers [1,2,3,16] that describe a system for extracting semantic content from unstructured text document streams, we have developed a clustering system. The implemented system, streaming hierarchical partitioning, hierarchically clusters streaming content.…”

“…As part of an ongoing research project, we have developed a novel algorithmic approach for extracting semantic content from voluminous data streams [1][2][3]. The approach is applicable to multilingual documents and multiple encodings, which can be automatically identified and converted into a common structure [4].…”

“…Thus we have developed novel, hardware-accelerated approaches to detecting known and unknown content, at line speeds [1][2][3].…”