Debellor: A Data Mining Platform with Stream Architecture

Abstract

Abstract. This paper introduces Debellor (www.debellor.org) -an open source extensible data mining platform with stream-based architecture, where all data transfers between elementary algorithms take the form of a stream of samples. Data streaming enables implementation of scalable algorithms, which can efficiently process large volumes of data, exceeding available memory. This is very important for data mining research and applications, since the most challenging data mining tasks involve voluminous data, either produced by a data source or generated at some intermediate stage of a complex data processing network.Advantages of data streaming are illustrated by experiments with clustering time series. The experimental results show that even for moderatesize data sets streaming is indispensable for successful execution of algorithms, otherwise the algorithms run hundreds times slower or just crash due to memory shortage.Stream architecture is particularly useful in such application domains as time series analysis, image recognition or mining data streams. It is also the only efficient architecture for implementation of online algorithms.The algorithms currently available on Debellor platform include all classifiers from Rseslib and Weka libraries and all filters from Weka.

Towards Online Data Mining System for Enterprises

“…From the point of view of software solutions designed for stream processing and analysis, we have identified two groups. The first one is represented by research projects like Stream Mill Miner (Thakkar et al, 2011) or Debellor (Wojnarski, 2008) that focus on data stream mining. In the second group there are so called Data Stream Management Systems (DSMSs) which are designed for receiving data streams from external sources, their processing and pushing results to the output.…”