Kd-tree and adaptive radius (KD-AR Stream) based real-time data stream clustering

Abstract

Highlights:Graphical/Tabular Abstract  Fully online data stream clustering  Evolutionary based clustering  Adaptive radius  Time based summarization  Memory for past status of clusters Figure A. Comparision of clustering quality and run-time complexity of algorithms on KDD datasetPurpose: The aim of this article to propose a new data stream clustering algorithm, which has an adaptive radius, can adapt itself to the evolutionary structure of streaming data and works in a fully online manner. Theory and Methods:In this study, kd-tree is used to forming and splitting clusters, adaptive radius approach is used to support increasing and decreasing the size of clusters, active/inactive status of clusters is used to adapt to the evolutionary structure of streaming data and all the operations are done online. In order to create a new cluster, the data that does not belong to any cluster are placed in a kd-tree, and the rangesearch operation is performed on those data according to predefined variables r (the radius of candidate cluster) and N (the number of data must be in the area). After forming the clusters, the radius of each cluster could be increased or decreased over time if necessary. Some clusters may be split and some may be merged over time because of dynamically changing structure of streaming data. Inactivation and reactivation of the status of clusters is used to allow for the identification of clusters formed in the same region at a different time interval with same cluster labels in accordance with the nature of the streaming data contrary to literature. This feature increases clustering quality of the proposed method. A summarization method that consist of time window and sliding window is used to support time based summarization without reduce performance. Results:To verify the effectiveness of KD-AR Stream algorithm, it is compared with SE-Stream, DPStream, and CEDAS on a variety of well-known datasets in terms of clustering quality and run-time complexity. The results show that KD-AR Stream outperforms other algorithms with a higher clustering success in a reasonable time as shown in Fig. A. Conclusion:The aim of this study is to propose a novel data stream clustering algorithm that adapts to the dynamic structure of the streaming data. The aim achieved by using the five evolutionary process which are appearance, activation/inactivation, self-evolution, merge, and split. According to the results, the proposed method is very successful in terms of clustering quality and run-time complexity.

Akan veri kümeleme probleminde ağaç veri yapılarının performans karşılaştırması

“…n işlenen veri sayısı, d her verinin sahip olduğu nitelik sayısı ve k ise geri döndürülen veri sayısı olmak üzere önerdiğimiz algoritmaların kullandığı ağaç oluşturma ve alan arama algoritmalarına ait çalışma zamanları Tablo 1'de verilmiştir. [29] 2017 Benzer 495020 Mikro-küme yaklaşımı 0,2415 pcStream2 [30] 2017 Aynı 50.000 Temel bileşenler analizi 0,7625 KD-AR Stream [37] 2019 Aynı 50.000 K-boyutlu ağaç 0,6818 TSHC [31] 2022 Yazarlar, bu çalışmanın yapılmasında desteklerini esirgemeyen Prof. Dr. Hacer Karacan'a teşekkür eder.…”

“…Bu çalışmada akan veri kümeleme için ağaç tabanlı iki yeni algoritma geliştirilerek uygulanmıştır. Geliştirilen bu algoritmalar daha önce Şenol ve Karacan tarafından önerilen KD-AR Stream [37] algoritmasından esinlenmektedir. Yapılan deneysel çalışmalarda, akan veri kümeleme için farklı sayıda kayıt ve boyutlara sahip 18 açık veri kümesi kullanılmıştır.…”

ImpSlidingWindow: Kayan Pencere Tabanlı Akan Veri Özetleme Yönteminin Performansını Arttırmaya Yönelik Yeni Bir Model

“…Akan veri kümeleme alanında pek çok çalışma önerilmiştir. STREAM (O'Callaghan, Mishra, Meyerson, Guha ve Motwani, 2002), CluStream (Aggarwal, Han, Wang ve Yu, 2003), STING (Wang, Yang ve Muntz, 1997), D-Stream (Tu ve Chen, 2009), MR-Stream (Wan, Ng, Dang, Yu ve Zhang, 2009), ClusTree (Kranen, Assent, Baldauf ve Seidl, 2011), HPStream (Charu, Jiawei, Jianyong ve Philip, 2004), DUCstream (Gao, Li, Zhang ve Tan, 2005), DenStream (Cao, Estert, Qian ve Zhou), E-Stream (Udommanetanakit, Rakthanmanon ve Waiyamai, 2007), SE-Stream (Chairukwattana, Kangkachit, Rakthanmanon ve Waiyamai, 2013), DD-Stream (Jia, Tan ve Yong, 2008), STREAMKM++ (Ackermann, Martens, Raupach, Swierkot, Lammersen ve Sohler, 2012), HDDStream (Ntoutsi, Zimek, Palpanas, Kröger ve Kriegel, 2012), LeaDen-Stream (Amini ve Wah, 2013), DBSTREAM (Hahsler ve Bolaños, 2016), FEAC-Stream (Silva, Hruschka ve Gama, 2017), DPStream (Xu, Wang, Li, Deng ve Gou, 2017), CEDAS (Hyde, Angelov ve MacKenzie, 2017), LLDStream (Laohakiat, Phimoltares ve Lursinsap, 2017), BOCEDS (Ahmed, 2019), StreamSW (Reddy ve Bindu, 2018) ve KD-AR Stream (Şenol ve Karacan, 2019) bunlardan bazıları olarak sayılabilir.…”

“…, CEDAS(Hyde, Angelov ve MacKenzie, 2017), LLDStream(Laohakiat, Phimoltares ve Lursinsap, 2017), BOCEDS(Ahmed, 2019), StreamSW(Reddy ve Bindu, 2018) ve KD-AR Stream(Şenol ve Karacan, 2019) bunlardan bazıları olarak sayılabilir.Akan veri kümeleme yaklaşımlarının odaklandığı temel noktalardan biri veriyi hızlı bir şekilde işlemektir. Çünkü teknolojinin gelişmesi ile beraber çok hızlı bir veri akışı söz konsudur.…”