No abstract

No abstract

Real time anomaly detection is very popular topic nowadays this because the number of data generated every day is larger and larger. Facing with the phenomena of Big Data is not an easy task. The main aim of this research is to fine appropriate architecture for real-time big data analytic and its main task is to detect anomalies in this real-time data. In this paper we show the implementation of anomaly detection algorithm in real time infrastructure in order to find anomalies as soon as possible. We have proposed architecture for real time anomaly detection by adding some new components and the main part of the infrastructure is Timelion which enable implementation of different algorithms for anomaly detection. The research is focused to develop infrastructure to monitor ednevnik (education national system in Macedonia) application server and to detect errors in order to scale up the performance.

Anomaly detection is very important in every sector as health, education, business, etc. Knowing what is going wrong with data/digital system help peoples from every sector to take decision. Detection anomalies in real time Big Data is nowadays very crucial. Dealing with real time data requires speed, for this reason the aim of this paper is to measure the performance of our previously proposed HW-GA algorithm compared with other anomaly detection algorithms. Many factors will be analyzed which may affect the performance of HW-GA as visualization of result, amount of data and performance of computers. Algorithm execution time and CPU usage are the parameters which will be measured to evaluate the performance of HW-GA algorithm. Also, another aim of this paper is to test the HW-GA algorithm with large amount of data to verify if it will find the possible anomalies and the result to compare with other algorithms. The experiments will be done in R with different datasets as real data Covid-19 and e-dnevnik data and three benchmarks from Numenta datasets. The real data have not known anomalies but in the benchmark data the anomalies are known this is in order to evaluate how the algorithms work in both situations. The novelty of this paper is that the performance will be tested in three different computers which one of them is high performance computer.

No abstract