Adaptive PCA-based feature drift detection using statistical measure

FBDD: feature-based drift detector for batch processing data

“…The PCA-FDD is a technique for detecting changes in streaming data. It involves using principal component analysis (PCA) to detect feature drift in the adjacent chunks of data [11]. The PCA-FDD method determines the principal data components (primary eigenvectors) in adjacent chunks and monitors changes in these components over time.…”

“…In the PCA-FDD method, a predetermined number of principal components is selected, while in the FBDD method, the most crucial feature is determined by the LASSO procedure. The PCA-FDD algorithm was implemented based on [11] and was used in comparative studies.…”

“…In practice, Digits08 and Digits17 data sets were limited to two classes. Courtesy of Prof. M. Kantardzic [14], we obtained from him the original data of these collections, which, among other things, were referred to by other authors [11]. The Digit08 dataset includes 1499 instances with 16 attributes and Digit17 set consists of 1557 instances and 16 attributes.…”

“…• Electricity. The electricity market dataset represents pricing data collected from New South Wales ( Australia), which fluctuates based on the supply and the demand components of the market [7,11,14,16]. It has 45312 instances and 8 attributes.…”

“…However, observing the features of a data stream can provide a better understanding of the nature of data changes than finding observed learning model errors. The existing literature, except for a few cases [1,11,12], has not dealt with a specific type of concept drift, i.e., feature drift, which refers to possible changes in the relevance of data stream features over time. Statistical methods mainly detect feature drift, as they do not require knowledge of sample labels.…”