the theory of random vector functional link network (RVFLN) has provided a breakthrough in the design of neural networks (NNs) since it conveys solid theoretical justification of randomized learning. Existing works in RVFLN are hardly scalable for data stream analytics because they are inherent to the issue of complexity as a result of the absence of structural learning scenarios. A novel class of RVLFN, namely parsimonious random vector functional link network (pRVFLN), is proposed in this paper. pRVFLN adopts a fully flexible and adaptive working principle where its network structure can be configured from scratch and automatically generated in accordance with nonlinearity and time-varying property of system being modelled. pRVFLN is equipped with complexity reduction scenarios where inconsequential hidden nodes can be pruned and input features can be dynamically selected. pRVFLN puts into perspective an online active learning mechanism which expedites the training process and relieves operator's labelling efforts. In addition, pRVFLN introduces a non-parametric type of hidden node, developed using an interval-valued data cloud. The hidden node completely reflects the real data distribution and is not constrained by a specific shape of the cluster. All learning procedures of pRVFLN follow a strictly single-pass learning mode, which is applicable for online time-critical applications. The advantage of pRVFLN was verified through numerous simulations with real-world data streams. It was benchmarked with recently published algorithms where it demonstrated comparable and even higher predictive accuracies while imposing the lowest complexities. Furthermore, the robustness of pRVFLN was investigated and a new conclusion is made to the scope of random parameters where it plays vital role to the success of randomized learning.
I. IntroductionFor decades, research in artificial neural networks has mainly investigated the best way to determine network free parameters, which reduces error as close as possible to zero [18]. Various approaches [55], [56] were proposed, but a large volume of work is based on a first or second-order derivative approach in respect to the loss function [19], [36]. Due to the rapid technological progress in data storage, capture, and transmission [34], the machine learning community has encountered an information explosion, which calls for scalable data analytics. Significant growth of the problem space has led to a scalability issue for conventional machine learning approaches, which require iterating entire batches of data over multiple epochs. This phenomenon results in a strong demand for a simple, fast machine learning algorithm to be well-suited for deployment in numerous data-rich applications [58]. This provides a strong case for research in the area of randomized neural networks (RNNs) [10], [13], [52], which was very popular in late 80's and early 90's. RNNs offer an algorithmic framework, which allows them to generate most of the network parameters randomly while still retaining reaso...