Efficient extreme learning machine via very sparse random projection

Chen, Chuangquan; Vong, Chi‐Man; Wong, Chi-Man; Wang, Weiru; Wong, Pak-Kin

doi:10.1007/s00500-018-3128-7

Cited by 21 publications

(7 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al [20] proved that very sparse matrix can obtain the similar results with random gaussian matrix, and x can be defined as

or even

. In our work, we define

for building the very sparse measurement matrix.…”

Section: Methodsmentioning

confidence: 99%

Compressed feature vector-based effective object recognition model in detection of COVID-19

Chen

Mao²,

Liu³

et al. 2022

Pattern Recognition Letters

View full text Add to dashboard Cite

To better understand the structure of the COVID-19, and to improve the recognition speed, an effective recognition model based on compressed feature vector is proposed. Object recognition plays an important role in computer vison aera. To improve the recognition accuracy, most recent approaches always adopt a set of complicated hand-craft feature vectors and build the complex classifiers. Although such approaches achieve the favourable performance on recognition accuracy, they are inefficient. To raise the recognition speed without decreasing the accuracy loss, this paper proposed an efficient recognition modeltrained witha kind of compressed feature vectors. Firstly, we propose a kind of compressed feature vector based on the theory of compressive sensing. A sparse matrix is adopted to compress feature vector from very high dimensions to very low dimensions, which reduces the computation complexity and saves enough information for model training and predicting. Moreover, to improve the inference efficiency during the classification stage, an efficient recognition model is built by a novel optimization approach, which reduces the support vectors of kernel-support vector machine (kernel SVM). The SVM model is established with whether the subject is infected with the COVID-19 as the dependent variable, and the age, gender, nationality, and other factors as independent variables. The proposed approach iteratively builds a compact set of the support vectors from the original kernel SVM, and then the new generated model achieves approximate recognition accuracy with the original kernel SVM. Additionally, with the reduction of support vectors, the recognition time of new generated is greatly improved. Finally, the COVID-19 patients have specific epidemiological characteristics, and the SVM recognition model has strong fitting ability. From the extensive experimental results conducted on two datasets, the proposed object recognition model achieves favourable performance not only on recognition accuracy but also on recognition speed.

show abstract

“…Chen et al [20] proved that very sparse matrix can obtain the similar results with random gaussian matrix, and x can be defined as

or even

. In our work, we define

for building the very sparse measurement matrix.…”

Section: Methodsmentioning

confidence: 99%

Compressed feature vector-based effective object recognition model in detection of COVID-19

Chen

Mao²,

Liu³

et al. 2022

Pattern Recognition Letters

View full text Add to dashboard Cite

show abstract

“…This may easily results to the memory-overflow issue in the applications with large and(or) N. Instead of directly calculating −1 using Netwon-Raphson method with the full training data, we propose a Quasi-Newton (QN) method using gradient ∇ ( ) to approximate −1 for SBELM. In QN-SBELM, the approximated −1 can be employed to formula (8) to update the ARD prior and no full training data loaded into memory beforehand required. Therefore, it is scalable to large problems.…”

Section: Inverse-free Sbelmmentioning

confidence: 99%

“…Many ELM variants have been developed to address some particular problems. Examples include ELMs for online sequential learning [3], ELMs for imbalanced problems [4], ELMs for semi-supervised learning [5], ELMs for unsupervised learning [6], ELMs for compressive learning [8], etc. However, as analyzed in [9] [10], most of them still suffer from heavy overfitting and large model sizes for benchmark applications.…”

Section: Introductionmentioning

confidence: 99%

An Inverse-Free and Scalable Sparse Bayesian Extreme Learning Machine for Classification Problems

et al. 2021

Self Cite

View full text Add to dashboard Cite

Sparse Bayesian Extreme Learning Machine (SBELM) constructs an extremely sparse and probabilistic model with low computational cost and high generalization. However, the update rule of hyperparameters(ARD prior) in SBELM involves using the diagonal elements from the inversion of the covariance matrix with the full training dataset, which raises the following two issues. Firstly, inverting the Hessian matrix may suffer ill-conditioning issues in some cases, which hinders SBELM from converging. Secondly, it may result in the memory-overflow issue with computational memory ( 3 ) (L: number of hidden nodes) to invert the big covariance matrix for updating the ARD priors. To address these issues, an inverse-free SBELM called QN-SBELM is proposed in this paper, which integrates the gradient-based Quasi-Newton (QN) method into SBELM to approximate the inverse covariance matrix. It takes( 2 ) computational complexity and is simultaneously scalable to large problems. QN-SBELM was evaluated on benchmark datasets of different sizes. Experimental results verify that QN-SBELM achieves more accurate results than SBELM with a sparser model, and also provides more stable solutions and a great extension to large-scale problems.

show abstract

“…For real word application, we can increase the size of hash vector to improve the performance while Malytics still requires a light computation. Motivated from [45], we replaced the tf -simashing weights (i.e -1 and 1 values) with a sparse matrix including -1, 1 and 0 [46]. We set the sparsity to 1% and the size of tf -simashing vector is 3000.…”

Section: Modelmentioning

confidence: 99%

Malytics: A Malware Detection Scheme

et al. 2018

View full text Add to dashboard Cite

An important problem of cyber-security is malware analysis. Besides good precision and recognition rate, a malware detection scheme needs to be able to generalize well for novel malware families (a.k.a zero-day attacks). It is important that the system does not require excessive computation particularly for deployment on the mobile devices.In this paper, we propose a novel scheme to detect malware which we call Malytics. It is not dependent on any particular tool or operating system. It extracts static features of any given binary file to distinguish malware from benign. Malytics consists of three stages: feature extraction, similarity measurement and classification. The three phases are implemented by a neural network with two hidden layers and an output layer. We show feature extraction, which is performed by tf -simhashing, is equivalent to the first layer of a particular neural network. We evaluate Malytics performance on both Android and Windows platforms. Malytics outperforms a wide range of learning-based techniques and also individual state-of-the-art models on both platforms. We also show Malytics is resilient and robust in addressing zero-day malware samples. The F1-score of Malytics is 97.21% and 99.45% on Android dex file and Windows PE files respectively, in the applied datasets. The speed and efficiency of Malytics are also evaluated.

show abstract

Efficient extreme learning machine via very sparse random projection

Cited by 21 publications

References 34 publications

Compressed feature vector-based effective object recognition model in detection of COVID-19

Compressed feature vector-based effective object recognition model in detection of COVID-19

An Inverse-Free and Scalable Sparse Bayesian Extreme Learning Machine for Classification Problems

Malytics: A Malware Detection Scheme

Contact Info

Product

Resources

About