Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks

Abstract

Abstract-Low-dimensional statistics of measurements play an important role in detection problems, including those encountered in sensor networks. In this work, we focus on learning low-dimensional linear statistics of high-dimensional measurement data along with decision rules defined in the low-dimensional space in the case when the probability density of the measurements and class labels is not given, but a training set of samples from this distribution is given. We pose a joint optimization problem for linear dimensionality reduction and margin-based classification, and develop a coordinate descent algorithm on the Stiefel manifold for its solution. Although the coordinate descent is not guaranteed to find the globally optimal solution, crucially, its alternating structure enables us to extend it for sensor networks with a message-passing approach requiring little communication. Linear dimensionality reduction prevents overfitting when learning from finite training data. In the sensor network setting, dimensionality reduction not only prevents overfitting, but also reduces power consumption due to communication. The learned reduced-dimensional space and decision rule is shown to be consistent and its Rademacher complexity is characterized. Experimental results are presented for a variety of datasets, including those from existing sensor networks, demonstrating the potential of our methodology in comparison with other dimensionality reduction approaches.

A training set reduction algorithm for feed-forward neural network using minimum boundary vector distance selection

“…These techniques, however, do not produce consistent output due to the use of neural network to determine whether or not an input vector should be included in the reduced training set. Researches [9], [10], [11], [12], [13] trade the importances of some features or attributes in the dataset for a smaller training set. They are useless when no feature or attribute in the dataset can be sacrificed.…”

“…The algorithm produces a more consistent output compared with the output from research works [4], [5], [6], [7], [8] because no neural network rule is used to determine whether or not an input vector should be included in the reduced training set. In addition, the algorithm does not sacrifice any feature or dimension of the training set in the reduction process, unlike the techniques from research works [9], [10], [11], [12], [13],…”

Balancing lifetime and classification accuracy of wireless sensor networks

“…So the training samples are acquired under different activity states ω ∈ Ω. Thus studying the relationship between accuracy and lifetime is not simply a matter of joining the corresponding expressions (5) and (12).…”

Mixed H2/H∞-Based Fusion Estimation for Energy-Limited Multi-Sensors in Wearable Body Networks

“…Here, a normal idea is to reduce communication traffic between sensors and the FC at each period. Therefore, two common methods have been used in existing research: the dimensionality reduction method [8,9] and the quantization method [10]. The idea of the dimensionality reduction method is used to convert high-dimensional data into low-dimensional data through specific mechanisms.…”