Redundant Feature Selection for Telemetry Data

Abstract

Abstract. Feature sets in many domains often contain many irrelevant and redundant features, both of which have a negative effect on the performance and complexity of agents that use the data [8]. Supervised feature selection aims to overcome this problem by selecting features that are highly related to the class labels, yet unrelated to each other. One proposed technique to select good features with few inter-dependencies is minimal Redundancy Maximal Relevance (mRMR) [11], but this can be impractical with large feature sets. In many situations, features are extracted from signal data such as vehicle telemetry, medical sensors, or financial time-series, and it is possible for feature redundancies to exist both between features extracted from the same signal (intra-signal), and between features extracted from different signals (inter-signal). We propose a two stage selection process to take advantage of these different types of redundancy, considering intra-signal and inter-signal redundancies separately. We illustrate the process on vehicle telemetry signal data collected in a driver distraction monitoring project. We evaluate it using several machine learning algorithms: Random Forest; Naïve Bayes; and C4.5 Decision Tree. Our results show that this two stage process significantly reduces the computation required because of inter-dependency calculations, while having minimal detrimental effect on the performance of the feature sets produced.

Investigating the Feasibility of Vehicle Telemetry Data as a Means of Predicting Driver Workload

“…We apply supervised feature selection to choose the features from this full set, and in particular we use Symmetric Uncertainty (SU) (Witten et al, 2011) and minimal Redundancy Maximal Relevancy (mRMR) selection (Peng, Long, & Ding, 2005;Hermana et al, 2013;Taylor et al, 2014). SU is a variant of Mutual Information (MI), that is normalized by the mean entropy of the two variables to mitigate the bias MI has towards features of high dimensionalities.…”

“…This is repeated until a given number of features is chosen from the set. In this paper, as in Taylor et al (2014), we first select one extracted feature from each signal, before combining them in a second stage of selection. In the classification evaluations first fifteen selected features are used.…”

“…TheDMDcontainsnumerousfeaturesthatareeitherirrelevanttothepredictivetaskorredundantto others.Bothirrelevanceandredundancyinafeaturesethaveanegativeeffectontheperformance andcomplexityofmodelsbuiltondata (Kohavi&John,1997;Guyon&Elisseeff,2003;Hermanaet al,2013).Thedoorlockstatusisunlikelytoprovideanyinsightintotheworkloadlevelofthedriver, forexample,andenginespeedishighlyredundanttothevehiclespeed.Forsimplicity,thesignalsin theDMDwerereducedbyhandtothe30listedinTable6.Thismeantthattherewere3480features intotalthatcouldpotentiallybeusedtobuildamodel,afterfeatureextraction. Supervisedfeatureselectionwasappliedinchoosingthefeaturesfromthisfullset.Inparticular, SymmetricUncertainty(SU)(Wittenetal.,2011)andminimalRedundancyMaximalRelevancy (mRMR) selection (Peng, Long, & Ding, 2005;Hermana et al, 2013;Taylor et al, 2014) were used.SUisavariantofMutualInformation(MI),thatisnormalisedbythemeanentropyofthetwo variablestomitigatethebiasMIhastowardsfeaturesofhighdimensionalities.MIisdefinedas:…”