Sound-based Transportation Mode Recognition with Smartphones

Wang, Lin; Roggen, Daniel

doi:10.1109/icassp.2019.8682917

Cited by 20 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The mono-modal classifiers employed in this paper are adapted directly from our previous work [18], [23], [24], which are comparable to the state of the art. The sound and vision classifiers are among the first works that are applied to transportation mode recognition.…”

Section: Discussionmentioning

confidence: 99%

“…Fig. 5(b) illustrates the deep architecture of the sound-based classifier (T sound ), which we initially developed in [24]. The convolutional neural network consists of an input layer, two CNN and two FCNN blocks, and an output decision block.…”

Section: B Sound Classifiermentioning

confidence: 99%

“…Due to privacy issues, few transportation and locomotion datasets are publicly available with sound and vision modalities. Only a few work has been reported on transportation mode recognition with vision [23] or sound [24]- [26], and to our knowledge no work has addressed the combination of vision or sound with each other and with motion.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Transportation mode recognition fusing wearable motion, sound and vision sensors

et al. 2020

Self Cite

View full text Add to dashboard Cite

We present the first work that investigates the potential of improving the performance of transportation mode recognition through fusing multimodal data from wearable sensors: motion, sound and vision. We first train three independent deep neural network (DNN) classifiers, which work with the three types of sensors, respectively. We then propose two schemes that fuse the classification results from the three mono-modal classifiers. The first scheme makes an ensemble decision with fixed rules including Sum, Product, Majority Voting, and Borda Count. The second scheme is an adaptive fuser built as another classifier (including Naive Bayes, Decision Tree, Random Forest and Neural Network) that learns enhanced predictions by combining the outputs from the three mono-modal classifiers. We verify the advantage of the proposed method with the state-of-the-art Sussex-Huawei Locomotion and Transportation (SHL) dataset recognizing the eight transportation activities: Still, Walk, Run, Bike, Bus, Car, Train and Subway. We achieve F1 scores of 79.4%, 82.1% and 72.8% with the mono-modal motion, sound and vision classifiers, respectively. The F1 score is remarkably improved to 94.5% and 95.5% by the two data fusion schemes, respectively. The recognition performance can be further improved with a post-processing scheme that exploits the temporal continuity of transportation. When assessing generalization of the model to unseen data, we show that while performance is reduced-as expected-for each individual classifier, the benefits of fusion are retained with performance improved by 15 percentage points. Besides the actual performance increase, this work, most importantly, opens up the possibility for dynamically fusing modalities to achieve distinct power-performance trade-off at run time.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Sound Classifiermentioning

confidence: 99%

See 1 more Smart Citation

Transportation mode recognition fusing wearable motion, sound and vision sensors

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…A user often carries a wearable device (e.g smartphone, smartwatch) during travel, which is embedded with multimodal sensors including motion sensors, GPS (global positioning system), microphone and camera. While most work on locomotion and transportation recognition has used motion sensors [1,2,[8][9][10][11][12]14] or sound [4,5,13], our recent work has shown that vision is also an important modality to recognise modes of locomotion and transportation [6]. We showed that 8 activities (Still, Walk, Run, Bike, Car, Bus, Train, Subway) can be recognised with an F1 score of 82.1% for the best classifier.…”

Section: Introductionmentioning

confidence: 85%

Benchmarking deep classifiers on mobile devices for vision-based transportation recognition

Richoz

Pérez-Uribe

Birch

et al. 2019

Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of The

Self Cite

View full text Add to dashboard Cite

Vision-based human activity recognition can provide rich contextual information but has traditionally been computationally prohibitive. We present a characterisation of five convolutional neural networks (DenseNet169, MobileNet, ResNet50, VGG16, VGG19) implemented with TensorFlow Lite running on three state of the art Android mobile phones. The networks have been trained to recognise 8 modes of transportation from camera images using the SHL Locomotion and Transportation dataset. We analyse the effect of thread count and back-ends services (CPU, GPU, Android Neural Network API) to classify the images provided by the rear camera of the phones. We report processing time and classification accuracy. CCS CONCEPTS• Computing methodologies → Activity recognition and understanding; • Theory of computation → Discrete optimization; • Software and its engineering → Designing software.

show abstract

“…The mode of transportation or locomotion is an important contextual that enables applications such as activity and health monitoring, individual environmental impact monitoring, and intelligent service adaptation [16][17][18][19][20][21][22][23][24]. Several prior works looked at recognizing modes of transportation from smartphone sensors, such as motion [25,26], GPS [27][28][29][30][31][32], sound [33], image [34] and the fusion of multiple sensors [35]. The potential of exploiting GSM and WiFi signals for transportation mode recognition has also been reported [36][37][38].…”

Section: Introductionmentioning

confidence: 99%

Locomotion and Transportation Mode Recognition from GPS and Radio Signals: Summary of SHL Challenge 2021

Wang

Ciliberto

Gjoreski

et al. 2021

Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of The

Self Cite

View full text Add to dashboard Cite

In this paper we summarize the contributions of participants to the fourth Sussex-Huawei Locomotion-Transportation (SHL) Recognition Challenge organized at the HASCA Workshop of UbiComp/ISWC 2021. The goal of this machine learning/data science challenge is to recognize eight locomotion and transportation activities (Still, Walk, Run, Bike, Bus, Car, Train, Subway) from the radio sensor data (GPS location, GPS reception, WiFi reception and Cell reception) of a smartphone in a user-independent manner. The training and testing data are collected by different users with a smartphone placed at the Hips position. We introduce the dataset used in the challenge and the protocol of the competition. We present a meta-analysis of the contributions from 15 submissions, their approaches, the software tools used, computational cost and the achieved results. The challenge evaluates the recognition performance by comparing predicted to ground-truth labels at every second, but puts no constraints on the maximum decision window length. Overall, two submissions achieved F1 scores between 70% and 80%, one between 60% and 70%, five between 50% and 60%, and seven below 50%. Due to the technical challenges of data synchronization, sensor unavailability and sensor diversity, the overall performance based on GPS and radio sensors is lower than the performance achieved by motion sensors in previous challenges (SHL 2018(SHL -2020. Finally, we present

show abstract

Sound-based Transportation Mode Recognition with Smartphones

Cited by 20 publications

References 24 publications

Transportation mode recognition fusing wearable motion, sound and vision sensors

Transportation mode recognition fusing wearable motion, sound and vision sensors

Benchmarking deep classifiers on mobile devices for vision-based transportation recognition

Locomotion and Transportation Mode Recognition from GPS and Radio Signals: Summary of SHL Challenge 2021

Contact Info

Product

Resources

About