Weakly Supervised Learning for Evaluating Road Surface Condition from Wheelchair Driving Data

Abstract

Providing accessibility information about sidewalks for people with difficulties with moving is an important social issue. We previously proposed a fully supervised machine learning approach for providing accessibility information by estimating road surface conditions using wheelchair accelerometer data with manually annotated road surface condition labels. However, manually annotating road surface condition labels is expensive and impractical for extensive data. This paper proposes and evaluates a novel method for estimating road surface conditions without human annotation by applying weakly supervised learning. The proposed method only relies on positional information while driving for weak supervision to learn road surface conditions. Our results demonstrate that the proposed method learns detailed and subtle features of road surface conditions, such as the difference in ascending and descending of a slope, the angle of slopes, the exact locations of curbs, and the slight differences of similar pavements. The results demonstrate that the proposed method learns feature representations that are discriminative for a road surface classification task. When the amount of labeled data is 10% or less in a semi-supervised setting, the proposed method outperforms a fully supervised method that uses manually annotated labels to learn feature representations of road surface conditions.

Classification of the Sidewalk Condition Using Self-Supervised Transfer Learning for Wheelchair Safety Driving

“…However, studies from [ 17 , 18 , 19 , 20 ] mainly focused on detecting the damages of a motorcar road and required a smartphone to be installed on the dashboard of a vehicle, thus the method is not suitable for wheelchair users. In contrast, Watanabe et al and Iwasawa et al [ 21 , 22 ] tried to recognize the status of the sidewalk by using acceleration data. However, this kind of data is not only largely sensitive to the outdoor conditions and inherent vibrations/noises of a wheelchair but also not feasible to provide users with intuitive information about the defects observed.…”

“…With a recent growth of computer vision and machine learning technologies, there have been various attempts to automatically detect and report defects on roads and sidewalks. Previous approaches primarily captured RGB road images or sensor data (e.g., accelerometer and gyroscope) and exploited deep learning and machine learning algorithms for both detecting road cracks/potholes [ 17 , 18 , 19 , 20 ] and recognizing sidewalk anomalies [ 21 , 22 ]. These methods can automatically detect the defects on the road surface but still have the following limitations: (1) the captured RGB images are not helpful to classify the road condition under low-light conditions (e.g., nighttime) and (2) sensors can produce noisy data or restrict the user’s natural movements, adversely affecting the overall performance.…”

Wheelchair Behavior Recognition for Visualizing Sidewalk Accessibility by Deep Neural Networks

“…This section introduces our weakly supervised method to extract representations of road surface conditions [Watanabe et al, 2020]. Our method uses positional information collected while driving as low-cost weak supervision to learn road surface conditions and does not depend on human annotations.…”

Stetho Touch: Touch Action Recognition System by Deep Learning with Stethoscope Acoustic Sensing

“…The kernel sizes of each convolutional and pooling layer were nine, nine, six, six, and three, and the convolution depth (depth size) is 96, 96, 48, 48, 32 steps, stride size was one. This network was based on Evaluating Road Surface Condition from Wheelchair Driving Data [53].…”