A First-person Database for Detecting Barriers for Pedestrians

Abstract

Egocentric vision, which relates to the continuous interpretation of images captured by wearable cameras, is increasingly being utilized in several applications to enhance the quality of citizens' life, especially for those with visual or motion impairments. The development of sophisticated egocentric computer vision techniques requires automatic analysis of large databases of first-person point of view visual data collected through wearable devices. In this paper, we present our initial findings regarding the use of wearable cameras for enhancing the pedestrians' safety while walking in city sidewalks. For this purpose, we create a first-person database that entails annotations on common barriers that may put pedestrians in danger. Furthermore, we derive a framework for collecting visual lifelogging data and define 24 different categories of sidewalk barriers. Our dataset consists of 1796 annotated images covering 1969 instances of barriers. The analysis of the dataset by means of object classification algorithms, depict encouraging results for further study.

A Smartphone Application Designed to Detect Obstacles for Pedestrians’ Safety

“…A method for the recognition of obstacles in egocentric data for pedestrians' safety was presented in [15]. The obstacle recognition was based on a CNN which was specifically trained to classify different obstacle types.…”

“…For this purpose, a dedicated annotated dataset was used which consists of images collected using a smartphone camera while a pedestrian walked in urban areas. A total number of 1854 bounding boxes related to 15 Unlike CIT2ADM, the application we present in this work incorporates egocentric vision algorithms for the automatic recognition of barriers and damages, intended to aid users in reporting obstacles more quickly. Additionally, we evaluated the user experience, enabling the derivation of results related to the operation of the application that will allow its improvement to deal effectively with users' needs.…”

“…Additionally, a Machine Learning (ML) algorithm that provides an automated list of obstacle type recommendations was embedded in the application. For this functionality, we utilised a related publicly available CNN model [15] that was tuned using transfer learning, based on a dedicated dataset of typical obstacles found in urban environments [15]. To incorporate the ML model in the application, we used the TensorFlow Lite converter Python library 4 to convert the CNN to the mobile compatible TensorFlow Lite format.…”

“…4c). For a full list of urban obstacle types used, see the list in [15]. In the scenario where the obstacle does not fall under any of the predefined types, the user has the option of entering the obstacle type manually.…”

“…The proposed application allows pedestrians to easily report any obstacle they encounter while walking in a city. A Convolutional Neural Network (CNN), trained specifically for recognising various barriers and damages [15], aids users in quickly choosing the correct type of barrier or damage. The evaluation of the prototype application involves 25 volunteers who tested the application in real conditions and answered an online questionnaire regarding user experience.…”