Dynamic Vision Sensors for Human Activity Recognition

Baby, Stefanie Anna; Vinod, B.; Chinni, Chaitanya; Mitra, Kaushik

doi:10.1109/acpr.2017.136

Cited by 27 publications

(17 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work presents a VLSI architecture for FPGA to accelerate the HOTS algorithm. The system was tested with a gesture recognition dataset [42], obtaining an accuracy loss of 1.2% from the algorithm implemented in [36]. The estimation of the power consumption is 77 mW with Xilinx X-Power, working with a frequency of 100 MHz and implying a 50% toggle rate.…”

Section: Discussionmentioning

confidence: 99%

Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA

Tapiador-Morales

Maro²,

Jiménez-Fernández

et al. 2020

Sensors

View full text Add to dashboard Cite

Neuromorphic vision sensors detect changes in luminosity taking inspiration from mammalian retina and providing a stream of events with high temporal resolution, also known as Dynamic Vision Sensors (DVS). This continuous stream of events can be used to extract spatio-temporal patterns from a scene. A time-surface represents a spatio-temporal context for a given spatial radius around an incoming event from a sensor at a specific time history. Time-surfaces can be organized in a hierarchical way to extract features from input events using the Hierarchy Of Time-Surfaces algorithm, hereinafter HOTS. HOTS can be organized in consecutive layers to extract combination of features in a similar way as some deep-learning algorithms do. This work introduces a novel FPGA architecture for accelerating HOTS network. This architecture is mainly based on block-RAM memory and the non-restoring square root algorithm, requiring basic components and enabling it for low-power low-latency embedded applications. The presented architecture has been tested on a Zynq 7100 platform at 100 MHz. The results show that the latencies are in the range of 1 μ s to 6.7 μ s, requiring a maximum dynamic power consumption of 77 mW. This system was tested with a gesture recognition dataset, obtaining an accuracy loss for 16-bit precision of only 1.2% with respect to the original software HOTS.

show abstract

Section: Discussionmentioning

confidence: 99%

Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA

Tapiador-Morales

Maro²,

Jiménez-Fernández

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…There is only a little work on exploring the neuromorphic data beyond object detection addressing highly semantic applications, such as, multi class action recognition, which still poses an important challenge. As mentioned in Section I, work on using NVS data for HAR is still in early stages [39]- [43]. Most of these methods start with temporally aggregating the polarities into a collection of NVS data frames by considering a non-overlapping time window corresponding to the frame rate of conventional APS cameras.…”

Section: Related Workmentioning

confidence: 99%

“…A time stamp aggregation algorithm is used to create the frames from the events, where these frames are fed into CNN for classification with a 92.90% of accuracy. In [39], three 2D motion maps (on x-y, x-z and y-z planes) and Motion Boundary Histogram (MBH) are constructed from the events. Speeded Up Robust Features (SURF) are extracted through grid search on the 2D motion maps followed by k-means clustering to create a Bag of visual vocabulary (BoVV) of k words from motion maps and MBH.…”

Section: Related Workmentioning

confidence: 99%

“…Most recent works exploring NVS data for human motion analysis consists of low semantic tasks, such as, hand or finger movement analysis [30]- [37] and human fall detection [38]. However, exploring NVS data for higher-level semantic tasks, such as, multi-class HAR, is still in early stages [39]- [43]. One reason for this slow progress of NVS domain HAR is the high cost of NVS devices compared to the conventional APS cameras [8] leading to insufficient annotated NVS domain HAR training datasets [41], [44].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Making Sense of Neuromorphic Event Data for Human Action Recognition

2021

View full text Add to dashboard Cite

Neuromorphic vision sensors provide low power sensing and capture salient spatial-temporal events. The majority of the existing neuromorphic sensing work focus on object detection. However, since they only record the events, they provide an efficient signal domain for privacy aware surveillance tasks. This paper explores how the neuromorphic vision sensor data streams can be analysed for human action recognition, which is a challenging application. The proposed method is based on handcrafted features. It consists of a pre-processing step for removing the noisy events followed by the extraction of handcrafted local and global feature vectors corresponding to the underlying human action. The local features are extracted considering a set of high-order descriptive statistics from the spatio-temporal events in a time window slice, while the global features are extracted by considering the frequencies of occurrences of the temporal event sequences. Then, low complexity classifiers, such as, support vector machines (SVM) and K-Nearest Neighbours (KNN), are trained using these feature vectors. The proposed method evaluation uses three groups of datasets: Emulator-based, re-recording-based and native NVS-based. The proposed method has outperformed the existing methods in terms of human action recognition accuracy rates by 0.54%, 19.3% and 25.61% for E-KTH, E-UCF11 and E-HMDB51 datasets, respectively. This paper also reports results for three further datasets: E-UCF50, R-UCF50, and N-Actions, which are reported for the first time for human action recognition on neuromorphic vision sensor domain.INDEX TERMS Neuromorphic vision sensing (NVS), event cameras, dynamic vision sensing (DVS), Human action recognition (HAR), Local features, Global features,

show abstract

“…Consequently, k-fold CV could overestimate the performance of the classification by relying on correlations within subjects. Stefanie Anna Baby [5] proposed a model which includes a dynamic vision sensor, it processes only the foreground objects that being the humans captured the camera while completely ignoring the unnecessary processing of backgrounds thus decreasing processing requirements and greatly enhancing the detection efficiency of the given model. The difference in intensity of every pixel solely depends upon the texture edge and hence the output extracted from DVS is very sparse.…”

Section: Introductionmentioning

confidence: 99%

Human Activity Recognition in Video Surveillance – A Survey

Urankar¹

2020

IJRASET

View full text Add to dashboard Cite

The act of detection of the activity of humans has gained traction nowadays and is a hot field for research and development. The ability of human activity detection (HAR) is a key prospect of development in fields of human-computer interface, computer vision and in mass video surveillance in public areas such as train stations, ATM machines, schools and colleges and at traffic signals on streets. HAR offers a cost-effective method for surveillance. Analysis based on methods used for classification for different datasets is presented in this survey paper.

show abstract

Dynamic Vision Sensors for Human Activity Recognition

Cited by 27 publications

References 20 publications

Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA

Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA

Making Sense of Neuromorphic Event Data for Human Action Recognition

Human Activity Recognition in Video Surveillance – A Survey

Contact Info

Product

Resources

About