Event Augmentation for Contact Force Measurements

Naeini, Fariborz Baghaei; Kachole, Sanket; Muthusamy, Rajkumar; Makris, Dimitrios; Zweiri, Yahya

doi:10.1109/access.2022.3224584

Cited by 9 publications

(9 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This mixing technique enhances the diversity of the training data and has been evaluated on a set of event-based recognition benchmarks as well. Naeini et al [58] propose spatial, noise, and time-series augmentations to improve contact-force estimation. Spatial augmentations include rotations and resizing.…”

Section: Augmentation Methods For Event-based Visionmentioning

confidence: 99%

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

El Shair,

Hassani,

Rawashdeh

2023

IEEE Access

View full text Add to dashboard Cite

Event-based vision is a novel perception modality that offers several advantages, such as high dynamic range and robustness to motion blur. In order to process events in batches and utilize modern computer vision deep-learning architectures, an intermediate representation is required. Nevertheless, constructing an effective batch representation is non-trivial. In this paper, we propose a novel representation for event-based vision, called the compact spatio-temporal representation (CSTR). The CSTR encodes an event batch's spatial, temporal, and polarity information in a 3-channel image-like format. It achieves this by calculating the mean of the events' timestamps in combination with the event count at each spatial position in the frame. This representation shows robustness to motion-overlapping, high event density, and varying event-batch durations. Due to its compact 3-channel form, the CSTR is directly compatible with modern computer vision architectures, serving as an excellent choice for deploying event-based solutions. In addition, we complement the CSTR with an augmentation framework that introduces randomized training variations to the spatial, temporal, and polarity characteristics of event data. Experimentation over different object and action recognition datasets shows that the CSTR outperforms other representations of similar complexity under a consistent baseline. Further, the CSTR is made more robust and significantly benefits from the proposed augmentation framework, considerably addressing the sparseness in event-based datasets.

show abstract

Section: Augmentation Methods For Event-based Visionmentioning

confidence: 99%

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

El Shair,

Hassani,

Rawashdeh

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Furthermore, the downtime between consecutive frames can contain information that is of crucial importance to precision machining. To alleviate these issues, neuromorphic event-based cameras are being employed for vision-based tactile sensing [10,15,17,[32][33][34][35][36]. Neuromorphic cameras (also known as event-based cameras) are a relatively new technology, first introduced in [37], that aims to mimic how the human eye works.…”

Section: Neuromorphic Vision-based Tactile Sensingmentioning

confidence: 99%

“…To our knowledge, the authors do not provide an algorithm for obtaining the markers' position beyond the case where the sensor is resting. Fariborz et al [33,34] use Conv-LSTMs on event-frames constructed from event streams to estimate contact forces. Faris et al [18] uses CNN over accumulated event heatmaps to detect slip.…”

Section: Challenges With Event-based Vision and Existing Solutionsmentioning

confidence: 99%

TactiGraph: An Asynchronous Graph Neural Network for Contact Angle Prediction Using Neuromorphic Vision-Based Tactile Sensing

Sajwani¹,

Ayyad²,

Alkendi³

et al. 2023

Preprint

View full text Add to dashboard Cite

Vision-based tactile sensors (VBTS) have become the de facto method of giving robots the ability to obtain tactile feedback from their environment. Unlike other solutions to tactile sensing, VBTS offers high spatial resolution feedback without compromising on instrumentation costs or incurring additional maintenance expenses. However, conventional cameras used in VBTS have a fixed update rate and output redundant data, leading to computational overhead downstream. In this work, we present a neuromorphic vision-based tactile sensor (N-VBTS) that employs observations from an event-based camera for contact angle prediction. Particularly, we design and develop a novel graph neural network, dubbed TactiGraph, that asynchronously operates on graphs constructed from raw N-VBTS streams exploiting their spatiotemporal correlations to perform predictions. Although conventional VBTS uses an internal illumination source, TactiGraph is reported to perform efficiently in both scenarios, with and without an internal illumination source. Rigorous experimental results revealed that TactiGraph achieved a mean absolute error of 0.62∘ in predicting the contact angle and was faster and more efficient than both conventional VBTS and other N-VBTS, with lower instrumentation costs. Specifically, N-VBTS requires only 5.5% of the compute-time needed by VBTS when both are tested on the same scenario.

show abstract

“…The position and orientation of the DAVIS sensor are carefully chosen to facilitate this integration. A similar setup has been used and validated by several recent studies in the field 4,6,20,41,44 Conducting the above experiment on a robot for smart grasping under the above conditions is costly and time-consuming. The setup itself requires a lot of time and resources, making it expensive.…”

Section: Neuromorphic Grasping Frameworkmentioning

confidence: 99%

Efficient Event-Based Robotic Grasping Perception using Hyperdimensional Computing

Hassan,

Zou,

Chen

et al. 2023

Preprint

View full text Add to dashboard Cite

Grasping is Fundamental in various robotic applications, particularly within industrial contexts. Accurate inference of object properties is a crucial step toward enhancing grasping quality. Dynamic and Active Vision Sensors (DAVIS), increasingly utilized for robotic grasping, offer superior energy efficiency, lower latency, and higher temporal resolution than traditional cameras. However, the data they generate can be complex and noisy, necessitating substantial pre-processing. In response to these challenges, we introduce GraspHD, an innovative end-to-end algorithm that leverages brain-inspired hyperdimensional computing (HDC) to learn about the size and hardness of objects and estimate the grasping force. This novel approach circumvents the need for resource-intensive pre-processing steps, capitalizing on the simplicity and inherent parallelism of HDC operations. Our comprehensive analysis reveals that GraspHD surpasses state-of-the-art approaches in terms of overall classification accuracy. We have also implemented GraspHD on an FPGA to evaluate system efficiency. The results demonstrate that GraspHD operates at a speed 10x faster and offers an energy efficiency 26 times higher than existing learning algorithms, all while maintaining robust performance in noisy environments. These findings underscore the significant potential of GraspHD as a more efficient and effective solution for real-time robotic grasping applications.

show abstract

Event Augmentation for Contact Force Measurements

Cited by 9 publications

References 31 publications

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

TactiGraph: An Asynchronous Graph Neural Network for Contact Angle Prediction Using Neuromorphic Vision-Based Tactile Sensing

Efficient Event-Based Robotic Grasping Perception using Hyperdimensional Computing

Contact Info

Product

Resources

About