A Hybrid Neuromorphic Object Tracking and Classification Framework for Real-Time Systems

Ussa, Andres; Senthil, Rajen, Chockalingam; Singla, Deepak; Acharya, Jyotibdha; Chuanrong, Gideon Fu; Basu, Arindam; Ramesh, Bharath

doi:10.1109/tnnls.2023.3243679

Cited by 10 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RVTs also provide quick inference (¡12 ms on a T4 GPU) [29]. Researchers used a C++ implementation to demonstrate a continuous-time tracker that better utilizes the asynchronous and low latency characteristics of neuromorphic vision sensors by processing each event separately [30]. Researchers solved the issue of optimum surveillance by demonstrating an anti-occlusion framework based on imaging techniques [31].…”

Section: Literature Reviewmentioning

confidence: 99%

YOLO/SBHK-Net: Occlusion Aware Robot Vision Neural Network with Compound Model Scaling and Re-parameterized Convolution

Banerjee,

Kabir

2024

Preprint

View full text Add to dashboard Cite

Robot Vision is the technique of enabling robots to process visual data from the environment by utilizing a combination of camera hardware and computer algorithms. Advanced deep neural networks have significantly played a vital role in indulging robots to make more sense out of complex visual data at different circumstances, especially in object detection and continuous tracking. In this research, we initiated a unique and cutting-edge backbone neural network for the conventional YOLO algorithm which we named as SBHK-Net. The network boosted up the performance of the existing YOLO algorithm drastically which manifests a strong potential of improving tracking and recognition accuracies of other conventional algorithms in the robot vision industry as well. It has the greatest accuracy 59.2% AP among all known real-time object detectors with 30 FPS or above on GPU RTX3060, and it outperforms all other known object detectors in the range of 5 FPS to 160 FPS. We used YOLOv7 as our reference point for the core research. The transformer-based detector SWINL Cascade-Mask R-CNN (9.2 FPS A100, 53.9% AP) and the convolutional detector ConvNeXt-XL Cascade-Mask R-CNN (8.6 FPS A100, 55.2% AP) are both outperformed by the SBHK-Net core object detector (56 FPS RTX3060, 56.4% AP) in terms of speed and accuracy, respectively. In terms of speed and accuracy, it surpasses a number of other object detectors, including DINO-5scale-R50, ViT-Adapter-B, Scaled-YOLOv4, YOLOv5, DETR, Deformable DETR, and YOLOR and YoLOX. The source code of this research is available at https://github.com/ac005sheekar/SBHKNet.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

YOLO/SBHK-Net: Occlusion Aware Robot Vision Neural Network with Compound Model Scaling and Re-parameterized Convolution

Banerjee,

Kabir

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The recommended technique outperforms previous and current approaches on two MOT challenge datasets. Ussa et al [22] offer a real-time, hybrid neuromorphic architecture for object tracking and categorizing low-power, embedded devices. Hybrid frames and event strategies save energy and increase performance.…”

Section: Literature Reviewmentioning

confidence: 99%

Trackez: An IoT-Based 3D-Object Tracking From 2D Pixel Matrix Using Mez and FSL Algorithm

et al. 2023

View full text Add to dashboard Cite

The imaging devices sense light reflected from objects and reconstruct images using the 2D-sensor matrix. It is a 2D Cartesian coordinate system where the depth dimension is absent. The absence of a depth axis on 2D images imposes challenges in locating and tracking objects in a 3D environment. Real-time object tracking faces another challenge imposed by network latency. This paper presents the development and analysis of a real-time, real-world object tracker called Trackez, which is capable of tracking within the top hemisphere. It uses Machine Vision at the IoT Edge (Mez) technology to mitigate latency sensitivity. A novel algorithm, Follow-Satisfy-Loop (FSL), has been developed and implemented in this paper that optimally tracks the target. It does not require the depth-axis. The simple and innovative design and incorporation of Mez technology have made the proposed object tracker a latency-insensitive, Z-axis-independent, and effective system. The Trackez reduces the average latency by 85.08% and improves the average accuracy by 81.71%. The object tracker accurately tracks objects moving in regular and irregular patterns at up to 5.4ft/s speed. This accurate, latency tolerant, and Z-axis independent tracking system contributes to developing a better robotics system that requires object tracking.

show abstract

“…The energy efficiency of neuromorphic systems is being improved through the development of new circuit designs, low-power device technologies, and architectural optimizations. 279–281 Another area that requires improvement is the attainment of high-fidelity and precision in neural computation. Even while existing neuromorphic hardware can mimic the essential functions of synapses and neurons, there are still disparities between them and biological systems in terms of behavior and reaction.…”

Section: Challenges In Neuromorphic Processors Between Expectations A...mentioning

confidence: 99%

Computing of neuromorphic materials: an emerging approach for bioengineering solutions

Prakash,

Gupta,

Mehta

et al. 2023

Mater. Adv.

View full text Add to dashboard Cite

show abstract

A Hybrid Neuromorphic Object Tracking and Classification Framework for Real-Time Systems

Cited by 10 publications

References 36 publications

YOLO/SBHK-Net: Occlusion Aware Robot Vision Neural Network with Compound Model Scaling and Re-parameterized Convolution

YOLO/SBHK-Net: Occlusion Aware Robot Vision Neural Network with Compound Model Scaling and Re-parameterized Convolution

Trackez: An IoT-Based 3D-Object Tracking From 2D Pixel Matrix Using Mez and FSL Algorithm

Computing of neuromorphic materials: an emerging approach for bioengineering solutions

Contact Info

Product

Resources

About