milliEye

Shuai, Xian; Shen, Yuh Chiang; Tang, Yu; Shi, Shuyao; Ji, Luping; Xing, Guoliang

doi:10.1145/3450268.3453532

Cited by 49 publications

(18 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…And the accuracy of 3D object detection is measured by the F1 score, i.e. the harmonic mean of precision and recall, which is widely adopted in recent work [19,28,61,75,77]. The reason why we do not use AP (Average Precision) is that it requires the confidence score to calculate, which is only available in the output of DNN-based detectors.…”

Section: Methodsmentioning

confidence: 99%

2D-Empowered 3D Object Detection on the Edge

Li¹,

Cai²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

3D object detection has a pivotal role in a wide range of applications, most notably autonomous driving and robotics. These applications are commonly deployed on edge devices to promptly interact with the environment, and often require near real-time response. With limited computation power, it is challenging to execute 3D detection on the edge using highly complex neural networks. Common approaches such as offloading to the cloud brings latency overheads due to the large amount of 3D point cloud data during transmission. To resolve the tension between wimpy edge devices and compute-intensive inference workloads, we explore the possibility of transforming fast 2D detection results to extrapolate 3D bounding boxes. To this end, we present Moby, a novel system that demonstrates the feasibility and potential of our approach. Our main contributions are two-fold: First, we design a 2D-to-3D transformation pipeline that takes as input the point cloud data from LiDAR and 2D bounding boxes from camera that are captured at exactly the same time, and generate 3D bounding boxes efficiently and accurately based on detection results of the previous frames without running 3D detectors. Second, we design a frame offloading scheduler that dynamically launches a 3D detection when the error of 2D-to-3D transformation accumulates to a certain level, so the subsequent transformations can draw upon the latest 3D detection results with better accuracy. Extensive evaluation on NVIDIA Jetson TX2 with the autonomous driving dataset KITTI and real-world 4G/LTE traces shows that, Moby reduces the end-to-end latency by up to 91.9% with mild accuracy drop compared to baselines. Further, Moby shows excellent energy efficiency by saving power consumption and memory footprint up to 75.7% and 48.1%, respectively. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing systems and tools; • Computer systems organization → Robotics.

show abstract

Section: Methodsmentioning

confidence: 99%

2D-Empowered 3D Object Detection on the Edge

Li¹,

Cai²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Its data can be represented as 2D maps and processed by Convolution Neural Networks for object detection [104], [105], segmentation [102], and classification [8]. Alternatively, radar data can also be represented as point clouds [84], [100].…”

Section: Sensor Fusionmentioning

confidence: 99%

“…Because of its advantages, camera-radar fusion is receiving increasing attention in the literature. Many works already consider this multi-modal perception in detection tasks [10]- [13], [84]- [91], while few of them apply the sensor fusion approach in segmentation tasks [9]. Therefore, there is plenty of room for exploring camera-radar fusion in vehicular perception.…”

Section: Sensor Fusionmentioning

confidence: 99%

Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics

Manfio¹,

Osório²

2023

Preprint

View full text Add to dashboard Cite

One of the main paths towards the reduction of traffic accidents is the increase in vehicle safety through driver assistance systems or even systems with a complete level of autonomy. In these types of systems, tasks such as obstacle detection and segmentation, especially the Deep Learning-based ones, play a fundamental role in scene understanding for correct and safe navigation. Besides that, the wide variety of sensors in vehicles nowadays provides a rich set of alternatives for improvement in the robustness of perception in challenging situations, such as navigation under lighting and weather adverse conditions. Despite the current focus given to the subject, the literature lacks studies on radar-based and radar-camera fusionbased perception. Hence, this work aims to carry out a study on the current scenario of camera and radar-based perception for ADAS and autonomous vehicles. Concepts and characteristics related to both sensors, as well as to their fusion, are presented. Additionally, we give an overview of the Deep Learning-based detection and segmentation tasks, and the main datasets, metrics, challenges, and open questions in vehicle perception.

show abstract

“…Recently, millimeter wave (mmWave) radars emerged as a lowcost sensing modality and have been adopted for human detection and tracking [38,42]. The following features of mmWave radars form a good basis for achieving accurate interpersonal distance tracking.…”

Section: Introductionmentioning

confidence: 99%

“…The design of ImmTrack needs to address the following two challenges. First, the point cloud from the radar is usually sparse and noisy [38], making it difficult to separate and track multiple users during their close contacts. Second, as radar and IMU capture different aspects of movements, the cross-modality matching is non-trivial.…”

Section: Introductionmentioning

confidence: 99%

Interpersonal Distance Tracking with mmWave Radar and IMUs

Dai¹,

Shuai²,

Tan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Tracking interpersonal distances is essential for real-time social distancing management and ex-post contact tracing to prevent spreads of contagious diseases. Bluetooth neighbor discovery has been employed for such purposes in combating COVID-19, but does not provide satisfactory spatiotemporal resolutions. This paper presents ImmTrack, a system that uses a millimeter wave radar and exploits the inertial measurement data from user-carried smartphones or wearables to track interpersonal distances. By matching the movement traces reconstructed from the radar and inertial data, the pseudo identities of the inertial data can be transferred to the radar sensing results in the global coordinate system. The re-identified, radar-sensed movement trajectories are then used to track interpersonal distances. In a broader sense, ImmTrack is the first system that fuses data from millimeter wave radar and inertial measurement units for simultaneous user tracking and re-identification. Evaluation with up to 27 people in various indoor/outdoor environments shows ImmTrack's decimeters-seconds spatiotemporal accuracy in contact tracing, which is similar to that of the privacyintrusive camera surveillance and significantly outperforms the Bluetooth neighbor discovery approach. CCS CONCEPTS• Human-centered computing → Ambient intelligence; Mobile computing; • Computer systems organization → Sensor networks.

show abstract

milliEye

Cited by 49 publications

References 31 publications

2D-Empowered 3D Object Detection on the Edge

2D-Empowered 3D Object Detection on the Edge

Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics

Interpersonal Distance Tracking with mmWave Radar and IMUs

Contact Info

Product

Resources

About