EVA²: Exploiting Temporal Redundancy in Live Computer Vision

Buckler, Mark; Bedoukian, Philip; Jayasuriya, Suren; Sampson, Adrian

doi:10.1109/isca.2018.00051

Cited by 62 publications

(36 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motion-based Algorithms Our ISM algorithm shares a similar key observation as some recent motion-based vision algorithms such as EVA 2 [14] and Euphrates [78], in that correlation across frames in a video stream can be used to simplify continuous vision tasks. Euphrates [78] focuses on computing regions-of-interest (ROIs) in object detection and tracking tasks.…”

Section: Related Workmentioning

confidence: 98%

See 1 more Smart Citation

Asv

Whatmough

Zhu

2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

Estimating depth from stereo vision cameras, i.e., "depth from stereo", is critical to emerging intelligent applications deployed in energy-and performance-constrained devices, such as augmented reality headsets and mobile autonomous robots. While existing stereo vision systems make trade-offs between accuracy, performance and energy-efficiency, we describe ASV, an accelerated stereo vision system that simultaneously improves both performance and energy-efficiency while achieving high accuracy.The key to ASV is to exploit unique characteristics inherent to stereo vision, and apply stereo-specific optimizations, both algorithmically and computationally. We make two contributions. Firstly, we propose a new stereo algorithm, invariant-based stereo matching (ISM), that achieves significant speedup while retaining high accuracy. The algorithm combines classic "hand-crafted" stereo algorithms with recent developments in Deep Neural Networks (DNNs), by leveraging the correspondence invariant unique to stereo vision systems. Secondly, we observe that the bottleneck of the ISM algorithm is the DNN inference, and in particular the deconvolution operations that introduce massive compute-inefficiencies. We propose a set of software optimizations that mitigate these inefficiencies. We show that with less than 0.5% hardware area overhead, these algorithmic and computational optimizations can be effectively integrated within a conventional DNN accelerator. Overall, ASV achieves 5× speedup and 85% energy saving with 0.02% accuracy loss compared to today's DNN-based stereo vision systems.

show abstract

Section: Related Workmentioning

confidence: 98%

“…In ASV, the sequencer also chooses key frames. Although complex adaptive schemes are feasible [14,78], we found that a simple strategy to statically set the key-frame window suffices (Sec. 7.2).…”

Section: Hardware Architecturementioning

confidence: 99%

Asv

Whatmough

Zhu

2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

show abstract

“…Recent work [8,82] demonstrates the utility of simulation frameworks for edge computing on drones; cote is an analogous utility for the orbital edge. Machine inference accelerators [14,15,24,33] could significantly shorten full-coverage CNP pipeline depths, although some that rely on temporal data redundancy [10] may have limited benefit for devices capturing images at the GTFR.…”

Section: Related Workmentioning

confidence: 99%

Orbital Edge Computing

Denby

Lucia

2020

Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Syste

162

View full text Add to dashboard Cite

Advances in nanosatellite technology and a declining cost of access to space have fostered an emergence of large constellations of sensor-equipped satellites in low-Earth orbit. Many of these satellite systems operate under a "bent-pipe" architecture, in which ground stations send commands to orbit and satellites reply with raw data. In this work, we observe that a bent-pipe architecture for Earth-observing satellites breaks down as constellation population increases. Communication is limited by the physical configuration and constraints of the system over time, such as ground station location, nanosatellite antenna size, and energy harvested on orbit. We show quantitatively that nanosatellite constellation capabilities are determined by physical system constraints.We propose an Orbital Edge Computing (OEC) architecture to address the limitations of a bent-pipe architecture. OEC supports edge computing at each camera-equipped nanosatellite so that sensed data may be processed locally when downlinking is not possible. In order to address edge processing latencies, OEC systems organize satellite constellations into computational pipelines. These pipelines parallelize both data collection and data processing based on geographic location and without the need for cross-link coordination. OEC satellites explicitly model constraints of the physical environment via a runtime service. This service uses orbit parameters, physical models, and ground station positions to trigger data collection, predict energy availability, and prepare for communication. We show that an OEC architecture can reduce ground infrastructure over 24× compared to a bent-pipe architecture, and we show that pipelines can reduce system edge processing latency over 617×.Computational nanosatellite architects face three key challenges. First, physical constraints on nanosatellite design (e.g.

show abstract

“…To reduce the frame latency, prior work has mostly focused on improving the performance of the vision computation stage, i.e., the back-end of the vision pipeline. Back-end optimization techniques include designing more efficient vision algorithms (e.g., simplified Deep Neural Network (DNN) models [16,35,52,67], compressing/pruning DNN models [29,31]) andleveraging motion information [10,22,71]. Back-end optimizations are effective when the back-end dominates the frame latency.…”

Section: Limitations Of Optimizing Only Vision Algorithmsmentioning

confidence: 99%

Low-Latency Proactive Continuous Vision

Gan

Qiu

Chen

et al. 2020

Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

Continuous vision is the cornerstone of a diverse range of intelligent applications found on emerging computing platforms such as autonomous machines and Augmented Reality glasses. A critical issue in today's continuous vision systems is their long end-to-end frame latency, which significantly impacts the system agility and user experience. We find that the long latency is fundamentally caused by the serialized execution model of today's continuous vision pipeline, whose key stages, including sensing, imaging, and vision computations, execute sequentially, leading to long frame latency. This paper seeks to reduce the end-to-end latency of continuous vision tasks. Our key idea is a new proactive vision execution model that breaks the sequential execution of the vision pipeline. Specifically, we propose to allow the pipeline front-end (sensing and imaging) to predict future frames; the pipeline back-end (vision algorithms) then predictively operates on the future frames to reduce frame latency. While the proactive execution model is generally applicable to any vision systems, we demonstrate its effectiveness using an implementation on resource-constrained mobile Systemson-a-chips (SoC). Our system, PVF, incorporates two techniques to overcome key challenges that arise in deploying proactive vision in mobile systems: it enables multiple outstanding speculative frames by exploiting the hardware heterogeneities in mobile SoCs; it reduces the energy overhead of prediction by exploiting the error-resilient nature of vision algorithms. We show that PVF reduces the frame latency by up to 92% under the same energy. CCS Concepts • Computer systems organization → Heterogeneous (hybrid) systems; Special purpose systems.

show abstract

EVA²: Exploiting Temporal Redundancy in Live Computer Vision

Cited by 62 publications

References 58 publications

Asv

Asv

Orbital Edge Computing

Low-Latency Proactive Continuous Vision

Contact Info

Product

Resources

About