Developing Low-Power Image Processing Applications with the TULIPP Reference Platform Instance

Kalb, Tobias; Kalms, Lester; Göhringer, Diana; Pons, Carlota; Muddukrishna, Ananya; Jahre, Magnus; Ruf, Boitumelo; Schuchert, Tobias; Tchouchenkov, Igor; Ehrenstråhle, Carl; Peterson, Magnus; Christensen, Flemming; Paolillo, Antonio; Rodriguez, Ben; Millet, Philippe

doi:10.1007/978-3-319-92792-3_10

Cited by 4 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, the use of FPGAs for real-time embedded image processing involves a cumbersome and time-consuming development, optimization and deployment process. In order to reduce development costs of such systems, substantial effort is done to enhance the process of high-level synthesis (HLS) and, in turn, alleviate the development of algorithms for FPGAs with more high-level languages such as C/C++ [18][19][20].…”

Section: Embedded Stereo Processing On Fpgasmentioning

confidence: 99%

ReS2tAC -- UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices

Ruf,

Mohrs,

Weinmann

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

With the emergence of low-cost robotic systems, such as unmanned aerial vehicle, the importance of embedded high-performance image processing increased more and more. For a long time, FPGAs were the only processing hardware, that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded systems. However, the recently increasing availability of embedded GPU-based systems, such as the NVIDIA Jetson series, comprised of an ARM CPU and a NVIDIA Tegra GPU, allows for massively parallel embedded computing on graphics hardware. With this in mind, we propose an approach for real-time embedded stereo processing on ARM and CUDA enabled devices, which is based on the popular and widely used Semi-Global Matching algorithm. In this, we propose an optimization of the algorithm for embedded CUDA GPUs, by utilizing massively parallel computing, as well as utilizing the NEON intrinsics to optimize the algorithm for vectorized SIMD processing on embedded ARM CPUs. We have evaluated our approach with different configurations on two public stereo benchmark datasets to demonstrate that they can reach an error rate as low as 3.3 %. Furthermore, our experiments show that the fastest configuration of our approach reaches up to 46 FPS on VGA image resolution. Finally, in a use-case specific qualitative evaluation, we have evaluated the power consumption of our approach and deployed it on the DJI Manifold 2-G attached to a DJI Matrice 210v2 RTK unmanned aerial vehicle (UAV), demonstrating its suitability for real-time stereo processing onboard a UAV.

show abstract

Section: Embedded Stereo Processing On Fpgasmentioning

confidence: 99%

ReS2tAC -- UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices

Ruf,

Mohrs,

Weinmann

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Figure 1 shows the system architecture of the VCS-1 board which is based on the Xilinx UltraScale+ multi-processor systems-on-chip (MPSoC) hardware of the PS and PL ends [24]. The Lynsyn board has a built-in microprogrammed control unit (MCU) [25] and FPGA [26], which work together with the host computer to operate the Tulipp profiling tools. The Lynsyn board based on the MCU and FPGA samples up to 1 kilo sample per second (kS/s) on the current sensors, and the PMU-RL algorithm analyses the power information to provide feedback for the optimization action in order to control the PL end with profiling tools.…”

Section: System Modelmentioning

confidence: 99%

Energy and Performance Trade-Off Optimization in Heterogeneous Computing via Reinforcement Learning

Machado

Zahid

et al. 2020

Electronics

View full text Add to dashboard Cite

This paper suggests an optimisation approach in heterogeneous computing systems to balance energy power consumption and efficiency. The work proposes a power measurement utility for a reinforcement learning (PMU-RL) algorithm to dynamically adjust the resource utilisation of heterogeneous platforms in order to minimise power consumption. A reinforcement learning (RL) technique is applied to analyse and optimise the resource utilisation of field programmable gate array (FPGA) control state capabilities, which is built for a simulation environment with a Xilinx ZYNQ multi-processor systems-on-chip (MPSoC) board. In this study, the balance operation mode for improving power consumption and performance is established to dynamically change the programmable logic (PL) end work state. It is based on an RL algorithm that can quickly discover the optimization effect of PL on different workloads to improve energy efficiency. The results demonstrate a substantial reduction of 18% in energy consumption without affecting the application’s performance. Thus, the proposed PMU-RL technique has the potential to be considered for other heterogeneous computing platforms.

show abstract

“…The strict requirements of ADAS systems pushes technology development with respect to hardware, software, and algorithms. In the T project, we focused on implementing pedestrian detection using Viola-Jones classifiers on the heterogeneous T hardware platform using a High-Level Synthesis (HLS) [6,11]. The key result was that we were able to implement a near-real-time pedestrian detection system with a fraction of the manpower required to implement similar systems using a traditional Register Transfer Level (RTL) approach.…”

Section: The T Automotive Use Casementioning

confidence: 99%

Challenges in the Realm of Embedded Real-Time Image Processing

Millet

Grinberg

Jahre

2020

Towards Ubiquitous Low-Power Image Processing Platforms

Self Cite

View full text Add to dashboard Cite

The development of power-efficient solutions gives new embedded products the ability to analyse images and thereby brings more intelligence to embedded systems -providing more and better services of higher quality as well as advanced capabilities such as self-adaptation and autonomy. This will allow cars to drive safer, medical devices to assist surgeons, and autonomous drones to find people that have gotten lost. For small-series products, one needs to find an embedded platform that provides enough performance, does not exceed the target price, and has sufficiently low power consumption. As these requirements are typically conflicting, image processing engineers spend considerable time identifying the best possible trade-off for their algorithm implementation on the chosen platform. Providing a common platform that allows the efficient implementation of image processing systems across diverse application domains -a key objective of our T project -requires a solid understanding of the constraints and challenges of each domain. In this paper, we report the key challenges we identified within the medical, Unmanned Aerial Vehicle (UAV), and automotive domains to aid the community in developing the next generation of embedded image processing systems.

show abstract

Developing Low-Power Image Processing Applications with the TULIPP Reference Platform Instance

Cited by 4 publications

References 9 publications

ReS2tAC -- UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices

ReS2tAC -- UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices

Energy and Performance Trade-Off Optimization in Heterogeneous Computing via Reinforcement Learning

Challenges in the Realm of Embedded Real-Time Image Processing

Contact Info

Product

Resources

About